System

Abstract

A system is provided.SOLUTION: A system comprising: means for measuring ambient temperature with a temperature sensor; means for measuring water pressure with a water pressure sensor; means for measuring ambient sound with a sound sensor; means for generating a warning signal when the measured temperature exceeds a certain range; means for generating a warning signal when the measured water pressure exceeds a certain range; and means for transmitting the generated warning signal to an administrator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While conventional GPS devices are effective at tracking location information, they have the problem of being unable to detect environmental changes and potential dangers in real time and respond immediately. For example, they lack the functionality to capture environmental information such as sudden changes in temperature, abnormal increases in water pressure, or the generation of dangerous sounds, and issue appropriate warnings based on that information. This has resulted in cases where managers are slow to detect danger, making it difficult to respond quickly. There is a need to improve safety by solving these issues. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system that measures ambient temperature, water pressure, and sound in real time, generates a warning signal if these values ​​exceed a set range, and has means for quickly sending the warning signal to an administrator. The system of the present invention is equipped with a temperature sensor, a water pressure sensor, and a sound sensor, and acquires data from these sensors at regular intervals. If the data acquired from the sensors exceeds a set threshold, a warning signal is generated and sent to an administrator, allowing the administrator to take immediate action. The system also has means for notifying the administrator based on the warning signal sent to them. This can significantly improve safety.

[0006] A "temperature sensor" is a device for measuring the temperature of an environment and providing that data.

[0007] A "water pressure sensor" is a device that measures the surrounding water pressure and provides that data.

[0008] A "sound sensor" is a device that measures surrounding sounds and provides the data.

[0009] A "warning signal" is a signal that is generated to notify of danger when data measured by a sensor exceeds a set threshold.

[0010] "Administrator" means a person or entity responsible for operating and monitoring the system, receiving warning signals, and taking appropriate action.

[0011] A "regular interval" is a set period of time for acquiring data from a sensor.

[0012] A "threshold" is a reference value for determining whether the data measured by the sensor has exceeded the normal range.

[0013] "Notification" is an action to notify an administrator based on a warning signal.

[0014] "Real-time" means processing or responding immediately in real time.

[0015] A "system" is a collection of integrated devices or programs in which multiple components work together to achieve a specific purpose. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. This system operates in cooperation with the terminal, server, and user.

[0038] Device behavior

[0039] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). For each sensor, it sets its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The device acquires data from each sensor at regular intervals (e.g., every second). For example, it reads 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0040] The device then verifies the acquired data and generates a warning signal if the sensor value exceeds the set threshold. For the temperature sensor, the threshold is below -5°C or above 40°C. For the water pressure sensor, the threshold is above 7, and for the sound sensor, the threshold is a sound level above 85dB.

[0041] Once a warning signal is generated, the terminal sends the warning signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" is generated and sent to the server.

[0042] Server Operation

[0043] The server receives the alert signal from the device, analyzes the received message, and takes appropriate action based on its content. For example, it may send a notification to an administrator based on the type and location of the alert. Notifications are typically sent via email or a dedicated application.

[0044] The server also logs the warning signals it receives and stores them in a database for later analysis, which can be used to analyze long-term trends and take measures to prevent recurrence.

[0045] User response

[0046] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take necessary safety measures.

[0047] Specific examples

[0048] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety can be ensured by the on-site staff taking measures such as turning on the cooling equipment.

[0049] In this way, the system of the present invention improves on-site safety by detecting environmental abnormalities in real time and responding quickly.

[0050] The processing flow will be explained below.

[0051] Step 1:

[0052] The device initializes the sensor.

[0053] When the device starts up, it initializes the temperature sensor, water pressure sensor, and sound sensor.

[0054] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0055] Step 2:

[0056] The device acquires data from the sensors.

[0057] Obtain data from each sensor at regular intervals (e.g., every second).

[0058] For example, obtain data of 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0059] Step 3:

[0060] The terminal verifies the acquired data.

[0061] Check whether the temperature, water pressure, and sound sensor values ​​exceed the set thresholds.

[0062] For example, if the temperature is 45°C, it will detect that this exceeds the threshold of 40°C.

[0063] Step 4:

[0064] The terminal generates an alert signal.

[0065] If the data exceeds a set threshold, a warning signal is generated.

[0066] For example, generate the message "Temperature warning: 45°C at Site-A".

[0067] Step 5:

[0068] The terminal sends an alert signal to the server.

[0069] The generated alert signal is sent to the server.

[0070] For example, send the message "Temperature warning: 45°C at Site-A" to the server.

[0071] Step 6:

[0072] The server receives the alert signal.

[0073] Analyze the warning signal received from the device.

[0074] Check the message content to determine what type of warning it is.

[0075] Step 7:

[0076] The server takes the appropriate action.

[0077] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0078] For example, send a notification to the administrator saying "Temperature warning: 45°C at Site-A".

[0079] Step 8:

[0080] The server logs the warning signal.

[0081] The contents of the received warning signals are logged and stored in a database.

[0082] This allows for later analysis and reporting.

[0083] Step 9:

[0084] The user receives a notification.

[0085] The administrator receives notifications from the server.

[0086] For example, an administrator sees the notification "Temperature warning: 45°C at Site-A."

[0087] Step 10:

[0088] The user reassesss the situation on the ground and responds.

[0089] Check the situation on-site and take necessary measures.

[0090] For example, a manager may dispatch on-site staff and instruct them to take measures such as operating cooling equipment.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In on-site environmental monitoring, there is a need for a system that can quickly detect abnormalities in temperature, pressure, and sound and notify the appropriate parties. However, conventional systems often have problems with real-time response and accuracy, and are often insufficient to respond to emergency situations. This can lead to delays in appropriate responses to ensure on-site safety.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring pressure with a pressure sensor, means for measuring the ambient sound with a sound sensor, means for generating a warning signal when the measured temperature exceeds a certain range, means for generating a warning signal when the measured pressure exceeds a certain range, means for generating a warning signal when the measured sound exceeds a certain range, means for transmitting the generated warning signals to a remote server, and means for the remote server to analyze the warning signals and notify an appropriate administrator. This makes it possible to detect abnormalities in real time and quickly notify the administrator.

[0096] A "temperature sensor" is a device for measuring the ambient temperature.

[0097] A "pressure sensor" is a device for measuring ambient pressure.

[0098] A "sound sensor" is a device for measuring surrounding sounds.

[0099] "Certain range" refers to a predetermined safe tolerance range for temperature, pressure, and sound.

[0100] A "warning signal" is a signal that indicates an abnormality and is generated when a value measured by a sensor exceeds a certain range.

[0101] A "remote server" is a central control unit that receives and analyzes data sent by sensors and takes appropriate action.

[0102] "Analysis" is the process of examining the content of received data in detail and interpreting its meaning and intent.

[0103] "Administrator" refers to the person or organization responsible for receiving system alert notifications and taking appropriate action.

[0104] "Notification" is the act or means of communicating information to a specific recipient.

[0105] "Real-time" means that data is processed as soon as it is generated.

[0106] The system according to the present invention uses temperature, pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify an administrator when a certain threshold is exceeded. This system operates in cooperation with terminals, a server, and users.

[0107] Device behavior

[0108] When the system starts up, the device initializes the temperature sensor, pressure sensor, and sound sensor. Initialization involves setting the type of each sensor (temperature, pressure, sound) and initial measurement value (e.g., 0). Specifically, the temperature sensor measures the ambient temperature, the pressure sensor measures pressure, and the sound sensor measures the ambient sound. After initialization, the device acquires data from each sensor at regular intervals (e.g., every second). The acquired data is measured as specific examples, such as 25°C for the temperature sensor, 5 for the pressure sensor, and 60dB for the sound sensor.

[0109] Next, the terminal verifies the acquired data. This is the process of checking whether the value of each sensor exceeds the set threshold (temperature: below -5°C or above 40°C, pressure: above 7, sound: above 85dB). If the threshold is exceeded, the terminal generates a warning signal. For example, if the temperature sensor value exceeds 45°C, the terminal generates a warning signal saying "Temperature warning: 45°C at Site-A."

[0110] The generated warning signal is then sent to the server, ensuring that a remote administrator is promptly notified of any abnormalities.

[0111] Server Operation

[0112] The server receives the warning signal sent from the terminal. After receiving it, the server analyzes the contents of the warning message and identifies the type of abnormality and its location. Based on the results of this analysis, the server sends a notification to the appropriate administrator. Notifications are generally sent via email or a dedicated application. For example, a notification stating "Temperature warning: 45°C at Site-A" may be sent to the administrator.

[0113] Additionally, the server logs the received warning signals and stores them in a database, which can later be used as a reference for long-term analysis and for developing measures to prevent recurrence.

[0114] User response

[0115] The user receives a notification from the server. They check the notification and take prompt action to reassess the situation at the site. For example, if they receive a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take safety measures such as turning on cooling equipment. This ensures the safety of the site.

[0116] Specific examples

[0117] As an example, consider safety management at a construction site. A terminal installed at a construction site is equipped with a temperature sensor, a pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety is ensured when the on-site staff activates the cooling equipment.

[0118] Prompt Sentence Examples

[0119] An example prompt for generating an explanation for this system using a generative AI model might look something like this:

[0120] "Describe a system that uses temperature, pressure, and sound sensors to detect environmental abnormalities in real time, and generates a warning signal to notify an administrator. Please generate a detailed description of how the terminal, server, and user work together."

[0121] The above is an embodiment of the present invention. The present invention is a system that improves safety at a work site by detecting environmental abnormalities in real time and quickly notifying a manager.

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] System program processing flow

[0124] Terminal handling

[0125] Step 1:

[0126] The device initializes the temperature sensor, pressure sensor, and sound sensor at system startup. During initialization, the type of each sensor (temperature, pressure, sound) and the initial measurement value (e.g., 0) are set. The sensor object is provided as input, and the output sets the initial value of each sensor.

[0127] Step 2:

[0128] The device acquires data from each sensor at regular intervals (e.g., every second). It acquires the current measurement environment value of each sensor as input and generates a data list like the one below as output. For example, data of 25°C is obtained from the temperature sensor, 5 from the pressure sensor, and 60dB from the sound sensor.

[0129] Step 3:

[0130] The terminal verifies the acquired sensor data. It receives the sensor measurement data as input and checks whether each value exceeds the set threshold. As output, if an abnormal value is detected, a warning signal is generated. For example, if the temperature sensor value records 45°C, this value exceeds the threshold of 40°C, so a warning signal is generated.

[0131] Step 4:

[0132] The terminal generates a warning signal. It receives sensor data exceeding a threshold as input and generates a warning signal message as output. For example, a warning signal saying "Temperature warning: 45°C at Site-A" is generated.

[0133] Step 5:

[0134] The terminal that generated the alert signal sends the signal to the server. As input, it takes the generated alert signal message and sends it to the server. As output, it confirms that the communication process of sending the alert signal to the server was successful.

[0135] Server Processing

[0136] Step 1:

[0137] The server receives the warning signal sent from the terminal. It receives the warning signal message from the terminal as input and confirms the receipt as output.

[0138] Step 2:

[0139] The server analyzes the received warning message. It takes the warning signal message as input and analyzes its content (e.g., warning type, location). It gets the analysis result as output and determines the appropriate response.

[0140] Step 3:

[0141] The server sends notifications to the appropriate administrators based on the analysis results. It takes the warning analysis results as input, generates and sends a notification message, and outputs a message such as "Temperature warning: 45°C at Site-A" via email or a dedicated application.

[0142] Step 4:

[0143] The server logs the received warning signals and stores them in a database. It takes the content of the warning signal as input and generates an entry to store in the database. As output, a database entry is created for long-term analysis and to take measures to prevent recurrence.

[0144] User response

[0145] Step 1:

[0146] The user receives notifications from the server. As input, the user gets the notification message sent from the server, and as output, the user checks its contents.

[0147] Step 2:

[0148] The user checks the notification content and takes prompt action to reassess the situation at the site. The user takes the notification message (e.g., "Temperature warning: 45°C at Site-A") as input and determines the on-site response action as output.

[0149] Step 3:

[0150] The user contacts the on-site staff and takes necessary safety measures. The input is the notification content, and the output is the on-site staff taking specific measures (e.g., turning on the cooling equipment).

[0151] These are the specific processing steps of this system. Each processing step clearly defines the input, data processing, calculation, and output, and the specific operations at each step are explained in detail. This allows you to understand how each component works together to fulfill the functions of the system as a whole.

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Environmental monitoring within factories is an important element in ensuring safety, but it is difficult to detect abnormalities in real time using a wide variety of sensors and take immediate appropriate action. With conventional systems, there are cases where the response after detecting an abnormality is delayed, which can lead to dangerous situations. In particular, when multiple abnormalities occur simultaneously, it is difficult to keep up with the response.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes means for analyzing the received warning signal and taking appropriate action based on the content of the signal, means for transmitting the generated warning signal to an administrator, and means for notifying the administrator, thereby enabling rapid analysis and notification when an abnormality is detected, allowing the administrator to take immediate action.

[0157] A "temperature sensor" is a device that measures the ambient temperature and transmits that data.

[0158] A "water pressure sensor" is a device that measures the water pressure in pipes or tanks and transmits that data.

[0159] A "sound sensor" is a device that measures the volume of the surrounding sound and transmits that data.

[0160] A "warning signal" is an alert message that is generated when a set threshold is exceeded.

[0161] The "server" is a central system that receives data from sensors and manages analysis and warning signals.

[0162] An "administrator" is a person responsible for monitoring the system and responding to any abnormalities that occur.

[0163] "Analysis" is the process of determining appropriate action based on received data and warning signals.

[0164] "Notification" is the act of transmitting abnormality or warning information to an administrator.

[0165] This invention provides a system that monitors the factory environment in real time and responds quickly if an abnormality is detected. The system consists of a factory robot equipped with temperature, water pressure, and sound sensors, a server that analyzes and notifies data, and an administrator's smartphone or tablet.

[0166] Hardware and software used

[0167] Factory robot: Equipped with multiple sensors (temperature, water pressure, sound sensors).

[0168] Server: Receives data, analyzes it, and notifies administrators. Specific server software used includes the Flask framework and an SMTP server.

[0169] Smartphone or tablet: A device used by the administrator to receive alert notifications through a dedicated application.

[0170] Data collection and analysis process

[0171] First, the terminal (factory robot) initializes each sensor when it starts up. The sensors acquire data every second, for example, 45°C from the temperature sensor, 8 from the water pressure sensor, and 90dB from the water sensor. These data are compared with certain thresholds, and if they exceed the threshold, a warning signal is generated.

[0172] Server Roles

[0173] The server receives the generated warning signal and analyzes its contents. For example, if a warning signal such as "Temperature warning: 45°C at Site-A" is received, it immediately takes action based on the contents. Based on the analysis results, a notification is sent to the administrator via email or a dedicated application.

[0174] Administrator response

[0175] The administrator receives the notification from the server and responds promptly. After checking the contents of the notification, they dispatch staff to the site and take safety measures, such as turning on cooling equipment.

[0176] Specific examples

[0177] For example, suppose a factory uses a factory robot equipped with a temperature sensor, a water pressure sensor, and a sound sensor. The robot collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator receives the notification and instructs the field staff to check the situation and take necessary action. For example, the following warning signal is generated on the server: "Temperature warning: 45°C reached at Site-A."

[0178] Prompt Sentence Examples

[0179] "Generate alerts based on sensor values ​​and notify administrators. Create a mechanism to send an alert signal to the server if the temperature exceeds 40 degrees, the water pressure exceeds 7, or the sound level exceeds 85dB."

[0180] In this way, the system can detect abnormalities in real time and respond quickly to improve safety within the factory.

[0181] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0182] Step 1:

[0183] When the device starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). The type of each sensor (temperature, water pressure, sound) and the initial measurement value (e.g., 0) are set as input, and the sensor's ready status is obtained as output. Specific operations include calibrating the sensors and setting their initial values.

[0184] Step 2:

[0185] The device acquires data from each sensor at regular intervals (e.g., every second). The input here is raw data from the sensor, and the output is a dataset of measured temperature, pressure, and sound. Specifically, it follows the data collection protocol of each sensor and acquires temperature (e.g., water temperature) from the temperature sensor, pressure from the water pressure sensor, and ambient sound data from the sound sensor.

[0186] Step 3:

[0187] The terminal verifies the acquired data and generates an alert signal if the threshold is exceeded. The input here is the data set obtained from the sensor and the threshold, and the output is the alert signal generated if the threshold is exceeded. Specifically, it checks whether the temperature sensor data is below -5°C or above 40°C, whether the water pressure sensor data is above 7, and whether the sound sensor data is above 85dB, and generates an alert signal if the conditions are met.

[0188] Step 4:

[0189] The terminal sends the generated warning signal to the server. The input is the generated warning signal, and the output is the warning message sent to the server. As a specific operation, the terminal sends the warning message (e.g., "Temperature warning: 45°C at Site-A") to the server using an HTTP request or other communication means.

[0190] Step 5:

[0191] The server receives an alert signal from the terminal. The input is the alert signal sent from the terminal, and the output is the received alert data. Specifically, the server listens for communication at a specific endpoint and analyzes the data appropriately when it receives an alert signal.

[0192] Step 6:

[0193] The server analyzes and takes appropriate action based on the content of the received warning signal. The input is the received warning data, and the output is the analysis result and notification action. Specifically, the server processes the type of warning (temperature, water pressure, sound) and location information to send a notification to the appropriate administrator.

[0194] Step 7:

[0195] The server sends a notification to the administrator. The input is the analyzed warning data, and the output is a notification to the administrator's device. Specifically, the server sends an email or generates a push notification to convey the information to the administrator.

[0196] Step 8:

[0197] The administrator receives the notification from the server and takes action. The input is the notification message from the server, and the output is the start of the action. Specifically, the administrator checks the notification content, quickly reassessss the situation on site, and instructs the necessary safety measures. For example, specific actions are taken, such as turning on cooling equipment or dispatching on-site staff.

[0198] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0199] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, a more advanced safety management system can be achieved.

[0200] Device behavior

[0201] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor) and the emotion engine. Each sensor is set to its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The emotion engine performs initial settings to analyze voice and facial expression data.

[0202] The device acquires data from each sensor at regular intervals (e.g., every second) and also collects the user's voice and facial expression data. For example, the device acquires 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and also collects the user's voice and facial expression data.

[0203] Emotion Engine Operation

[0204] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state. For example, it analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. If the user's emotional state is abnormal (e.g., extreme stress or anger), the emotion engine detects it and generates a warning signal.

[0205] Validating data and generating warning signals

[0206] The device verifies the data obtained from each sensor and the judgment results from the emotion engine. It checks whether the sensor values ​​for temperature, water pressure, and sound exceed the set thresholds and whether the emotion engine detects any abnormalities in the user. For example, if the temperature is 45°C or the emotion engine detects stress, it generates a warning signal.

[0207] Sending to the server and notifying

[0208] Once a warning signal is generated, the terminal sends the signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" is generated and sent to the server.

[0209] Server Operation

[0210] The server receives warning signals from the devices, analyzes the received messages, and takes appropriate action based on their content. For example, it sends notifications to administrators via email or a dedicated application based on the type and location of the warning. It also logs the received warning signals for long-term data analysis.

[0211] User response

[0212] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A," the user can dispatch staff to the site and take the necessary safety measures.

[0213] Specific examples

[0214] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This terminal collects data from each sensor every second, and also collects the user's voice and facial expression data. If the temperature reaches 45°C or the emotion engine determines that the user is in a state of high stress, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, the on-site staff can take measures such as turning on the cooling equipment or instructing stressed staff to take a break, thereby ensuring safety.

[0215] In this way, the system of the present invention detects environmental abnormalities and the user's emotional state in real time and responds quickly, thereby significantly improving safety on site.

[0216] The processing flow will be explained below.

[0217] Step 1:

[0218] The device initializes the sensors and emotion engine.

[0219] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor and emotion engine.

[0220] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0221] The emotion engine performs initial setup to analyze voice and facial expression data.

[0222] Step 2:

[0223] The device receives data from sensors and an emotion engine.

[0224] Data is acquired from each sensor at regular intervals (e.g., every second), and the user's voice and facial expression data are also collected.

[0225] For example, data of 25°C is obtained from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and data on the user's voice and facial expression is also collected.

[0226] Step 3:

[0227] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state.

[0228] It analyzes voice tone and facial expressions to determine whether the user is feeling stressed.

[0229] For example, if the user is determined to be in a high stress state.

[0230] Step 4:

[0231] The device verifies the data from each sensor and the emotion engine.

[0232] Check whether the value of each sensor exceeds the set threshold and whether the emotion engine detects an abnormal emotional state.

[0233] For example, if the temperature is 45°C or if the emotion engine detects stress.

[0234] Step 5:

[0235] The terminal generates an alert signal.

[0236] If the data exceeds a set threshold, a warning signal is generated.

[0237] For example, messages such as "Temperature warning: 45°C at Site-A" and "User stress detected: High stress at Site-A" are generated.

[0238] Step 6:

[0239] The terminal sends an alert signal to the server.

[0240] The generated alert signal is sent to the server.

[0241] For example, messages such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" are sent to the server.

[0242] Step 7:

[0243] The server receives the alert signal.

[0244] Analyze the warning signal received from the device.

[0245] Check the message content to determine what type of warning it is.

[0246] Step 8:

[0247] The server takes the appropriate action.

[0248] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0249] For example, send notifications to the administrator such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A".

[0250] Step 9:

[0251] The server logs the warning signal.

[0252] The contents of the received warning signals are logged and stored in a database.

[0253] This allows for later analysis and reporting.

[0254] Step 10:

[0255] The user receives a notification.

[0256] The administrator receives notifications from the server.

[0257] For example, an administrator sees notifications such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A."

[0258] Step 11:

[0259] The user reassesss the situation on the ground and responds.

[0260] Check the situation on-site and take necessary measures.

[0261] For example, a manager might dispatch on-site staff to run cooling equipment or give stressed staff a break.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] Conventional safety management systems only monitor ambient temperature, water pressure, and sound, generating a warning signal if any of these values ​​exceed a certain range. While these systems are effective at detecting abnormalities in the physical environment, they are unable to consider the user's emotional state or psychological stress, making them unable to implement comprehensive safety measures. To ensure safety, including when on-site workers are under high stress, it is important to recognize their emotional state and issue a warning if there is a problem.

[0265] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for measuring the ambient temperature using a temperature sensor, a means for measuring the water pressure using a water pressure sensor, a means for measuring the ambient sound using a sound sensor, and an emotion analysis engine as a means for recognizing the user's emotion. This enables comprehensive environmental monitoring including the user's emotional state.

[0266] A "temperature sensor" is a sensor that measures the ambient temperature and is a device for acquiring environmental temperature data.

[0267] A "water pressure sensor" is a sensor that measures the surrounding water pressure and is a device for obtaining liquid pressure data.

[0268] A "sound sensor" is a sensor that measures surrounding sounds and is a device that obtains the volume and frequency of sounds.

[0269] An "emotion analysis engine" is a system for recognizing a user's emotional state, and is a device that analyzes voice and facial expression data to determine the user's psychological state.

[0270] A "warning signal" is a notification signal that is generated when data measured by a sensor exceeds a set threshold or when an emotion analysis engine detects an abnormal emotional state.

[0271] "Administrator" means the person or organization operating and monitoring the system and responsible for receiving and responding to generated alert signals.

[0272] "Means of acquiring data" refers to the functions and methods for collecting information from sensors and sentiment analysis engines.

[0273] The "fixed interval" refers to the time interval at which data collection is performed, and is set to acquire data every second, for example.

[0274] The "notification means" refers to a function and method for sending a warning signal to an administrator, and is a device that completes the notification by using email or a dedicated application.

[0275] The system starts by initializing the device's temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor sets its initial measurement value according to its type (temperature, water pressure, sound), and the emotion analysis engine performs initial settings to analyze voice and facial expression data.

[0276] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, while also collecting the user's voice and facial expression data. For example, the temperature sensor collects data at 25°C, the water pressure sensor collects data at 5, and the sound sensor collects data at 60dB. The emotion analysis engine then analyzes the user's emotional state.

[0277] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. Based on the analysis results, it determines the user's emotional state as "normal" or "abnormal."

[0278] The device generates a warning signal if an abnormality is detected based on data from each sensor and the emotion analysis engine. For example, if the temperature reaches 45°C, the water pressure exceeds a preset threshold, the sound exceeds a threshold, or the emotion analysis engine detects a high level of stress, it will determine this as an abnormality and generate a warning signal.

[0279] When an alert signal is generated, the device sends it to the server. The alert signal is converted into JSON or XML format and sent to the server's specified API endpoint. The server analyzes the received alert signal and takes appropriate action based on its content. For example, it may notify the administrator of the alert content via email or a dedicated application, and record the received alert signal in a database.

[0280] The user receives a notification from the server, checks its contents, contacts the on-site staff to confirm the situation, and takes any necessary measures. For example, if the user receives a notification that reads, "The temperature sensor at the construction site has detected 45°C. Please check the on-site situation and the status of the staff and take any necessary measures," the user can contact the on-site staff and instruct them to operate the cooling equipment or take a break.

[0281] In this way, this system uses temperature sensors, water pressure sensors, sound sensors, and an emotion analysis engine to comprehensively monitor the surrounding environment and the user's emotional state, and can respond quickly if an abnormality is detected. The program can be implemented using programming languages ​​such as C, Python, and Java (registered trademark), and an appropriately selected software library can be used to control the various sensors and emotion analysis engine and analyze data.

[0282] To cite a specific example, in safety management at construction sites, this system collects environmental data and user emotional data every second from devices installed on-site. If the temperature reaches 45°C or if the emotion analysis engine determines a high stress state, it generates a warning signal and sends it to a server. The server analyzes the warning signal and promptly notifies the administrator. The administrator can immediately contact on-site staff and instruct them on appropriate responses, ensuring safety.

[0283] An example of a text prompt is: "The temperature sensor at the construction site has detected a temperature of 45°C. Please check the situation at the site and the status of the staff and take necessary measures."

[0284] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0285] Step 1: Initialize your device

[0286] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor is set to an initial measurement value for its type, and the emotion analysis engine is initialized to analyze voice and facial expression data. Specifically, it starts up the temperature sensor and sets its initial measurement value to 0°C. It starts up the water pressure sensor and sets its initial measurement value to 0. It starts up the sound sensor and sets its initial measurement value to 0dB. It starts up the emotion analysis engine and configures the voice analysis module and facial expression analysis module.

[0287] Input: System start signal

[0288] Output: Initialized sensors and sentiment analysis engine

[0289] Step 2: Collecting data from sensors

[0290] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, and also collects the user's voice and facial expression data. For example, the temperature sensor collects 25°C, the water pressure sensor collects 5, and the sound sensor collects 60dB. In addition, the device captures the user's voice with a microphone and captures their facial expressions with a camera.

[0291] Input: Initialized sensors and sentiment analysis engine

[0292] Output: Collected temperature, water pressure, and sound data, as well as user voice and facial expression data

[0293] Step 3: Analysis by the sentiment analysis engine

[0294] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expression to determine whether the user is feeling stressed. For example, the voice analysis module analyzes the tone, pitch, and speed of the voice, while the facial expression analysis module analyzes changes in facial expression to determine whether the user is feeling "normal" or "abnormal."

[0295] Input: Collected voice and facial expression data

[0296] Output: User's emotional state (normal or abnormal)

[0297] Step 4: Validate data and generate warning signals

[0298] The device generates a warning signal if an abnormality is detected based on data from each sensor and the results of the emotion analysis engine, such as when the temperature reaches 45°C, the water pressure exceeds a set threshold, the sound exceeds a set threshold, or the emotion analysis engine detects a high level of stress.

[0299] Input: Data from each sensor and the results of the emotion analysis engine

[0300] Output: Generated warning signal

[0301] Step 5: Sending a warning signal to the server

[0302] The device sends the generated alert signal to the server. The alert signal is converted into JSON or XML format and sent to the specified API endpoint of the server. Specifically, the device converts the generated alert signal into JSON and sends an HTTP request to the server's API.

[0303] Input: Generated warning signal

[0304] Output: Warning signal sent to the server

[0305] Step 6: Server analyzes and notifies warning signals

[0306] The server analyzes the received warning signal and takes appropriate action based on its content. It notifies the administrator of the warning content via email or a dedicated application and records the received warning signal in a database. For example, the server generates a message such as "Temperature warning: 45°C at Site-A" and notifies the administrator.

[0307] Input: Alert signal sent to the server

[0308] Output: Analyzed warning signals and administrator notifications

[0309] Step 7: User Action

[0310] The user receives a notification from the server, checks the contents, contacts the on-site staff, has them check the situation, and takes necessary measures, such as instructing the on-site staff to operate the cooling equipment or take a break.

[0311] Input: Notification from the server

[0312] Output: On-site staff response and implementation of measures

[0313] (Application example 2)

[0314] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] In business environments such as factories, there is a need to monitor the operation of machines and the work status of workers in real time and respond quickly when abnormalities occur. However, there is currently a lack of systems that can comprehensively manage data from various sensors, such as temperature sensors, water pressure sensors, and sound sensors, as well as the emotional state of workers, making it difficult to improve safety management. Another issue is that it is difficult to detect workers' stress levels and emotional abnormalities in real time and take appropriate action.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0317] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring water pressure with a water pressure sensor, means for measuring ambient sound with a sound sensor, means including an emotion engine for analyzing the user's voice and facial expression data to determine the emotional state, means for generating an alarm signal when the measured temperature exceeds a certain range, means for generating an alarm signal when the measured water pressure exceeds a certain range, means for generating an alarm signal when the measured sound exceeds a certain range, means for generating an alarm signal when the user's emotional state is determined to be abnormal, and means for transmitting the generated alarm signal to a manager. This makes it possible to monitor environmental abnormalities and the emotional state of workers in real time, and to respond quickly when an abnormality occurs.

[0318] A "temperature sensor" is a device that detects the ambient temperature and outputs the measured value.

[0319] A "water pressure sensor" is a device that detects the pressure of a liquid and outputs the measured value.

[0320] A "sound sensor" is a device that detects surrounding sounds and outputs their measured values.

[0321] An "emotion engine" is an algorithm or software that analyzes a user's voice and facial expression data to determine their emotional state.

[0322] A "warning signal" is a warning message that is generated when a measurement exceeds a set threshold.

[0323] "Administrator" means the person or entity responsible for monitoring and managing the System.

[0324] The "regular interval" refers to a predetermined periodic time interval.

[0325] "User's emotional state" refers to the emotional or psychological state that the user is currently experiencing.

[0326] "Notification" means the act or process of communicating warning signals or other important information to management.

[0327] To implement this invention, we first need to build a system that includes a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This system is installed in a factory environment and monitors environmental data and the user's emotional state in real time.

[0328] Natural language description of the program

[0329] The server initializes the temperature sensor (water pressure sensor and sound sensor devices such as DHT11, MPX5010, KY-038, etc.) and emotion engine (such as Amazon's Rekognition). Each sensor acquires the ambient temperature, pressure, and sound level in real time, and the emotion engine analyzes the user's voice and facial expression data.

[0330] This allows the server to continuously collect data from each sensor, and if an abnormality is detected, it will immediately generate a warning signal and notify the administrator. For example, if the temperature sensor detects a temperature above 45°C, or if the emotion engine determines that the user's stress level is high, this information will be sent to the server as a warning signal.

[0331] Upon receiving the warning signal, the server analyzes its contents and takes appropriate action based on the abnormality. For example, it sends a warning message to the administrator via a dedicated notification application. All warning signals are also recorded on the server for later analysis and improvement activities.

[0332] Hardware and software used

[0333] The system uses the following hardware and software:

[0334] Temperature sensors: DHT11, DHT22

[0335] Water pressure sensor: MPX5010, MS5803

[0336] Sound sensor: KY-038, MAX9814

[0337] Emotion engine: Amazon Rekognition, etc.

[0338] Notification system: Twilio, Slack, or a dedicated notification system

[0339] Specific examples

[0340] When a temperature sensor in a factory exceeds 45°C, the server generates a warning signal saying "Temperature warning: 45°C at Site-A" and sends a real-time notification to the administrator via a dedicated notification application. Furthermore, when the emotion engine detects a high user stress level, a warning message saying "User stress detected: High stress at Site-A" is sent to the administrator. In this way, the system supports quick and accurate safety measures.

[0341] Prompt Sentence Examples

[0342] The following prompt sentences can be used by the generative AI model to allow the emotion engine to properly parse the user's emotional state:

[0343] "Analyze the following data to determine the user's emotional state: voice data = some_voice_data, facial expression data = some_expression_data. If the user is stressed, respond with 'high_stress'."

[0344] This allows the emotion engine to determine the user's emotional state in real time based on their voice and facial expression data.

[0345] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0346] Step 1: Initialize your device

[0347] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion engine. Specifically, it sets the types and initial measurement values ​​of these sensors and engines. The input of this step is the initial setting data for the sensors and engines, and the output is the initialized sensors and engines.

[0348] Step 2: Acquiring Sensor Data

[0349] The terminal acquires data from the temperature sensor, water pressure sensor, and sound sensor at regular intervals (e.g., every second). It acquires the temperature, pressure, and sound level measurements and records them in a database. The input of this step is real-time data from each sensor, and the output is the acquired temperature, pressure, and sound level data.

[0350] Step 3: Collecting emotion data

[0351] The device collects the user's voice and facial expression data. Specifically, it acquires voice and image data from input devices such as microphones and cameras and sends them to the emotion engine. The input to this step is the user's voice and facial expression data, and the output is the data to be analyzed that is sent to the emotion engine.

[0352] Step 4: Sentiment Analysis

[0353] The emotion engine analyzes the collected voice and facial expression data to determine the user's emotional state. Specifically, it analyzes changes in voice tone and facial expressions to assess stress levels. The input for this step is voice and facial expression data, and the output is a judgment result regarding the user's emotional state.

[0354] Step 5: Validate the data

[0355] The server verifies the data obtained from each sensor and the judgment result from the emotion engine. It compares the temperature, water pressure, and sound sensor measurements and the emotion engine's judgment result (e.g., high stress) with a threshold. The input of this step is the sensor data and emotion judgment result, and the output is the detection of an abnormal state that exceeds the threshold.

[0356] Step 6: Generate a warning signal

[0357] The server generates a warning signal when an abnormality is detected. Specifically, it creates a warning message based on temperature, water pressure, sound level, and abnormal emotional state exceeding the set threshold. The input of this step is the abnormality detection result, and the output is the generated warning signal.

[0358] Step 7: Sending a warning signal

[0359] The server sends the generated warning signal to an administrator, for example, by email or a dedicated notification application, notifying the administrator of the warning message in real time. The input of this step is the warning signal, and the output is the warning message sent to the administrator.

[0360] Step 8: Administrator Action

[0361] The administrator receives the notification from the server, checks the situation on-site, and takes appropriate action, such as activating a cooling device in the case of high temperatures, or instructing workers to take breaks in the case of high stress, etc. The input of this step is the warning message sent to the administrator, and the output is the execution of the response action.

[0362] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0363] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0364] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0365] [Second embodiment]

[0366] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0367] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0368] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0369] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0370] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0371] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0372] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0373] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0374] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0375] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0376] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0377] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0378] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. This system operates in cooperation with the terminal, server, and user.

[0379] Device behavior

[0380] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). For each sensor, it sets its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The device acquires data from each sensor at regular intervals (e.g., every second). For example, it reads 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0381] The device then verifies the acquired data and generates a warning signal if the sensor value exceeds the set threshold. For the temperature sensor, the threshold is below -5°C or above 40°C. For the water pressure sensor, the threshold is above 7, and for the sound sensor, the threshold is a sound level above 85dB.

[0382] Once a warning signal is generated, the terminal sends the warning signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" is generated and sent to the server.

[0383] Server Operation

[0384] The server receives the alert signal from the device, analyzes the received message, and takes appropriate action based on its content. For example, it may send a notification to an administrator based on the type and location of the alert. Notifications are typically sent via email or a dedicated application.

[0385] The server also logs the warning signals it receives and stores them in a database for later analysis, which can be used to analyze long-term trends and take measures to prevent recurrence.

[0386] User response

[0387] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take necessary safety measures.

[0388] Specific examples

[0389] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety can be ensured by the on-site staff taking measures such as turning on the cooling equipment.

[0390] In this way, the system of the present invention improves on-site safety by detecting environmental abnormalities in real time and responding quickly.

[0391] The processing flow will be explained below.

[0392] Step 1:

[0393] The device initializes the sensor.

[0394] When the device starts up, it initializes the temperature sensor, water pressure sensor, and sound sensor.

[0395] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0396] Step 2:

[0397] The device acquires data from the sensors.

[0398] Obtain data from each sensor at regular intervals (e.g., every second).

[0399] For example, obtain data of 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0400] Step 3:

[0401] The terminal verifies the acquired data.

[0402] Check whether the temperature, water pressure, and sound sensor values ​​exceed the set thresholds.

[0403] For example, if the temperature is 45°C, it will detect that this exceeds the threshold of 40°C.

[0404] Step 4:

[0405] The terminal generates an alert signal.

[0406] If the data exceeds a set threshold, a warning signal is generated.

[0407] For example, generate the message "Temperature warning: 45°C at Site-A".

[0408] Step 5:

[0409] The terminal sends an alert signal to the server.

[0410] The generated alert signal is sent to the server.

[0411] For example, send the message "Temperature warning: 45°C at Site-A" to the server.

[0412] Step 6:

[0413] The server receives the alert signal.

[0414] Analyze the warning signal received from the device.

[0415] Check the message content to determine what type of warning it is.

[0416] Step 7:

[0417] The server takes the appropriate action.

[0418] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0419] For example, send a notification to the administrator saying "Temperature warning: 45°C at Site-A".

[0420] Step 8:

[0421] The server logs the warning signal.

[0422] The contents of the received warning signals are logged and stored in a database.

[0423] This allows for later analysis and reporting.

[0424] Step 9:

[0425] The user receives a notification.

[0426] The administrator receives notifications from the server.

[0427] For example, an administrator sees the notification "Temperature warning: 45°C at Site-A."

[0428] Step 10:

[0429] The user reassesss the situation on the ground and responds.

[0430] Check the situation on-site and take necessary measures.

[0431] For example, a manager may dispatch on-site staff and instruct them to take measures such as operating cooling equipment.

[0432] Example 1

[0433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0434] In on-site environmental monitoring, there is a need for a system that can quickly detect abnormalities in temperature, pressure, and sound and notify the appropriate parties. However, conventional systems often have problems with real-time response and accuracy, and are often insufficient to respond to emergency situations. This can lead to delays in appropriate responses to ensure on-site safety.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0436] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring pressure with a pressure sensor, means for measuring the ambient sound with a sound sensor, means for generating a warning signal when the measured temperature exceeds a certain range, means for generating a warning signal when the measured pressure exceeds a certain range, means for generating a warning signal when the measured sound exceeds a certain range, means for transmitting the generated warning signals to a remote server, and means for the remote server to analyze the warning signals and notify an appropriate administrator. This makes it possible to detect abnormalities in real time and quickly notify the administrator.

[0437] A "temperature sensor" is a device for measuring the ambient temperature.

[0438] A "pressure sensor" is a device for measuring ambient pressure.

[0439] A "sound sensor" is a device for measuring surrounding sounds.

[0440] "Certain range" refers to a predetermined safe tolerance range for temperature, pressure, and sound.

[0441] A "warning signal" is a signal that indicates an abnormality and is generated when a value measured by a sensor exceeds a certain range.

[0442] A "remote server" is a central control unit that receives and analyzes data sent by sensors and takes appropriate action.

[0443] "Analysis" is the process of examining the content of received data in detail and interpreting its meaning and intent.

[0444] "Administrator" refers to the person or organization responsible for receiving system alert notifications and taking appropriate action.

[0445] "Notification" is the act or means of communicating information to a specific recipient.

[0446] "Real-time" means that data is processed as soon as it is generated.

[0447] The system according to the present invention uses temperature, pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify an administrator when a certain threshold is exceeded. This system operates in cooperation with terminals, a server, and users.

[0448] Device behavior

[0449] When the system starts up, the device initializes the temperature sensor, pressure sensor, and sound sensor. Initialization involves setting the type of each sensor (temperature, pressure, sound) and initial measurement value (e.g., 0). Specifically, the temperature sensor measures the ambient temperature, the pressure sensor measures pressure, and the sound sensor measures the ambient sound. After initialization, the device acquires data from each sensor at regular intervals (e.g., every second). The acquired data is measured as specific examples, such as 25°C for the temperature sensor, 5 for the pressure sensor, and 60dB for the sound sensor.

[0450] Next, the terminal verifies the acquired data. This is the process of checking whether the value of each sensor exceeds the set threshold (temperature: below -5°C or above 40°C, pressure: above 7, sound: above 85dB). If the threshold is exceeded, the terminal generates a warning signal. For example, if the temperature sensor value exceeds 45°C, the terminal generates a warning signal saying "Temperature warning: 45°C at Site-A."

[0451] The generated warning signal is then sent to the server, ensuring that a remote administrator is promptly notified of any abnormalities.

[0452] Server Operation

[0453] The server receives the warning signal sent from the terminal. After receiving it, the server analyzes the contents of the warning message and identifies the type of abnormality and its location. Based on the results of this analysis, the server sends a notification to the appropriate administrator. Notifications are generally sent via email or a dedicated application. For example, a notification stating "Temperature warning: 45°C at Site-A" may be sent to the administrator.

[0454] Additionally, the server logs the received warning signals and stores them in a database, which can later be used as a reference for long-term analysis and for developing measures to prevent recurrence.

[0455] User response

[0456] The user receives a notification from the server. They check the notification and take prompt action to reassess the situation at the site. For example, if they receive a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take safety measures such as turning on cooling equipment. This ensures the safety of the site.

[0457] Specific examples

[0458] As an example, consider safety management at a construction site. A terminal installed at a construction site is equipped with a temperature sensor, a pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety is ensured when the on-site staff activates the cooling equipment.

[0459] Prompt Sentence Examples

[0460] An example prompt for generating an explanation for this system using a generative AI model might look something like this:

[0461] "Describe a system that uses temperature, pressure, and sound sensors to detect environmental abnormalities in real time, and generates a warning signal to notify an administrator. Please generate a detailed description of how the terminal, server, and user work together."

[0462] The above is an embodiment of the present invention. The present invention is a system that improves safety at a work site by detecting environmental abnormalities in real time and quickly notifying a manager.

[0463] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0464] System program processing flow

[0465] Terminal handling

[0466] Step 1:

[0467] The device initializes the temperature sensor, pressure sensor, and sound sensor at system startup. During initialization, the type of each sensor (temperature, pressure, sound) and the initial measurement value (e.g., 0) are set. The sensor object is provided as input, and the output sets the initial value of each sensor.

[0468] Step 2:

[0469] The device acquires data from each sensor at regular intervals (e.g., every second). It acquires the current measurement environment value of each sensor as input and generates a data list like the one below as output. For example, data of 25°C is obtained from the temperature sensor, 5 from the pressure sensor, and 60dB from the sound sensor.

[0470] Step 3:

[0471] The terminal verifies the acquired sensor data. It receives the sensor measurement data as input and checks whether each value exceeds the set threshold. As output, if an abnormal value is detected, a warning signal is generated. For example, if the temperature sensor value records 45°C, this value exceeds the threshold of 40°C, so a warning signal is generated.

[0472] Step 4:

[0473] The terminal generates a warning signal. It receives sensor data exceeding a threshold as input and generates a warning signal message as output. For example, a warning signal saying "Temperature warning: 45°C at Site-A" is generated.

[0474] Step 5:

[0475] The terminal that generated the alert signal sends the signal to the server. As input, it takes the generated alert signal message and sends it to the server. As output, it confirms that the communication process of sending the alert signal to the server was successful.

[0476] Server Processing

[0477] Step 1:

[0478] The server receives the warning signal sent from the terminal. It receives the warning signal message from the terminal as input and confirms the receipt as output.

[0479] Step 2:

[0480] The server analyzes the received warning message. It takes the warning signal message as input and analyzes its content (e.g., warning type, location). It gets the analysis result as output and determines the appropriate response.

[0481] Step 3:

[0482] The server sends notifications to the appropriate administrators based on the analysis results. It takes the warning analysis results as input, generates and sends a notification message, and outputs a message such as "Temperature warning: 45°C at Site-A" via email or a dedicated application.

[0483] Step 4:

[0484] The server logs the received warning signals and stores them in a database. It takes the content of the warning signal as input and generates an entry to store in the database. As output, a database entry is created for long-term analysis and to take measures to prevent recurrence.

[0485] User response

[0486] Step 1:

[0487] The user receives notifications from the server. As input, the user gets the notification message sent from the server, and as output, the user checks its contents.

[0488] Step 2:

[0489] The user checks the notification content and takes prompt action to reassess the situation at the site. The user takes the notification message (e.g., "Temperature warning: 45°C at Site-A") as input and determines the on-site response action as output.

[0490] Step 3:

[0491] The user contacts the on-site staff and takes necessary safety measures. The input is the notification content, and the output is the on-site staff taking specific measures (e.g., turning on the cooling equipment).

[0492] These are the specific processing steps of this system. Each processing step clearly defines the input, data processing, calculation, and output, and the specific operations at each step are explained in detail. This allows you to understand how each component works together to fulfill the functions of the system as a whole.

[0493] (Application example 1)

[0494] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0495] Environmental monitoring within factories is an important element in ensuring safety, but it is difficult to detect abnormalities in real time using a wide variety of sensors and take immediate appropriate action. With conventional systems, there are cases where the response after detecting an abnormality is delayed, which can lead to dangerous situations. In particular, when multiple abnormalities occur simultaneously, it is difficult to keep up with the response.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0497] In this invention, the server includes means for analyzing the received warning signal and taking appropriate action based on the content of the signal, means for transmitting the generated warning signal to an administrator, and means for notifying the administrator, thereby enabling rapid analysis and notification when an abnormality is detected, allowing the administrator to take immediate action.

[0498] A "temperature sensor" is a device that measures the ambient temperature and transmits that data.

[0499] A "water pressure sensor" is a device that measures the water pressure in pipes or tanks and transmits that data.

[0500] A "sound sensor" is a device that measures the volume of the surrounding sound and transmits that data.

[0501] A "warning signal" is an alert message that is generated when a set threshold is exceeded.

[0502] The "server" is a central system that receives data from sensors and manages analysis and warning signals.

[0503] An "administrator" is a person responsible for monitoring the system and responding to any abnormalities that occur.

[0504] "Analysis" is the process of determining appropriate action based on received data and warning signals.

[0505] "Notification" is the act of transmitting abnormality or warning information to an administrator.

[0506] This invention provides a system that monitors the factory environment in real time and responds quickly if an abnormality is detected. The system consists of a factory robot equipped with temperature, water pressure, and sound sensors, a server that analyzes and notifies data, and an administrator's smartphone or tablet.

[0507] Hardware and software used

[0508] Factory robot: Equipped with multiple sensors (temperature, water pressure, sound sensors).

[0509] Server: Receives data, analyzes it, and notifies administrators. Specific server software used includes the Flask framework and an SMTP server.

[0510] Smartphone or tablet: A device used by the administrator to receive alert notifications through a dedicated application.

[0511] Data collection and analysis process

[0512] First, the terminal (factory robot) initializes each sensor when it starts up. The sensors acquire data every second, for example, 45°C from the temperature sensor, 8 from the water pressure sensor, and 90dB from the water sensor. These data are compared with certain thresholds, and if they exceed the threshold, a warning signal is generated.

[0513] Server Roles

[0514] The server receives the generated warning signal and analyzes its contents. For example, if a warning signal such as "Temperature warning: 45°C at Site-A" is received, it immediately takes action based on the contents. Based on the analysis results, a notification is sent to the administrator via email or a dedicated application.

[0515] Administrator response

[0516] The administrator receives the notification from the server and responds promptly. After checking the contents of the notification, they dispatch staff to the site and take safety measures, such as turning on cooling equipment.

[0517] Specific examples

[0518] For example, suppose a factory uses a factory robot equipped with a temperature sensor, a water pressure sensor, and a sound sensor. The robot collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator receives the notification and instructs the field staff to check the situation and take necessary action. For example, the following warning signal is generated on the server: "Temperature warning: 45°C reached at Site-A."

[0519] Prompt Sentence Examples

[0520] "Generate alerts based on sensor values ​​and notify administrators. Create a mechanism to send an alert signal to the server if the temperature exceeds 40 degrees, the water pressure exceeds 7, or the sound level exceeds 85dB."

[0521] In this way, the system can detect abnormalities in real time and respond quickly to improve safety within the factory.

[0522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0523] Step 1:

[0524] When the device starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). The type of each sensor (temperature, water pressure, sound) and the initial measurement value (e.g., 0) are set as input, and the sensor's ready status is obtained as output. Specific operations include calibrating the sensors and setting their initial values.

[0525] Step 2:

[0526] The device acquires data from each sensor at regular intervals (e.g., every second). The input here is raw data from the sensor, and the output is a dataset of measured temperature, pressure, and sound. Specifically, it follows the data collection protocol of each sensor and acquires temperature (e.g., water temperature) from the temperature sensor, pressure from the water pressure sensor, and ambient sound data from the sound sensor.

[0527] Step 3:

[0528] The terminal verifies the acquired data and generates an alert signal if the threshold is exceeded. The input here is the data set obtained from the sensor and the threshold, and the output is the alert signal generated if the threshold is exceeded. Specifically, it checks whether the temperature sensor data is below -5°C or above 40°C, whether the water pressure sensor data is above 7, and whether the sound sensor data is above 85dB, and generates an alert signal if the conditions are met.

[0529] Step 4:

[0530] The terminal sends the generated warning signal to the server. The input is the generated warning signal, and the output is the warning message sent to the server. As a specific operation, the terminal sends the warning message (e.g., "Temperature warning: 45°C at Site-A") to the server using an HTTP request or other communication means.

[0531] Step 5:

[0532] The server receives an alert signal from the terminal. The input is the alert signal sent from the terminal, and the output is the received alert data. Specifically, the server listens for communication at a specific endpoint and analyzes the data appropriately when it receives an alert signal.

[0533] Step 6:

[0534] The server analyzes and takes appropriate action based on the content of the received warning signal. The input is the received warning data, and the output is the analysis result and notification action. Specifically, the server processes the type of warning (temperature, water pressure, sound) and location information to send a notification to the appropriate administrator.

[0535] Step 7:

[0536] The server sends a notification to the administrator. The input is the analyzed warning data, and the output is a notification to the administrator's device. Specifically, the server sends an email or generates a push notification to convey the information to the administrator.

[0537] Step 8:

[0538] The administrator receives the notification from the server and takes action. The input is the notification message from the server, and the output is the start of the action. Specifically, the administrator checks the notification content, quickly reassessss the situation on site, and instructs the necessary safety measures. For example, specific actions are taken, such as turning on cooling equipment or dispatching on-site staff.

[0539] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0540] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, a more advanced safety management system can be achieved.

[0541] Device behavior

[0542] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor) and the emotion engine. Each sensor is set to its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The emotion engine performs initial settings to analyze voice and facial expression data.

[0543] The device acquires data from each sensor at regular intervals (e.g., every second) and also collects the user's voice and facial expression data. For example, the device acquires 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and also collects the user's voice and facial expression data.

[0544] Emotion Engine Operation

[0545] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state. For example, it analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. If the user's emotional state is abnormal (e.g., extreme stress or anger), the emotion engine detects it and generates a warning signal.

[0546] Validating data and generating warning signals

[0547] The device verifies the data obtained from each sensor and the judgment results from the emotion engine. It checks whether the sensor values ​​for temperature, water pressure, and sound exceed the set thresholds and whether the emotion engine detects any abnormalities in the user. For example, if the temperature is 45°C or the emotion engine detects stress, it generates a warning signal.

[0548] Sending to the server and notifying

[0549] Once a warning signal is generated, the terminal sends the signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" is generated and sent to the server.

[0550] Server Operation

[0551] The server receives warning signals from the devices, analyzes the received messages, and takes appropriate action based on their content. For example, it sends notifications to administrators via email or a dedicated application based on the type and location of the warning. It also logs the received warning signals for long-term data analysis.

[0552] User response

[0553] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A," the user can dispatch staff to the site and take the necessary safety measures.

[0554] Specific examples

[0555] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This terminal collects data from each sensor every second, and also collects the user's voice and facial expression data. If the temperature reaches 45°C or the emotion engine determines that the user is in a state of high stress, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, the on-site staff can take measures such as turning on the cooling equipment or instructing stressed staff to take a break, thereby ensuring safety.

[0556] In this way, the system of the present invention detects environmental abnormalities and the user's emotional state in real time and responds quickly, thereby significantly improving safety on site.

[0557] The processing flow will be explained below.

[0558] Step 1:

[0559] The device initializes the sensors and emotion engine.

[0560] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor and emotion engine.

[0561] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0562] The emotion engine performs initial setup to analyze voice and facial expression data.

[0563] Step 2:

[0564] The device receives data from sensors and an emotion engine.

[0565] Data is acquired from each sensor at regular intervals (e.g., every second), and the user's voice and facial expression data are also collected.

[0566] For example, data of 25°C is obtained from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and data on the user's voice and facial expression is also collected.

[0567] Step 3:

[0568] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state.

[0569] It analyzes voice tone and facial expressions to determine whether the user is feeling stressed.

[0570] For example, if the user is determined to be in a high stress state.

[0571] Step 4:

[0572] The device verifies the data from each sensor and the emotion engine.

[0573] Check whether the value of each sensor exceeds the set threshold and whether the emotion engine detects an abnormal emotional state.

[0574] For example, if the temperature is 45°C or if the emotion engine detects stress.

[0575] Step 5:

[0576] The terminal generates an alert signal.

[0577] If the data exceeds a set threshold, a warning signal is generated.

[0578] For example, messages such as "Temperature warning: 45°C at Site-A" and "User stress detected: High stress at Site-A" are generated.

[0579] Step 6:

[0580] The terminal sends an alert signal to the server.

[0581] The generated alert signal is sent to the server.

[0582] For example, messages such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" are sent to the server.

[0583] Step 7:

[0584] The server receives the alert signal.

[0585] Analyze the warning signal received from the device.

[0586] Check the message content to determine what type of warning it is.

[0587] Step 8:

[0588] The server takes the appropriate action.

[0589] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0590] For example, send notifications to the administrator such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A".

[0591] Step 9:

[0592] The server logs the warning signal.

[0593] The contents of the received warning signals are logged and stored in a database.

[0594] This allows for later analysis and reporting.

[0595] Step 10:

[0596] The user receives a notification.

[0597] The administrator receives notifications from the server.

[0598] For example, an administrator sees notifications such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A."

[0599] Step 11:

[0600] The user reassesss the situation on the ground and responds.

[0601] Check the situation on-site and take necessary measures.

[0602] For example, a manager might dispatch on-site staff to run cooling equipment or give stressed staff a break.

[0603] Example 2

[0604] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0605] Conventional safety management systems only monitor ambient temperature, water pressure, and sound, generating a warning signal if any of these values ​​exceed a certain range. While these systems are effective at detecting abnormalities in the physical environment, they are unable to consider the user's emotional state or psychological stress, making them unable to implement comprehensive safety measures. To ensure safety, including when on-site workers are under high stress, it is important to recognize their emotional state and issue a warning if there is a problem.

[0606] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for measuring the ambient temperature using a temperature sensor, a means for measuring the water pressure using a water pressure sensor, a means for measuring the ambient sound using a sound sensor, and an emotion analysis engine as a means for recognizing the user's emotion. This enables comprehensive environmental monitoring including the user's emotional state.

[0607] A "temperature sensor" is a sensor that measures the ambient temperature and is a device for acquiring environmental temperature data.

[0608] A "water pressure sensor" is a sensor that measures the surrounding water pressure and is a device for obtaining liquid pressure data.

[0609] A "sound sensor" is a sensor that measures surrounding sounds and is a device that obtains the volume and frequency of sounds.

[0610] An "emotion analysis engine" is a system for recognizing a user's emotional state, and is a device that analyzes voice and facial expression data to determine the user's psychological state.

[0611] A "warning signal" is a notification signal that is generated when data measured by a sensor exceeds a set threshold or when an emotion analysis engine detects an abnormal emotional state.

[0612] "Administrator" means the person or organization operating and monitoring the system and responsible for receiving and responding to generated alert signals.

[0613] "Means of acquiring data" refers to the functions and methods for collecting information from sensors and sentiment analysis engines.

[0614] The "fixed interval" refers to the time interval at which data collection is performed, and is set to acquire data every second, for example.

[0615] The "notification means" refers to a function and method for sending a warning signal to an administrator, and is a device that completes the notification by using email or a dedicated application.

[0616] The system starts by initializing the device's temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor sets its initial measurement value according to its type (temperature, water pressure, sound), and the emotion analysis engine performs initial settings to analyze voice and facial expression data.

[0617] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, while also collecting the user's voice and facial expression data. For example, the temperature sensor collects data at 25°C, the water pressure sensor collects data at 5, and the sound sensor collects data at 60dB. The emotion analysis engine then analyzes the user's emotional state.

[0618] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. Based on the analysis results, it determines the user's emotional state as "normal" or "abnormal."

[0619] The device generates a warning signal if an abnormality is detected based on data from each sensor and the emotion analysis engine. For example, if the temperature reaches 45°C, the water pressure exceeds a preset threshold, the sound exceeds a threshold, or the emotion analysis engine detects a high level of stress, it will determine this as an abnormality and generate a warning signal.

[0620] When an alert signal is generated, the device sends it to the server. The alert signal is converted into JSON or XML format and sent to the server's specified API endpoint. The server analyzes the received alert signal and takes appropriate action based on its content. For example, it may notify the administrator of the alert content via email or a dedicated application, and record the received alert signal in a database.

[0621] The user receives a notification from the server, checks its contents, contacts the on-site staff to confirm the situation, and takes any necessary measures. For example, if the user receives a notification that reads, "The temperature sensor at the construction site has detected 45°C. Please check the on-site situation and the status of the staff and take any necessary measures," the user can contact the on-site staff and instruct them to operate the cooling equipment or take a break.

[0622] In this way, this system uses temperature sensors, water pressure sensors, sound sensors, and an emotion analysis engine to comprehensively monitor the surrounding environment and the user's emotional state, and can respond quickly if an abnormality is detected. The program can be implemented using programming languages ​​such as C, Python, and Java, and an appropriately selected software library can be used to control the various sensors and emotion analysis engine and analyze data.

[0623] To cite a specific example, in safety management at construction sites, this system collects environmental data and user emotional data every second from devices installed on-site. If the temperature reaches 45°C or if the emotion analysis engine determines a high stress state, it generates a warning signal and sends it to a server. The server analyzes the warning signal and promptly notifies the administrator. The administrator can immediately contact on-site staff and instruct them on appropriate responses, ensuring safety.

[0624] An example of a text prompt is: "The temperature sensor at the construction site has detected a temperature of 45°C. Please check the situation at the site and the status of the staff and take necessary measures."

[0625] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0626] Step 1: Initialize your device

[0627] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor is set to an initial measurement value for its type, and the emotion analysis engine is initialized to analyze voice and facial expression data. Specifically, it starts up the temperature sensor and sets its initial measurement value to 0°C. It starts up the water pressure sensor and sets its initial measurement value to 0. It starts up the sound sensor and sets its initial measurement value to 0dB. It starts up the emotion analysis engine and configures the voice analysis module and facial expression analysis module.

[0628] Input: System start signal

[0629] Output: Initialized sensors and sentiment analysis engine

[0630] Step 2: Collecting data from sensors

[0631] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, and also collects the user's voice and facial expression data. For example, the temperature sensor collects 25°C, the water pressure sensor collects 5, and the sound sensor collects 60dB. In addition, the device captures the user's voice with a microphone and captures their facial expressions with a camera.

[0632] Input: Initialized sensors and sentiment analysis engine

[0633] Output: Collected temperature, water pressure, and sound data, as well as user voice and facial expression data

[0634] Step 3: Analysis by the sentiment analysis engine

[0635] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expression to determine whether the user is feeling stressed. For example, the voice analysis module analyzes the tone, pitch, and speed of the voice, while the facial expression analysis module analyzes changes in facial expression to determine whether the user is feeling "normal" or "abnormal."

[0636] Input: Collected voice and facial expression data

[0637] Output: User's emotional state (normal or abnormal)

[0638] Step 4: Validate data and generate warning signals

[0639] The device generates a warning signal if an abnormality is detected based on data from each sensor and the results of the emotion analysis engine, such as when the temperature reaches 45°C, the water pressure exceeds a set threshold, the sound exceeds a set threshold, or the emotion analysis engine detects a high level of stress.

[0640] Input: Data from each sensor and the results of the emotion analysis engine

[0641] Output: Generated warning signal

[0642] Step 5: Sending a warning signal to the server

[0643] The device sends the generated alert signal to the server. The alert signal is converted into JSON or XML format and sent to the specified API endpoint of the server. Specifically, the device converts the generated alert signal into JSON and sends an HTTP request to the server's API.

[0644] Input: Generated warning signal

[0645] Output: Warning signal sent to the server

[0646] Step 6: Server analyzes and notifies warning signals

[0647] The server analyzes the received warning signal and takes appropriate action based on its content. It notifies the administrator of the warning content via email or a dedicated application and records the received warning signal in a database. For example, the server generates a message such as "Temperature warning: 45°C at Site-A" and notifies the administrator.

[0648] Input: Alert signal sent to the server

[0649] Output: Analyzed warning signals and administrator notifications

[0650] Step 7: User Action

[0651] The user receives a notification from the server, checks the contents, contacts the on-site staff, has them check the situation, and takes necessary measures, such as instructing the on-site staff to operate the cooling equipment or take a break.

[0652] Input: Notification from the server

[0653] Output: On-site staff response and implementation of measures

[0654] (Application example 2)

[0655] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0656] In business environments such as factories, there is a need to monitor the operation of machines and the work status of workers in real time and respond quickly when abnormalities occur. However, there is currently a lack of systems that can comprehensively manage data from various sensors, such as temperature sensors, water pressure sensors, and sound sensors, as well as the emotional state of workers, making it difficult to improve safety management. Another issue is that it is difficult to detect workers' stress levels and emotional abnormalities in real time and take appropriate action.

[0657] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0658] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring water pressure with a water pressure sensor, means for measuring ambient sound with a sound sensor, means including an emotion engine for analyzing the user's voice and facial expression data to determine the emotional state, means for generating an alarm signal when the measured temperature exceeds a certain range, means for generating an alarm signal when the measured water pressure exceeds a certain range, means for generating an alarm signal when the measured sound exceeds a certain range, means for generating an alarm signal when the user's emotional state is determined to be abnormal, and means for transmitting the generated alarm signal to a manager. This makes it possible to monitor environmental abnormalities and the emotional state of workers in real time, and to respond quickly when an abnormality occurs.

[0659] A "temperature sensor" is a device that detects the ambient temperature and outputs the measured value.

[0660] A "water pressure sensor" is a device that detects the pressure of a liquid and outputs the measured value.

[0661] A "sound sensor" is a device that detects surrounding sounds and outputs their measured values.

[0662] An "emotion engine" is an algorithm or software that analyzes a user's voice and facial expression data to determine their emotional state.

[0663] A "warning signal" is a warning message that is generated when a measurement exceeds a set threshold.

[0664] "Administrator" means the person or entity responsible for monitoring and managing the System.

[0665] The "regular interval" refers to a predetermined periodic time interval.

[0666] "User's emotional state" refers to the emotional or psychological state that the user is currently experiencing.

[0667] "Notification" means the act or process of communicating warning signals or other important information to management.

[0668] To implement this invention, we first need to build a system that includes a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This system is installed in a factory environment and monitors environmental data and the user's emotional state in real time.

[0669] Natural language description of the program

[0670] The server initializes the temperature sensor (water pressure sensor and sound sensor devices such as DHT11, MPX5010, KY-038, etc.) and emotion engine (such as Amazon's Rekognition). Each sensor acquires the ambient temperature, pressure, and sound level in real time, and the emotion engine analyzes the user's voice and facial expression data.

[0671] This allows the server to continuously collect data from each sensor, and if an abnormality is detected, it will immediately generate a warning signal and notify the administrator. For example, if the temperature sensor detects a temperature above 45°C, or if the emotion engine determines that the user's stress level is high, this information will be sent to the server as a warning signal.

[0672] Upon receiving the warning signal, the server analyzes its contents and takes appropriate action based on the abnormality. For example, it sends a warning message to the administrator via a dedicated notification application. All warning signals are also recorded on the server for later analysis and improvement activities.

[0673] Hardware and software used

[0674] The system uses the following hardware and software:

[0675] Temperature sensors: DHT11, DHT22

[0676] Water pressure sensor: MPX5010, MS5803

[0677] Sound sensor: KY-038, MAX9814

[0678] Emotion engine: Amazon Rekognition, etc.

[0679] Notification system: Twilio, Slack, or a dedicated notification system

[0680] Specific examples

[0681] When a temperature sensor in a factory exceeds 45°C, the server generates a warning signal saying "Temperature warning: 45°C at Site-A" and sends a real-time notification to the administrator via a dedicated notification application. Furthermore, when the emotion engine detects a high user stress level, a warning message saying "User stress detected: High stress at Site-A" is sent to the administrator. In this way, the system supports quick and accurate safety measures.

[0682] Prompt Sentence Examples

[0683] The following prompt sentences can be used by the generative AI model to allow the emotion engine to properly parse the user's emotional state:

[0684] "Analyze the following data to determine the user's emotional state: voice data = some_voice_data, facial expression data = some_expression_data. If the user is stressed, respond with 'high_stress'."

[0685] This allows the emotion engine to determine the user's emotional state in real time based on their voice and facial expression data.

[0686] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0687] Step 1: Initialize your device

[0688] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion engine. Specifically, it sets the types and initial measurement values ​​of these sensors and engines. The input of this step is the initial setting data for the sensors and engines, and the output is the initialized sensors and engines.

[0689] Step 2: Acquiring Sensor Data

[0690] The terminal acquires data from the temperature sensor, water pressure sensor, and sound sensor at regular intervals (e.g., every second). It acquires the temperature, pressure, and sound level measurements and records them in a database. The input of this step is real-time data from each sensor, and the output is the acquired temperature, pressure, and sound level data.

[0691] Step 3: Collecting emotion data

[0692] The device collects the user's voice and facial expression data. Specifically, it acquires voice and image data from input devices such as microphones and cameras and sends them to the emotion engine. The input to this step is the user's voice and facial expression data, and the output is the data to be analyzed that is sent to the emotion engine.

[0693] Step 4: Sentiment Analysis

[0694] The emotion engine analyzes the collected voice and facial expression data to determine the user's emotional state. Specifically, it analyzes changes in voice tone and facial expressions to assess stress levels. The input for this step is voice and facial expression data, and the output is a judgment result regarding the user's emotional state.

[0695] Step 5: Validate the data

[0696] The server verifies the data obtained from each sensor and the judgment result from the emotion engine. It compares the temperature, water pressure, and sound sensor measurements and the emotion engine's judgment result (e.g., high stress) with a threshold. The input of this step is the sensor data and emotion judgment result, and the output is the detection of an abnormal state that exceeds the threshold.

[0697] Step 6: Generate a warning signal

[0698] The server generates a warning signal when an abnormality is detected. Specifically, it creates a warning message based on temperature, water pressure, sound level, and abnormal emotional state exceeding the set threshold. The input of this step is the abnormality detection result, and the output is the generated warning signal.

[0699] Step 7: Sending a warning signal

[0700] The server sends the generated warning signal to an administrator, for example, by email or a dedicated notification application, notifying the administrator of the warning message in real time. The input of this step is the warning signal, and the output is the warning message sent to the administrator.

[0701] Step 8: Administrator Action

[0702] The administrator receives the notification from the server, checks the situation on-site, and takes appropriate action, such as activating a cooling device in the case of high temperatures, or instructing workers to take breaks in the case of high stress, etc. The input of this step is the warning message sent to the administrator, and the output is the execution of the response action.

[0703] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0704] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0705] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0706] [Third embodiment]

[0707] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0708] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0709] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0710] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0711] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0712] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0713] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0714] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0715] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0716] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0717] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0718] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0719] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. This system operates in cooperation with the terminal, server, and user.

[0720] Device behavior

[0721] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). For each sensor, it sets its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The device acquires data from each sensor at regular intervals (e.g., every second). For example, it reads 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0722] The device then verifies the acquired data and generates a warning signal if the sensor value exceeds the set threshold. For the temperature sensor, the threshold is below -5°C or above 40°C. For the water pressure sensor, the threshold is above 7, and for the sound sensor, the threshold is a sound level above 85dB.

[0723] Once a warning signal is generated, the terminal sends the warning signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" is generated and sent to the server.

[0724] Server Operation

[0725] The server receives the alert signal from the device, analyzes the received message, and takes appropriate action based on its content. For example, it may send a notification to an administrator based on the type and location of the alert. Notifications are typically sent via email or a dedicated application.

[0726] The server also logs the warning signals it receives and stores them in a database for later analysis, which can be used to analyze long-term trends and take measures to prevent recurrence.

[0727] User response

[0728] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take necessary safety measures.

[0729] Specific examples

[0730] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety can be ensured by the on-site staff taking measures such as turning on the cooling equipment.

[0731] In this way, the system of the present invention improves on-site safety by detecting environmental abnormalities in real time and responding quickly.

[0732] The processing flow will be explained below.

[0733] Step 1:

[0734] The device initializes the sensor.

[0735] When the device starts up, it initializes the temperature sensor, water pressure sensor, and sound sensor.

[0736] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0737] Step 2:

[0738] The device acquires data from the sensors.

[0739] Obtain data from each sensor at regular intervals (e.g., every second).

[0740] For example, obtain data of 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[0741] Step 3:

[0742] The terminal verifies the acquired data.

[0743] Check whether the temperature, water pressure, and sound sensor values ​​exceed the set thresholds.

[0744] For example, if the temperature is 45°C, it will detect that this exceeds the threshold of 40°C.

[0745] Step 4:

[0746] The terminal generates an alert signal.

[0747] If the data exceeds a set threshold, a warning signal is generated.

[0748] For example, generate the message "Temperature warning: 45°C at Site-A".

[0749] Step 5:

[0750] The terminal sends an alert signal to the server.

[0751] The generated alert signal is sent to the server.

[0752] For example, send the message "Temperature warning: 45°C at Site-A" to the server.

[0753] Step 6:

[0754] The server receives the alert signal.

[0755] Analyze the warning signal received from the device.

[0756] Check the message content to determine what type of warning it is.

[0757] Step 7:

[0758] The server takes the appropriate action.

[0759] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0760] For example, send a notification to the administrator saying "Temperature warning: 45°C at Site-A".

[0761] Step 8:

[0762] The server logs the warning signal.

[0763] The contents of the received warning signals are logged and stored in a database.

[0764] This allows for later analysis and reporting.

[0765] Step 9:

[0766] The user receives a notification.

[0767] The administrator receives notifications from the server.

[0768] For example, an administrator sees the notification "Temperature warning: 45°C at Site-A."

[0769] Step 10:

[0770] The user reassesss the situation on the ground and responds.

[0771] Check the situation on-site and take necessary measures.

[0772] For example, a manager may dispatch on-site staff and instruct them to take measures such as operating cooling equipment.

[0773] Example 1

[0774] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0775] In on-site environmental monitoring, there is a need for a system that can quickly detect abnormalities in temperature, pressure, and sound and notify the appropriate parties. However, conventional systems often have problems with real-time response and accuracy, and are often insufficient to respond to emergency situations. This can lead to delays in appropriate responses to ensure on-site safety.

[0776] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0777] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring pressure with a pressure sensor, means for measuring the ambient sound with a sound sensor, means for generating a warning signal when the measured temperature exceeds a certain range, means for generating a warning signal when the measured pressure exceeds a certain range, means for generating a warning signal when the measured sound exceeds a certain range, means for transmitting the generated warning signals to a remote server, and means for the remote server to analyze the warning signals and notify an appropriate administrator. This makes it possible to detect abnormalities in real time and quickly notify the administrator.

[0778] A "temperature sensor" is a device for measuring the ambient temperature.

[0779] A "pressure sensor" is a device for measuring ambient pressure.

[0780] A "sound sensor" is a device for measuring surrounding sounds.

[0781] "Certain range" refers to a predetermined safe tolerance range for temperature, pressure, and sound.

[0782] A "warning signal" is a signal that indicates an abnormality and is generated when a value measured by a sensor exceeds a certain range.

[0783] A "remote server" is a central control unit that receives and analyzes data sent by sensors and takes appropriate action.

[0784] "Analysis" is the process of examining the content of received data in detail and interpreting its meaning and intent.

[0785] "Administrator" refers to the person or organization responsible for receiving system alert notifications and taking appropriate action.

[0786] "Notification" is the act or means of communicating information to a specific recipient.

[0787] "Real-time" means that data is processed as soon as it is generated.

[0788] The system according to the present invention uses temperature, pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify an administrator when a certain threshold is exceeded. This system operates in cooperation with terminals, a server, and users.

[0789] Device behavior

[0790] When the system starts up, the device initializes the temperature sensor, pressure sensor, and sound sensor. Initialization involves setting the type of each sensor (temperature, pressure, sound) and initial measurement value (e.g., 0). Specifically, the temperature sensor measures the ambient temperature, the pressure sensor measures pressure, and the sound sensor measures the ambient sound. After initialization, the device acquires data from each sensor at regular intervals (e.g., every second). The acquired data is measured as specific examples, such as 25°C for the temperature sensor, 5 for the pressure sensor, and 60dB for the sound sensor.

[0791] Next, the terminal verifies the acquired data. This is the process of checking whether the value of each sensor exceeds the set threshold (temperature: below -5°C or above 40°C, pressure: above 7, sound: above 85dB). If the threshold is exceeded, the terminal generates a warning signal. For example, if the temperature sensor value exceeds 45°C, the terminal generates a warning signal saying "Temperature warning: 45°C at Site-A."

[0792] The generated warning signal is then sent to the server, ensuring that a remote administrator is promptly notified of any abnormalities.

[0793] Server Operation

[0794] The server receives the warning signal sent from the terminal. After receiving it, the server analyzes the contents of the warning message and identifies the type of abnormality and its location. Based on the results of this analysis, the server sends a notification to the appropriate administrator. Notifications are generally sent via email or a dedicated application. For example, a notification stating "Temperature warning: 45°C at Site-A" may be sent to the administrator.

[0795] Additionally, the server logs the received warning signals and stores them in a database, which can later be used as a reference for long-term analysis and for developing measures to prevent recurrence.

[0796] User response

[0797] The user receives a notification from the server. They check the notification and take prompt action to reassess the situation at the site. For example, if they receive a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take safety measures such as turning on cooling equipment. This ensures the safety of the site.

[0798] Specific examples

[0799] As an example, consider safety management at a construction site. A terminal installed at a construction site is equipped with a temperature sensor, a pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety is ensured when the on-site staff activates the cooling equipment.

[0800] Prompt Sentence Examples

[0801] An example prompt for generating an explanation for this system using a generative AI model might look something like this:

[0802] "Describe a system that uses temperature, pressure, and sound sensors to detect environmental abnormalities in real time, and generates a warning signal to notify an administrator. Please generate a detailed description of how the terminal, server, and user work together."

[0803] The above is an embodiment of the present invention. The present invention is a system that improves safety at a work site by detecting environmental abnormalities in real time and quickly notifying a manager.

[0804] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0805] System program processing flow

[0806] Terminal handling

[0807] Step 1:

[0808] The device initializes the temperature sensor, pressure sensor, and sound sensor at system startup. During initialization, the type of each sensor (temperature, pressure, sound) and the initial measurement value (e.g., 0) are set. The sensor object is provided as input, and the output sets the initial value of each sensor.

[0809] Step 2:

[0810] The device acquires data from each sensor at regular intervals (e.g., every second). It acquires the current measurement environment value of each sensor as input and generates a data list like the one below as output. For example, data of 25°C is obtained from the temperature sensor, 5 from the pressure sensor, and 60dB from the sound sensor.

[0811] Step 3:

[0812] The terminal verifies the acquired sensor data. It receives the sensor measurement data as input and checks whether each value exceeds the set threshold. As output, if an abnormal value is detected, a warning signal is generated. For example, if the temperature sensor value records 45°C, this value exceeds the threshold of 40°C, so a warning signal is generated.

[0813] Step 4:

[0814] The terminal generates a warning signal. It receives sensor data exceeding a threshold as input and generates a warning signal message as output. For example, a warning signal saying "Temperature warning: 45°C at Site-A" is generated.

[0815] Step 5:

[0816] The terminal that generated the alert signal sends the signal to the server. As input, it takes the generated alert signal message and sends it to the server. As output, it confirms that the communication process of sending the alert signal to the server was successful.

[0817] Server Processing

[0818] Step 1:

[0819] The server receives the warning signal sent from the terminal. It receives the warning signal message from the terminal as input and confirms the receipt as output.

[0820] Step 2:

[0821] The server analyzes the received warning message. It takes the warning signal message as input and analyzes its content (e.g., warning type, location). It gets the analysis result as output and determines the appropriate response.

[0822] Step 3:

[0823] The server sends notifications to the appropriate administrators based on the analysis results. It takes the warning analysis results as input, generates and sends a notification message, and outputs a message such as "Temperature warning: 45°C at Site-A" via email or a dedicated application.

[0824] Step 4:

[0825] The server logs the received warning signals and stores them in a database. It takes the content of the warning signal as input and generates an entry to store in the database. As output, a database entry is created for long-term analysis and to take measures to prevent recurrence.

[0826] User response

[0827] Step 1:

[0828] The user receives notifications from the server. As input, the user gets the notification message sent from the server, and as output, the user checks its contents.

[0829] Step 2:

[0830] The user checks the notification content and takes prompt action to reassess the situation at the site. The user takes the notification message (e.g., "Temperature warning: 45°C at Site-A") as input and determines the on-site response action as output.

[0831] Step 3:

[0832] The user contacts the on-site staff and takes necessary safety measures. The input is the notification content, and the output is the on-site staff taking specific measures (e.g., turning on the cooling equipment).

[0833] These are the specific processing steps of this system. Each processing step clearly defines the input, data processing, calculation, and output, and the specific operations at each step are explained in detail. This allows you to understand how each component works together to fulfill the functions of the system as a whole.

[0834] (Application example 1)

[0835] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0836] Environmental monitoring within factories is an important element in ensuring safety, but it is difficult to detect abnormalities in real time using a wide variety of sensors and take immediate appropriate action. With conventional systems, there are cases where the response after detecting an abnormality is delayed, which can lead to dangerous situations. In particular, when multiple abnormalities occur simultaneously, it is difficult to keep up with the response.

[0837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0838] In this invention, the server includes means for analyzing the received warning signal and taking appropriate action based on the content of the signal, means for transmitting the generated warning signal to an administrator, and means for notifying the administrator, thereby enabling rapid analysis and notification when an abnormality is detected, allowing the administrator to take immediate action.

[0839] A "temperature sensor" is a device that measures the ambient temperature and transmits that data.

[0840] A "water pressure sensor" is a device that measures the water pressure in pipes or tanks and transmits that data.

[0841] A "sound sensor" is a device that measures the volume of the surrounding sound and transmits that data.

[0842] A "warning signal" is an alert message that is generated when a set threshold is exceeded.

[0843] The "server" is a central system that receives data from sensors and manages analysis and warning signals.

[0844] An "administrator" is a person responsible for monitoring the system and responding to any abnormalities that occur.

[0845] "Analysis" is the process of determining appropriate action based on received data and warning signals.

[0846] "Notification" is the act of transmitting abnormality or warning information to an administrator.

[0847] This invention provides a system that monitors the factory environment in real time and responds quickly if an abnormality is detected. The system consists of a factory robot equipped with temperature, water pressure, and sound sensors, a server that analyzes and notifies data, and an administrator's smartphone or tablet.

[0848] Hardware and software used

[0849] Factory robot: Equipped with multiple sensors (temperature, water pressure, sound sensors).

[0850] Server: Receives data, analyzes it, and notifies administrators. Specific server software used includes the Flask framework and an SMTP server.

[0851] Smartphone or tablet: A device used by the administrator to receive alert notifications through a dedicated application.

[0852] Data collection and analysis process

[0853] First, the terminal (factory robot) initializes each sensor when it starts up. The sensors acquire data every second, for example, 45°C from the temperature sensor, 8 from the water pressure sensor, and 90dB from the water sensor. These data are compared with certain thresholds, and if they exceed the threshold, a warning signal is generated.

[0854] Server Roles

[0855] The server receives the generated warning signal and analyzes its contents. For example, if a warning signal such as "Temperature warning: 45°C at Site-A" is received, it immediately takes action based on the contents. Based on the analysis results, a notification is sent to the administrator via email or a dedicated application.

[0856] Administrator response

[0857] The administrator receives the notification from the server and responds promptly. After checking the contents of the notification, they dispatch staff to the site and take safety measures, such as turning on cooling equipment.

[0858] Specific examples

[0859] For example, suppose a factory uses a factory robot equipped with a temperature sensor, a water pressure sensor, and a sound sensor. The robot collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator receives the notification and instructs the field staff to check the situation and take necessary action. For example, the following warning signal is generated on the server: "Temperature warning: 45°C reached at Site-A."

[0860] Prompt Sentence Examples

[0861] "Generate alerts based on sensor values ​​and notify administrators. Create a mechanism to send an alert signal to the server if the temperature exceeds 40 degrees, the water pressure exceeds 7, or the sound level exceeds 85dB."

[0862] In this way, the system can detect abnormalities in real time and respond quickly to improve safety within the factory.

[0863] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0864] Step 1:

[0865] When the device starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). The type of each sensor (temperature, water pressure, sound) and the initial measurement value (e.g., 0) are set as input, and the sensor's ready status is obtained as output. Specific operations include calibrating the sensors and setting their initial values.

[0866] Step 2:

[0867] The device acquires data from each sensor at regular intervals (e.g., every second). The input here is raw data from the sensor, and the output is a dataset of measured temperature, pressure, and sound. Specifically, it follows the data collection protocol of each sensor and acquires temperature (e.g., water temperature) from the temperature sensor, pressure from the water pressure sensor, and ambient sound data from the sound sensor.

[0868] Step 3:

[0869] The terminal verifies the acquired data and generates an alert signal if the threshold is exceeded. The input here is the data set obtained from the sensor and the threshold, and the output is the alert signal generated if the threshold is exceeded. Specifically, it checks whether the temperature sensor data is below -5°C or above 40°C, whether the water pressure sensor data is above 7, and whether the sound sensor data is above 85dB, and generates an alert signal if the conditions are met.

[0870] Step 4:

[0871] The terminal sends the generated warning signal to the server. The input is the generated warning signal, and the output is the warning message sent to the server. As a specific operation, the terminal sends the warning message (e.g., "Temperature warning: 45°C at Site-A") to the server using an HTTP request or other communication means.

[0872] Step 5:

[0873] The server receives an alert signal from the terminal. The input is the alert signal sent from the terminal, and the output is the received alert data. Specifically, the server listens for communication at a specific endpoint and analyzes the data appropriately when it receives an alert signal.

[0874] Step 6:

[0875] The server analyzes and takes appropriate action based on the content of the received warning signal. The input is the received warning data, and the output is the analysis result and notification action. Specifically, the server processes the type of warning (temperature, water pressure, sound) and location information to send a notification to the appropriate administrator.

[0876] Step 7:

[0877] The server sends a notification to the administrator. The input is the analyzed warning data, and the output is a notification to the administrator's device. Specifically, the server sends an email or generates a push notification to convey the information to the administrator.

[0878] Step 8:

[0879] The administrator receives the notification from the server and takes action. The input is the notification message from the server, and the output is the start of the action. Specifically, the administrator checks the notification content, quickly reassessss the situation on site, and instructs the necessary safety measures. For example, specific actions are taken, such as turning on cooling equipment or dispatching on-site staff.

[0880] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0881] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, a more advanced safety management system can be achieved.

[0882] Device behavior

[0883] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor) and the emotion engine. Each sensor is set to its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The emotion engine performs initial settings to analyze voice and facial expression data.

[0884] The device acquires data from each sensor at regular intervals (e.g., every second) and also collects the user's voice and facial expression data. For example, the device acquires 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and also collects the user's voice and facial expression data.

[0885] Emotion Engine Operation

[0886] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state. For example, it analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. If the user's emotional state is abnormal (e.g., extreme stress or anger), the emotion engine detects it and generates a warning signal.

[0887] Validating data and generating warning signals

[0888] The device verifies the data obtained from each sensor and the judgment results from the emotion engine. It checks whether the sensor values ​​for temperature, water pressure, and sound exceed the set thresholds and whether the emotion engine detects any abnormalities in the user. For example, if the temperature is 45°C or the emotion engine detects stress, it generates a warning signal.

[0889] Sending to the server and notifying

[0890] Once a warning signal is generated, the terminal sends the signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" is generated and sent to the server.

[0891] Server Operation

[0892] The server receives warning signals from the devices, analyzes the received messages, and takes appropriate action based on their content. For example, it sends notifications to administrators via email or a dedicated application based on the type and location of the warning. It also logs the received warning signals for long-term data analysis.

[0893] User response

[0894] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A," the user can dispatch staff to the site and take the necessary safety measures.

[0895] Specific examples

[0896] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This terminal collects data from each sensor every second, and also collects the user's voice and facial expression data. If the temperature reaches 45°C or the emotion engine determines that the user is in a state of high stress, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, the on-site staff can take measures such as turning on the cooling equipment or instructing stressed staff to take a break, thereby ensuring safety.

[0897] In this way, the system of the present invention detects environmental abnormalities and the user's emotional state in real time and responds quickly, thereby significantly improving safety on site.

[0898] The processing flow will be explained below.

[0899] Step 1:

[0900] The device initializes the sensors and emotion engine.

[0901] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor and emotion engine.

[0902] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[0903] The emotion engine performs initial setup to analyze voice and facial expression data.

[0904] Step 2:

[0905] The device receives data from sensors and an emotion engine.

[0906] Data is acquired from each sensor at regular intervals (e.g., every second), and the user's voice and facial expression data are also collected.

[0907] For example, data of 25°C is obtained from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and data on the user's voice and facial expression is also collected.

[0908] Step 3:

[0909] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state.

[0910] It analyzes voice tone and facial expressions to determine whether the user is feeling stressed.

[0911] For example, if the user is determined to be in a high stress state.

[0912] Step 4:

[0913] The device verifies the data from each sensor and the emotion engine.

[0914] Check whether the value of each sensor exceeds the set threshold and whether the emotion engine detects an abnormal emotional state.

[0915] For example, if the temperature is 45°C or if the emotion engine detects stress.

[0916] Step 5:

[0917] The terminal generates an alert signal.

[0918] If the data exceeds a set threshold, a warning signal is generated.

[0919] For example, messages such as "Temperature warning: 45°C at Site-A" and "User stress detected: High stress at Site-A" are generated.

[0920] Step 6:

[0921] The terminal sends an alert signal to the server.

[0922] The generated alert signal is sent to the server.

[0923] For example, messages such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" are sent to the server.

[0924] Step 7:

[0925] The server receives the alert signal.

[0926] Analyze the warning signal received from the device.

[0927] Check the message content to determine what type of warning it is.

[0928] Step 8:

[0929] The server takes the appropriate action.

[0930] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[0931] For example, send notifications to the administrator such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A".

[0932] Step 9:

[0933] The server logs the warning signal.

[0934] The contents of the received warning signals are logged and stored in a database.

[0935] This allows for later analysis and reporting.

[0936] Step 10:

[0937] The user receives a notification.

[0938] The administrator receives notifications from the server.

[0939] For example, an administrator sees notifications such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A."

[0940] Step 11:

[0941] The user reassesss the situation on the ground and responds.

[0942] Check the situation on-site and take necessary measures.

[0943] For example, a manager might dispatch on-site staff to run cooling equipment or give stressed staff a break.

[0944] Example 2

[0945] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0946] Conventional safety management systems only monitor ambient temperature, water pressure, and sound, generating a warning signal if any of these values ​​exceed a certain range. While these systems are effective at detecting abnormalities in the physical environment, they are unable to consider the user's emotional state or psychological stress, making them unable to implement comprehensive safety measures. To ensure safety, including when on-site workers are under high stress, it is important to recognize their emotional state and issue a warning if there is a problem.

[0947] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for measuring the ambient temperature using a temperature sensor, a means for measuring the water pressure using a water pressure sensor, a means for measuring the ambient sound using a sound sensor, and an emotion analysis engine as a means for recognizing the user's emotion. This enables comprehensive environmental monitoring including the user's emotional state.

[0948] A "temperature sensor" is a sensor that measures the ambient temperature and is a device for acquiring environmental temperature data.

[0949] A "water pressure sensor" is a sensor that measures the surrounding water pressure and is a device for obtaining liquid pressure data.

[0950] A "sound sensor" is a sensor that measures surrounding sounds and is a device that obtains the volume and frequency of sounds.

[0951] An "emotion analysis engine" is a system for recognizing a user's emotional state, and is a device that analyzes voice and facial expression data to determine the user's psychological state.

[0952] A "warning signal" is a notification signal that is generated when data measured by a sensor exceeds a set threshold or when an emotion analysis engine detects an abnormal emotional state.

[0953] "Administrator" means the person or organization operating and monitoring the system and responsible for receiving and responding to generated alert signals.

[0954] "Means of acquiring data" refers to the functions and methods for collecting information from sensors and sentiment analysis engines.

[0955] The "fixed interval" refers to the time interval at which data collection is performed, and is set to acquire data every second, for example.

[0956] The "notification means" refers to a function and method for sending a warning signal to an administrator, and is a device that completes the notification by using email or a dedicated application.

[0957] The system starts by initializing the device's temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor sets its initial measurement value according to its type (temperature, water pressure, sound), and the emotion analysis engine performs initial settings to analyze voice and facial expression data.

[0958] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, while also collecting the user's voice and facial expression data. For example, the temperature sensor collects data at 25°C, the water pressure sensor collects data at 5, and the sound sensor collects data at 60dB. The emotion analysis engine then analyzes the user's emotional state.

[0959] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. Based on the analysis results, it determines the user's emotional state as "normal" or "abnormal."

[0960] The device generates a warning signal if an abnormality is detected based on data from each sensor and the emotion analysis engine. For example, if the temperature reaches 45°C, the water pressure exceeds a preset threshold, the sound exceeds a threshold, or the emotion analysis engine detects a high level of stress, it will determine this as an abnormality and generate a warning signal.

[0961] When an alert signal is generated, the device sends it to the server. The alert signal is converted into JSON or XML format and sent to the server's specified API endpoint. The server analyzes the received alert signal and takes appropriate action based on its content. For example, it may notify the administrator of the alert content via email or a dedicated application, and record the received alert signal in a database.

[0962] The user receives a notification from the server, checks its contents, contacts the on-site staff to confirm the situation, and takes any necessary measures. For example, if the user receives a notification that reads, "The temperature sensor at the construction site has detected 45°C. Please check the on-site situation and the status of the staff and take any necessary measures," the user can contact the on-site staff and instruct them to operate the cooling equipment or take a break.

[0963] In this way, this system uses temperature sensors, water pressure sensors, sound sensors, and an emotion analysis engine to comprehensively monitor the surrounding environment and the user's emotional state, and can respond quickly if an abnormality is detected. The program can be implemented using programming languages ​​such as C, Python, and Java, and an appropriately selected software library can be used to control the various sensors and emotion analysis engine and analyze data.

[0964] To cite a specific example, in safety management at construction sites, this system collects environmental data and user emotional data every second from devices installed on-site. If the temperature reaches 45°C or if the emotion analysis engine determines a high stress state, it generates a warning signal and sends it to a server. The server analyzes the warning signal and promptly notifies the administrator. The administrator can immediately contact on-site staff and instruct them on appropriate responses, ensuring safety.

[0965] An example of a text prompt is: "The temperature sensor at the construction site has detected a temperature of 45°C. Please check the situation at the site and the status of the staff and take necessary measures."

[0966] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0967] Step 1: Initialize your device

[0968] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor is set to an initial measurement value for its type, and the emotion analysis engine is initialized to analyze voice and facial expression data. Specifically, it starts up the temperature sensor and sets its initial measurement value to 0°C. It starts up the water pressure sensor and sets its initial measurement value to 0. It starts up the sound sensor and sets its initial measurement value to 0dB. It starts up the emotion analysis engine and configures the voice analysis module and facial expression analysis module.

[0969] Input: System start signal

[0970] Output: Initialized sensors and sentiment analysis engine

[0971] Step 2: Collecting data from sensors

[0972] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, and also collects the user's voice and facial expression data. For example, the temperature sensor collects 25°C, the water pressure sensor collects 5, and the sound sensor collects 60dB. In addition, the device captures the user's voice with a microphone and captures their facial expressions with a camera.

[0973] Input: Initialized sensors and sentiment analysis engine

[0974] Output: Collected temperature, water pressure, and sound data, as well as user voice and facial expression data

[0975] Step 3: Analysis by the sentiment analysis engine

[0976] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expression to determine whether the user is feeling stressed. For example, the voice analysis module analyzes the tone, pitch, and speed of the voice, while the facial expression analysis module analyzes changes in facial expression to determine whether the user is feeling "normal" or "abnormal."

[0977] Input: Collected voice and facial expression data

[0978] Output: User's emotional state (normal or abnormal)

[0979] Step 4: Validate data and generate warning signals

[0980] The device generates a warning signal if an abnormality is detected based on data from each sensor and the results of the emotion analysis engine, such as when the temperature reaches 45°C, the water pressure exceeds a set threshold, the sound exceeds a set threshold, or the emotion analysis engine detects a high level of stress.

[0981] Input: Data from each sensor and the results of the emotion analysis engine

[0982] Output: Generated warning signal

[0983] Step 5: Sending a warning signal to the server

[0984] The device sends the generated alert signal to the server. The alert signal is converted into JSON or XML format and sent to the specified API endpoint of the server. Specifically, the device converts the generated alert signal into JSON and sends an HTTP request to the server's API.

[0985] Input: Generated warning signal

[0986] Output: Warning signal sent to the server

[0987] Step 6: Server analyzes and notifies warning signals

[0988] The server analyzes the received warning signal and takes appropriate action based on its content. It notifies the administrator of the warning content via email or a dedicated application and records the received warning signal in a database. For example, the server generates a message such as "Temperature warning: 45°C at Site-A" and notifies the administrator.

[0989] Input: Alert signal sent to the server

[0990] Output: Analyzed warning signals and administrator notifications

[0991] Step 7: User Action

[0992] The user receives a notification from the server, checks the contents, contacts the on-site staff, has them check the situation, and takes necessary measures, such as instructing the on-site staff to operate the cooling equipment or take a break.

[0993] Input: Notification from the server

[0994] Output: On-site staff response and implementation of measures

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0997] In business environments such as factories, there is a need to monitor the operation of machines and the work status of workers in real time and respond quickly when abnormalities occur. However, there is currently a lack of systems that can comprehensively manage data from various sensors, such as temperature sensors, water pressure sensors, and sound sensors, as well as the emotional state of workers, making it difficult to improve safety management. Another issue is that it is difficult to detect workers' stress levels and emotional abnormalities in real time and take appropriate action.

[0998] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0999] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring water pressure with a water pressure sensor, means for measuring ambient sound with a sound sensor, means including an emotion engine for analyzing the user's voice and facial expression data to determine the emotional state, means for generating an alarm signal when the measured temperature exceeds a certain range, means for generating an alarm signal when the measured water pressure exceeds a certain range, means for generating an alarm signal when the measured sound exceeds a certain range, means for generating an alarm signal when the user's emotional state is determined to be abnormal, and means for transmitting the generated alarm signal to a manager. This makes it possible to monitor environmental abnormalities and the emotional state of workers in real time, and to respond quickly when an abnormality occurs.

[1000] A "temperature sensor" is a device that detects the ambient temperature and outputs the measured value.

[1001] A "water pressure sensor" is a device that detects the pressure of a liquid and outputs the measured value.

[1002] A "sound sensor" is a device that detects surrounding sounds and outputs their measured values.

[1003] An "emotion engine" is an algorithm or software that analyzes a user's voice and facial expression data to determine their emotional state.

[1004] A "warning signal" is a warning message that is generated when a measurement exceeds a set threshold.

[1005] "Administrator" means the person or entity responsible for monitoring and managing the System.

[1006] The "regular interval" refers to a predetermined periodic time interval.

[1007] "User's emotional state" refers to the emotional or psychological state that the user is currently experiencing.

[1008] "Notification" means the act or process of communicating warning signals or other important information to management.

[1009] To implement this invention, we first need to build a system that includes a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This system is installed in a factory environment and monitors environmental data and the user's emotional state in real time.

[1010] Natural language description of the program

[1011] The server initializes the temperature sensor (water pressure sensor and sound sensor devices such as DHT11, MPX5010, KY-038, etc.) and emotion engine (such as Amazon's Rekognition). Each sensor acquires the ambient temperature, pressure, and sound level in real time, and the emotion engine analyzes the user's voice and facial expression data.

[1012] This allows the server to continuously collect data from each sensor, and if an abnormality is detected, it will immediately generate a warning signal and notify the administrator. For example, if the temperature sensor detects a temperature above 45°C, or if the emotion engine determines that the user's stress level is high, this information will be sent to the server as a warning signal.

[1013] Upon receiving the warning signal, the server analyzes its contents and takes appropriate action based on the abnormality. For example, it sends a warning message to the administrator via a dedicated notification application. All warning signals are also recorded on the server for later analysis and improvement activities.

[1014] Hardware and software used

[1015] The system uses the following hardware and software:

[1016] Temperature sensors: DHT11, DHT22

[1017] Water pressure sensor: MPX5010, MS5803

[1018] Sound sensor: KY-038, MAX9814

[1019] Emotion engine: Amazon Rekognition, etc.

[1020] Notification system: Twilio, Slack, or a dedicated notification system

[1021] Specific examples

[1022] When a temperature sensor in a factory exceeds 45°C, the server generates a warning signal saying "Temperature warning: 45°C at Site-A" and sends a real-time notification to the administrator via a dedicated notification application. Furthermore, when the emotion engine detects a high user stress level, a warning message saying "User stress detected: High stress at Site-A" is sent to the administrator. In this way, the system supports quick and accurate safety measures.

[1023] Prompt Sentence Examples

[1024] The following prompt sentences can be used by the generative AI model to allow the emotion engine to properly parse the user's emotional state:

[1025] "Analyze the following data to determine the user's emotional state: voice data = some_voice_data, facial expression data = some_expression_data. If the user is stressed, respond with 'high_stress'."

[1026] This allows the emotion engine to determine the user's emotional state in real time based on their voice and facial expression data.

[1027] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1028] Step 1: Initialize your device

[1029] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion engine. Specifically, it sets the types and initial measurement values ​​of these sensors and engines. The input of this step is the initial setting data for the sensors and engines, and the output is the initialized sensors and engines.

[1030] Step 2: Acquiring Sensor Data

[1031] The terminal acquires data from the temperature sensor, water pressure sensor, and sound sensor at regular intervals (e.g., every second). It acquires the temperature, pressure, and sound level measurements and records them in a database. The input of this step is real-time data from each sensor, and the output is the acquired temperature, pressure, and sound level data.

[1032] Step 3: Collecting emotion data

[1033] The device collects the user's voice and facial expression data. Specifically, it acquires voice and image data from input devices such as microphones and cameras and sends them to the emotion engine. The input to this step is the user's voice and facial expression data, and the output is the data to be analyzed that is sent to the emotion engine.

[1034] Step 4: Sentiment Analysis

[1035] The emotion engine analyzes the collected voice and facial expression data to determine the user's emotional state. Specifically, it analyzes changes in voice tone and facial expressions to assess stress levels. The input for this step is voice and facial expression data, and the output is a judgment result regarding the user's emotional state.

[1036] Step 5: Validate the data

[1037] The server verifies the data obtained from each sensor and the judgment result from the emotion engine. It compares the temperature, water pressure, and sound sensor measurements and the emotion engine's judgment result (e.g., high stress) with a threshold. The input of this step is the sensor data and emotion judgment result, and the output is the detection of an abnormal state that exceeds the threshold.

[1038] Step 6: Generate a warning signal

[1039] The server generates a warning signal when an abnormality is detected. Specifically, it creates a warning message based on temperature, water pressure, sound level, and abnormal emotional state exceeding the set threshold. The input of this step is the abnormality detection result, and the output is the generated warning signal.

[1040] Step 7: Sending a warning signal

[1041] The server sends the generated warning signal to an administrator, for example, by email or a dedicated notification application, notifying the administrator of the warning message in real time. The input of this step is the warning signal, and the output is the warning message sent to the administrator.

[1042] Step 8: Administrator Action

[1043] The administrator receives the notification from the server, checks the situation on-site, and takes appropriate action, such as activating a cooling device in the case of high temperatures, or instructing workers to take breaks in the case of high stress, etc. The input of this step is the warning message sent to the administrator, and the output is the execution of the response action.

[1044] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1045] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1046] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1047] [Fourth embodiment]

[1048] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1049] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1050] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1051] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1052] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1053] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1054] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1055] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1056] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1057] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1058] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1059] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1060] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1061] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. This system operates in cooperation with the terminal, server, and user.

[1062] Device behavior

[1063] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). For each sensor, it sets its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The device acquires data from each sensor at regular intervals (e.g., every second). For example, it reads 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[1064] The device then verifies the acquired data and generates a warning signal if the sensor value exceeds the set threshold. For the temperature sensor, the threshold is below -5°C or above 40°C. For the water pressure sensor, the threshold is above 7, and for the sound sensor, the threshold is a sound level above 85dB.

[1065] Once a warning signal is generated, the terminal sends the warning signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" is generated and sent to the server.

[1066] Server Operation

[1067] The server receives the alert signal from the device, analyzes the received message, and takes appropriate action based on its content. For example, it may send a notification to an administrator based on the type and location of the alert. Notifications are typically sent via email or a dedicated application.

[1068] The server also logs the warning signals it receives and stores them in a database for later analysis, which can be used to analyze long-term trends and take measures to prevent recurrence.

[1069] User response

[1070] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take necessary safety measures.

[1071] Specific examples

[1072] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety can be ensured by the on-site staff taking measures such as turning on the cooling equipment.

[1073] In this way, the system of the present invention improves on-site safety by detecting environmental abnormalities in real time and responding quickly.

[1074] The processing flow will be explained below.

[1075] Step 1:

[1076] The device initializes the sensor.

[1077] When the device starts up, it initializes the temperature sensor, water pressure sensor, and sound sensor.

[1078] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[1079] Step 2:

[1080] The device acquires data from the sensors.

[1081] Obtain data from each sensor at regular intervals (e.g., every second).

[1082] For example, obtain data of 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor.

[1083] Step 3:

[1084] The terminal verifies the acquired data.

[1085] Check whether the temperature, water pressure, and sound sensor values ​​exceed the set thresholds.

[1086] For example, if the temperature is 45°C, it will detect that this exceeds the threshold of 40°C.

[1087] Step 4:

[1088] The terminal generates an alert signal.

[1089] If the data exceeds a set threshold, a warning signal is generated.

[1090] For example, generate the message "Temperature warning: 45°C at Site-A".

[1091] Step 5:

[1092] The terminal sends an alert signal to the server.

[1093] The generated alert signal is sent to the server.

[1094] For example, send the message "Temperature warning: 45°C at Site-A" to the server.

[1095] Step 6:

[1096] The server receives the alert signal.

[1097] Analyze the warning signal received from the device.

[1098] Check the message content to determine what type of warning it is.

[1099] Step 7:

[1100] The server takes the appropriate action.

[1101] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[1102] For example, send a notification to the administrator saying "Temperature warning: 45°C at Site-A".

[1103] Step 8:

[1104] The server logs the warning signal.

[1105] The contents of the received warning signals are logged and stored in a database.

[1106] This allows for later analysis and reporting.

[1107] Step 9:

[1108] The user receives a notification.

[1109] The administrator receives notifications from the server.

[1110] For example, an administrator sees the notification "Temperature warning: 45°C at Site-A."

[1111] Step 10:

[1112] The user reassesss the situation on the ground and responds.

[1113] Check the situation on-site and take necessary measures.

[1114] For example, a manager may dispatch on-site staff and instruct them to take measures such as operating cooling equipment.

[1115] Example 1

[1116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1117] In on-site environmental monitoring, there is a need for a system that can quickly detect abnormalities in temperature, pressure, and sound and notify the appropriate parties. However, conventional systems often have problems with real-time response and accuracy, and are often insufficient to respond to emergency situations. This can lead to delays in appropriate responses to ensure on-site safety.

[1118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1119] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring pressure with a pressure sensor, means for measuring the ambient sound with a sound sensor, means for generating a warning signal when the measured temperature exceeds a certain range, means for generating a warning signal when the measured pressure exceeds a certain range, means for generating a warning signal when the measured sound exceeds a certain range, means for transmitting the generated warning signals to a remote server, and means for the remote server to analyze the warning signals and notify an appropriate administrator. This makes it possible to detect abnormalities in real time and quickly notify the administrator.

[1120] A "temperature sensor" is a device for measuring the ambient temperature.

[1121] A "pressure sensor" is a device for measuring ambient pressure.

[1122] A "sound sensor" is a device for measuring surrounding sounds.

[1123] "Certain range" refers to a predetermined safe tolerance range for temperature, pressure, and sound.

[1124] A "warning signal" is a signal that indicates an abnormality and is generated when a value measured by a sensor exceeds a certain range.

[1125] A "remote server" is a central control unit that receives and analyzes data sent by sensors and takes appropriate action.

[1126] "Analysis" is the process of examining the content of received data in detail and interpreting its meaning and intent.

[1127] "Administrator" refers to the person or organization responsible for receiving system alert notifications and taking appropriate action.

[1128] "Notification" is the act or means of communicating information to a specific recipient.

[1129] "Real-time" means that data is processed as soon as it is generated.

[1130] The system according to the present invention uses temperature, pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify an administrator when a certain threshold is exceeded. This system operates in cooperation with terminals, a server, and users.

[1131] Device behavior

[1132] When the system starts up, the device initializes the temperature sensor, pressure sensor, and sound sensor. Initialization involves setting the type of each sensor (temperature, pressure, sound) and initial measurement value (e.g., 0). Specifically, the temperature sensor measures the ambient temperature, the pressure sensor measures pressure, and the sound sensor measures the ambient sound. After initialization, the device acquires data from each sensor at regular intervals (e.g., every second). The acquired data is measured as specific examples, such as 25°C for the temperature sensor, 5 for the pressure sensor, and 60dB for the sound sensor.

[1133] Next, the terminal verifies the acquired data. This is the process of checking whether the value of each sensor exceeds the set threshold (temperature: below -5°C or above 40°C, pressure: above 7, sound: above 85dB). If the threshold is exceeded, the terminal generates a warning signal. For example, if the temperature sensor value exceeds 45°C, the terminal generates a warning signal saying "Temperature warning: 45°C at Site-A."

[1134] The generated warning signal is then sent to the server, ensuring that a remote administrator is promptly notified of any abnormalities.

[1135] Server Operation

[1136] The server receives the warning signal sent from the terminal. After receiving it, the server analyzes the contents of the warning message and identifies the type of abnormality and its location. Based on the results of this analysis, the server sends a notification to the appropriate administrator. Notifications are generally sent via email or a dedicated application. For example, a notification stating "Temperature warning: 45°C at Site-A" may be sent to the administrator.

[1137] Additionally, the server logs the received warning signals and stores them in a database, which can later be used as a reference for long-term analysis and for developing measures to prevent recurrence.

[1138] User response

[1139] The user receives a notification from the server. They check the notification and take prompt action to reassess the situation at the site. For example, if they receive a notification saying "Temperature warning: 45°C at Site-A," the user can dispatch staff to the site and take safety measures such as turning on cooling equipment. This ensures the safety of the site.

[1140] Specific examples

[1141] As an example, consider safety management at a construction site. A terminal installed at a construction site is equipped with a temperature sensor, a pressure sensor, and a sound sensor. This terminal collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to a server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, safety is ensured when the on-site staff activates the cooling equipment.

[1142] Prompt Sentence Examples

[1143] An example prompt for generating an explanation for this system using a generative AI model might look something like this:

[1144] "Describe a system that uses temperature, pressure, and sound sensors to detect environmental abnormalities in real time, and generates a warning signal to notify an administrator. Please generate a detailed description of how the terminal, server, and user work together."

[1145] The above is an embodiment of the present invention. The present invention is a system that improves safety at a work site by detecting environmental abnormalities in real time and quickly notifying a manager.

[1146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1147] System program processing flow

[1148] Terminal handling

[1149] Step 1:

[1150] The device initializes the temperature sensor, pressure sensor, and sound sensor at system startup. During initialization, the type of each sensor (temperature, pressure, sound) and the initial measurement value (e.g., 0) are set. The sensor object is provided as input, and the output sets the initial value of each sensor.

[1151] Step 2:

[1152] The device acquires data from each sensor at regular intervals (e.g., every second). It acquires the current measurement environment value of each sensor as input and generates a data list like the one below as output. For example, data of 25°C is obtained from the temperature sensor, 5 from the pressure sensor, and 60dB from the sound sensor.

[1153] Step 3:

[1154] The terminal verifies the acquired sensor data. It receives the sensor measurement data as input and checks whether each value exceeds the set threshold. As output, if an abnormal value is detected, a warning signal is generated. For example, if the temperature sensor value records 45°C, this value exceeds the threshold of 40°C, so a warning signal is generated.

[1155] Step 4:

[1156] The terminal generates a warning signal. It receives sensor data exceeding a threshold as input and generates a warning signal message as output. For example, a warning signal saying "Temperature warning: 45°C at Site-A" is generated.

[1157] Step 5:

[1158] The terminal that generated the alert signal sends the signal to the server. As input, it takes the generated alert signal message and sends it to the server. As output, it confirms that the communication process of sending the alert signal to the server was successful.

[1159] Server Processing

[1160] Step 1:

[1161] The server receives the warning signal sent from the terminal. It receives the warning signal message from the terminal as input and confirms the receipt as output.

[1162] Step 2:

[1163] The server analyzes the received warning message. It takes the warning signal message as input and analyzes its content (e.g., warning type, location). It gets the analysis result as output and determines the appropriate response.

[1164] Step 3:

[1165] The server sends notifications to the appropriate administrators based on the analysis results. It takes the warning analysis results as input, generates and sends a notification message, and outputs a message such as "Temperature warning: 45°C at Site-A" via email or a dedicated application.

[1166] Step 4:

[1167] The server logs the received warning signals and stores them in a database. It takes the content of the warning signal as input and generates an entry to store in the database. As output, a database entry is created for long-term analysis and to take measures to prevent recurrence.

[1168] User response

[1169] Step 1:

[1170] The user receives notifications from the server. As input, the user gets the notification message sent from the server, and as output, the user checks its contents.

[1171] Step 2:

[1172] The user checks the notification content and takes prompt action to reassess the situation at the site. The user takes the notification message (e.g., "Temperature warning: 45°C at Site-A") as input and determines the on-site response action as output.

[1173] Step 3:

[1174] The user contacts the on-site staff and takes necessary safety measures. The input is the notification content, and the output is the on-site staff taking specific measures (e.g., turning on the cooling equipment).

[1175] These are the specific processing steps of this system. Each processing step clearly defines the input, data processing, calculation, and output, and the specific operations at each step are explained in detail. This allows you to understand how each component works together to fulfill the functions of the system as a whole.

[1176] (Application example 1)

[1177] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1178] Environmental monitoring within factories is an important element in ensuring safety, but it is difficult to detect abnormalities in real time using a wide variety of sensors and take immediate appropriate action. With conventional systems, there are cases where the response after detecting an abnormality is delayed, which can lead to dangerous situations. In particular, when multiple abnormalities occur simultaneously, it is difficult to keep up with the response.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1180] In this invention, the server includes means for analyzing the received warning signal and taking appropriate action based on the content of the signal, means for transmitting the generated warning signal to an administrator, and means for notifying the administrator, thereby enabling rapid analysis and notification when an abnormality is detected, allowing the administrator to take immediate action.

[1181] A "temperature sensor" is a device that measures the ambient temperature and transmits that data.

[1182] A "water pressure sensor" is a device that measures the water pressure in pipes or tanks and transmits that data.

[1183] A "sound sensor" is a device that measures the volume of the surrounding sound and transmits that data.

[1184] A "warning signal" is an alert message that is generated when a set threshold is exceeded.

[1185] The "server" is a central system that receives data from sensors and manages analysis and warning signals.

[1186] An "administrator" is a person responsible for monitoring the system and responding to any abnormalities that occur.

[1187] "Analysis" is the process of determining appropriate action based on received data and warning signals.

[1188] "Notification" is the act of transmitting abnormality or warning information to an administrator.

[1189] This invention provides a system that monitors the factory environment in real time and responds quickly if an abnormality is detected. The system consists of a factory robot equipped with temperature, water pressure, and sound sensors, a server that analyzes and notifies data, and an administrator's smartphone or tablet.

[1190] Hardware and software used

[1191] Factory robot: Equipped with multiple sensors (temperature, water pressure, sound sensors).

[1192] Server: Receives data, analyzes it, and notifies administrators. Specific server software used includes the Flask framework and an SMTP server.

[1193] Smartphone or tablet: A device used by the administrator to receive alert notifications through a dedicated application.

[1194] Data collection and analysis process

[1195] First, the terminal (factory robot) initializes each sensor when it starts up. The sensors acquire data every second, for example, 45°C from the temperature sensor, 8 from the water pressure sensor, and 90dB from the water sensor. These data are compared with certain thresholds, and if they exceed the threshold, a warning signal is generated.

[1196] Server Roles

[1197] The server receives the generated warning signal and analyzes its contents. For example, if a warning signal such as "Temperature warning: 45°C at Site-A" is received, it immediately takes action based on the contents. Based on the analysis results, a notification is sent to the administrator via email or a dedicated application.

[1198] Administrator response

[1199] The administrator receives the notification from the server and responds promptly. After checking the contents of the notification, they dispatch staff to the site and take safety measures, such as turning on cooling equipment.

[1200] Specific examples

[1201] For example, suppose a factory uses a factory robot equipped with a temperature sensor, a water pressure sensor, and a sound sensor. The robot collects data from each sensor every second, and the moment the temperature reaches 45°C, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator of its contents. The administrator receives the notification and instructs the field staff to check the situation and take necessary action. For example, the following warning signal is generated on the server: "Temperature warning: 45°C reached at Site-A."

[1202] Prompt Sentence Examples

[1203] "Generate alerts based on sensor values ​​and notify administrators. Create a mechanism to send an alert signal to the server if the temperature exceeds 40 degrees, the water pressure exceeds 7, or the sound level exceeds 85dB."

[1204] In this way, the system can detect abnormalities in real time and respond quickly to improve safety within the factory.

[1205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1206] Step 1:

[1207] When the device starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor). The type of each sensor (temperature, water pressure, sound) and the initial measurement value (e.g., 0) are set as input, and the sensor's ready status is obtained as output. Specific operations include calibrating the sensors and setting their initial values.

[1208] Step 2:

[1209] The device acquires data from each sensor at regular intervals (e.g., every second). The input here is raw data from the sensor, and the output is a dataset of measured temperature, pressure, and sound. Specifically, it follows the data collection protocol of each sensor and acquires temperature (e.g., water temperature) from the temperature sensor, pressure from the water pressure sensor, and ambient sound data from the sound sensor.

[1210] Step 3:

[1211] The terminal verifies the acquired data and generates an alert signal if the threshold is exceeded. The input here is the data set obtained from the sensor and the threshold, and the output is the alert signal generated if the threshold is exceeded. Specifically, it checks whether the temperature sensor data is below -5°C or above 40°C, whether the water pressure sensor data is above 7, and whether the sound sensor data is above 85dB, and generates an alert signal if the conditions are met.

[1212] Step 4:

[1213] The terminal sends the generated warning signal to the server. The input is the generated warning signal, and the output is the warning message sent to the server. As a specific operation, the terminal sends the warning message (e.g., "Temperature warning: 45°C at Site-A") to the server using an HTTP request or other communication means.

[1214] Step 5:

[1215] The server receives an alert signal from the terminal. The input is the alert signal sent from the terminal, and the output is the received alert data. Specifically, the server listens for communication at a specific endpoint and analyzes the data appropriately when it receives an alert signal.

[1216] Step 6:

[1217] The server analyzes and takes appropriate action based on the content of the received warning signal. The input is the received warning data, and the output is the analysis result and notification action. Specifically, the server processes the type of warning (temperature, water pressure, sound) and location information to send a notification to the appropriate administrator.

[1218] Step 7:

[1219] The server sends a notification to the administrator. The input is the analyzed warning data, and the output is a notification to the administrator's device. Specifically, the server sends an email or generates a push notification to convey the information to the administrator.

[1220] Step 8:

[1221] The administrator receives the notification from the server and takes action. The input is the notification message from the server, and the output is the start of the action. Specifically, the administrator checks the notification content, quickly reassessss the situation on site, and instructs the necessary safety measures. For example, specific actions are taken, such as turning on cooling equipment or dispatching on-site staff.

[1222] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1223] This system uses temperature, water pressure, and sound sensors to monitor the surrounding environment, and generates a warning signal to notify the administrator when a certain threshold is exceeded. Furthermore, by combining this with an emotion engine that recognizes the user's emotions, a more advanced safety management system can be achieved.

[1224] Device behavior

[1225] When the device first starts up, it initializes each sensor (temperature sensor, water pressure sensor, sound sensor) and the emotion engine. Each sensor is set to its type (temperature, water pressure, sound) and initial measurement value (e.g., 0). The emotion engine performs initial settings to analyze voice and facial expression data.

[1226] The device acquires data from each sensor at regular intervals (e.g., every second) and also collects the user's voice and facial expression data. For example, the device acquires 25°C from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and also collects the user's voice and facial expression data.

[1227] Emotion Engine Operation

[1228] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state. For example, it analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. If the user's emotional state is abnormal (e.g., extreme stress or anger), the emotion engine detects it and generates a warning signal.

[1229] Validating data and generating warning signals

[1230] The device verifies the data obtained from each sensor and the judgment results from the emotion engine. It checks whether the sensor values ​​for temperature, water pressure, and sound exceed the set thresholds and whether the emotion engine detects any abnormalities in the user. For example, if the temperature is 45°C or the emotion engine detects stress, it generates a warning signal.

[1231] Sending to the server and notifying

[1232] Once a warning signal is generated, the terminal sends the signal to the server. For example, a message such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" is generated and sent to the server.

[1233] Server Operation

[1234] The server receives warning signals from the devices, analyzes the received messages, and takes appropriate action based on their content. For example, it sends notifications to administrators via email or a dedicated application based on the type and location of the warning. It also logs the received warning signals for long-term data analysis.

[1235] User response

[1236] The user receives a notification from the server. The user checks the notification and takes prompt action to reassess the situation at the site. For example, if the user receives a notification such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A," the user can dispatch staff to the site and take the necessary safety measures.

[1237] Specific examples

[1238] Consider safety management at a construction site as an example. A terminal installed at a construction site is equipped with a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This terminal collects data from each sensor every second, and also collects the user's voice and facial expression data. If the temperature reaches 45°C or the emotion engine determines that the user is in a state of high stress, it generates a warning signal and sends it to the server. The server analyzes this warning signal and notifies the administrator. The administrator who receives this notification immediately contacts the on-site staff, has them confirm the situation, and instructs them on the necessary response. For example, the on-site staff can take measures such as turning on the cooling equipment or instructing stressed staff to take a break, thereby ensuring safety.

[1239] In this way, the system of the present invention detects environmental abnormalities and the user's emotional state in real time and responds quickly, thereby significantly improving safety on site.

[1240] The processing flow will be explained below.

[1241] Step 1:

[1242] The device initializes the sensors and emotion engine.

[1243] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor and emotion engine.

[1244] Set the sensor type (temperature, water pressure, sound) and initialize the reading to 0.

[1245] The emotion engine performs initial setup to analyze voice and facial expression data.

[1246] Step 2:

[1247] The device receives data from sensors and an emotion engine.

[1248] Data is acquired from each sensor at regular intervals (e.g., every second), and the user's voice and facial expression data are also collected.

[1249] For example, data of 25°C is obtained from the temperature sensor, 5 from the water pressure sensor, and 60dB from the sound sensor, and data on the user's voice and facial expression is also collected.

[1250] Step 3:

[1251] The emotion engine analyzes collected voice and facial expression data to determine the user's emotional state.

[1252] It analyzes voice tone and facial expressions to determine whether the user is feeling stressed.

[1253] For example, if the user is determined to be in a high stress state.

[1254] Step 4:

[1255] The device verifies the data from each sensor and the emotion engine.

[1256] Check whether the value of each sensor exceeds the set threshold and whether the emotion engine detects an abnormal emotional state.

[1257] For example, if the temperature is 45°C or if the emotion engine detects stress.

[1258] Step 5:

[1259] The terminal generates an alert signal.

[1260] If the data exceeds a set threshold, a warning signal is generated.

[1261] For example, messages such as "Temperature warning: 45°C at Site-A" and "User stress detected: High stress at Site-A" are generated.

[1262] Step 6:

[1263] The terminal sends an alert signal to the server.

[1264] The generated alert signal is sent to the server.

[1265] For example, messages such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A" are sent to the server.

[1266] Step 7:

[1267] The server receives the alert signal.

[1268] Analyze the warning signal received from the device.

[1269] Check the message content to determine what type of warning it is.

[1270] Step 8:

[1271] The server takes the appropriate action.

[1272] Based on the content of the warning signal, notifications are sent to administrators via email or application.

[1273] For example, send notifications to the administrator such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A".

[1274] Step 9:

[1275] The server logs the warning signal.

[1276] The contents of the received warning signals are logged and stored in a database.

[1277] This allows for later analysis and reporting.

[1278] Step 10:

[1279] The user receives a notification.

[1280] The administrator receives notifications from the server.

[1281] For example, an administrator sees notifications such as "Temperature warning: 45°C at Site-A" or "User stress detected: High stress at Site-A."

[1282] Step 11:

[1283] The user reassesss the situation on the ground and responds.

[1284] Check the situation on-site and take necessary measures.

[1285] For example, a manager might dispatch on-site staff to run cooling equipment or give stressed staff a break.

[1286] Example 2

[1287] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] Conventional safety management systems only monitor ambient temperature, water pressure, and sound, generating a warning signal if any of these values ​​exceed a certain range. While these systems are effective at detecting abnormalities in the physical environment, they are unable to consider the user's emotional state or psychological stress, making them unable to implement comprehensive safety measures. To ensure safety, including when on-site workers are under high stress, it is important to recognize their emotional state and issue a warning if there is a problem.

[1289] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for measuring the ambient temperature using a temperature sensor, a means for measuring the water pressure using a water pressure sensor, a means for measuring the ambient sound using a sound sensor, and an emotion analysis engine as a means for recognizing the user's emotion. This enables comprehensive environmental monitoring including the user's emotional state.

[1290] A "temperature sensor" is a sensor that measures the ambient temperature and is a device for acquiring environmental temperature data.

[1291] A "water pressure sensor" is a sensor that measures the surrounding water pressure and is a device for obtaining liquid pressure data.

[1292] A "sound sensor" is a sensor that measures surrounding sounds and is a device that obtains the volume and frequency of sounds.

[1293] An "emotion analysis engine" is a system for recognizing a user's emotional state, and is a device that analyzes voice and facial expression data to determine the user's psychological state.

[1294] A "warning signal" is a notification signal that is generated when data measured by a sensor exceeds a set threshold or when an emotion analysis engine detects an abnormal emotional state.

[1295] "Administrator" means the person or organization operating and monitoring the system and responsible for receiving and responding to generated alert signals.

[1296] "Means of acquiring data" refers to the functions and methods for collecting information from sensors and sentiment analysis engines.

[1297] The "fixed interval" refers to the time interval at which data collection is performed, and is set to acquire data every second, for example.

[1298] The "notification means" refers to a function and method for sending a warning signal to an administrator, and is a device that completes the notification by using email or a dedicated application.

[1299] The system starts by initializing the device's temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor sets its initial measurement value according to its type (temperature, water pressure, sound), and the emotion analysis engine performs initial settings to analyze voice and facial expression data.

[1300] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, while also collecting the user's voice and facial expression data. For example, the temperature sensor collects data at 25°C, the water pressure sensor collects data at 5, and the sound sensor collects data at 60dB. The emotion analysis engine then analyzes the user's emotional state.

[1301] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expressions to determine whether the user is feeling stressed. Based on the analysis results, it determines the user's emotional state as "normal" or "abnormal."

[1302] The device generates a warning signal if an abnormality is detected based on data from each sensor and the emotion analysis engine. For example, if the temperature reaches 45°C, the water pressure exceeds a preset threshold, the sound exceeds a threshold, or the emotion analysis engine detects a high level of stress, it will determine this as an abnormality and generate a warning signal.

[1303] When an alert signal is generated, the device sends it to the server. The alert signal is converted into JSON or XML format and sent to the server's specified API endpoint. The server analyzes the received alert signal and takes appropriate action based on its content. For example, it may notify the administrator of the alert content via email or a dedicated application, and record the received alert signal in a database.

[1304] The user receives a notification from the server, checks its contents, contacts the on-site staff to confirm the situation, and takes any necessary measures. For example, if the user receives a notification that reads, "The temperature sensor at the construction site has detected 45°C. Please check the on-site situation and the status of the staff and take any necessary measures," the user can contact the on-site staff and instruct them to operate the cooling equipment or take a break.

[1305] In this way, this system uses temperature sensors, water pressure sensors, sound sensors, and an emotion analysis engine to comprehensively monitor the surrounding environment and the user's emotional state, and can respond quickly if an abnormality is detected. The program can be implemented using programming languages ​​such as C, Python, and Java, and an appropriately selected software library can be used to control the various sensors and emotion analysis engine and analyze data.

[1306] To cite a specific example, in safety management at construction sites, this system collects environmental data and user emotional data every second from devices installed on-site. If the temperature reaches 45°C or if the emotion analysis engine determines a high stress state, it generates a warning signal and sends it to a server. The server analyzes the warning signal and promptly notifies the administrator. The administrator can immediately contact on-site staff and instruct them on appropriate responses, ensuring safety.

[1307] An example of a text prompt is: "The temperature sensor at the construction site has detected a temperature of 45°C. Please check the situation at the site and the status of the staff and take necessary measures."

[1308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1309] Step 1: Initialize your device

[1310] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion analysis engine. Each sensor is set to an initial measurement value for its type, and the emotion analysis engine is initialized to analyze voice and facial expression data. Specifically, it starts up the temperature sensor and sets its initial measurement value to 0°C. It starts up the water pressure sensor and sets its initial measurement value to 0. It starts up the sound sensor and sets its initial measurement value to 0dB. It starts up the emotion analysis engine and configures the voice analysis module and facial expression analysis module.

[1311] Input: System start signal

[1312] Output: Initialized sensors and sentiment analysis engine

[1313] Step 2: Collecting data from sensors

[1314] The device collects data every second from the temperature sensor, water pressure sensor, and sound sensor, and also collects the user's voice and facial expression data. For example, the temperature sensor collects 25°C, the water pressure sensor collects 5, and the sound sensor collects 60dB. In addition, the device captures the user's voice with a microphone and captures their facial expressions with a camera.

[1315] Input: Initialized sensors and sentiment analysis engine

[1316] Output: Collected temperature, water pressure, and sound data, as well as user voice and facial expression data

[1317] Step 3: Analysis by the sentiment analysis engine

[1318] The emotion analysis engine analyzes the collected voice and facial expression data to determine the user's emotional state. It analyzes changes in voice tone and facial expression to determine whether the user is feeling stressed. For example, the voice analysis module analyzes the tone, pitch, and speed of the voice, while the facial expression analysis module analyzes changes in facial expression to determine whether the user is feeling "normal" or "abnormal."

[1319] Input: Collected voice and facial expression data

[1320] Output: User's emotional state (normal or abnormal)

[1321] Step 4: Validate data and generate warning signals

[1322] The device generates a warning signal if an abnormality is detected based on data from each sensor and the results of the emotion analysis engine, such as when the temperature reaches 45°C, the water pressure exceeds a set threshold, the sound exceeds a set threshold, or the emotion analysis engine detects a high level of stress.

[1323] Input: Data from each sensor and the results of the emotion analysis engine

[1324] Output: Generated warning signal

[1325] Step 5: Sending a warning signal to the server

[1326] The device sends the generated alert signal to the server. The alert signal is converted into JSON or XML format and sent to the specified API endpoint of the server. Specifically, the device converts the generated alert signal into JSON and sends an HTTP request to the server's API.

[1327] Input: Generated warning signal

[1328] Output: Warning signal sent to the server

[1329] Step 6: Server analyzes and notifies warning signals

[1330] The server analyzes the received warning signal and takes appropriate action based on its content. It notifies the administrator of the warning content via email or a dedicated application and records the received warning signal in a database. For example, the server generates a message such as "Temperature warning: 45°C at Site-A" and notifies the administrator.

[1331] Input: Alert signal sent to the server

[1332] Output: Analyzed warning signals and administrator notifications

[1333] Step 7: User Action

[1334] The user receives a notification from the server, checks the contents, contacts the on-site staff, has them check the situation, and takes necessary measures, such as instructing the on-site staff to operate the cooling equipment or take a break.

[1335] Input: Notification from the server

[1336] Output: On-site staff response and implementation of measures

[1337] (Application example 2)

[1338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] In business environments such as factories, there is a need to monitor the operation of machines and the work status of workers in real time and respond quickly when abnormalities occur. However, there is currently a lack of systems that can comprehensively manage data from various sensors, such as temperature sensors, water pressure sensors, and sound sensors, as well as the emotional state of workers, making it difficult to improve safety management. Another issue is that it is difficult to detect workers' stress levels and emotional abnormalities in real time and take appropriate action.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1341] In this invention, the server includes means for measuring the ambient temperature with a temperature sensor, means for measuring water pressure with a water pressure sensor, means for measuring ambient sound with a sound sensor, means including an emotion engine for analyzing the user's voice and facial expression data to determine the emotional state, means for generating an alarm signal when the measured temperature exceeds a certain range, means for generating an alarm signal when the measured water pressure exceeds a certain range, means for generating an alarm signal when the measured sound exceeds a certain range, means for generating an alarm signal when the user's emotional state is determined to be abnormal, and means for transmitting the generated alarm signal to a manager. This makes it possible to monitor environmental abnormalities and the emotional state of workers in real time, and to respond quickly when an abnormality occurs.

[1342] A "temperature sensor" is a device that detects the ambient temperature and outputs the measured value.

[1343] A "water pressure sensor" is a device that detects the pressure of a liquid and outputs the measured value.

[1344] A "sound sensor" is a device that detects surrounding sounds and outputs their measured values.

[1345] An "emotion engine" is an algorithm or software that analyzes a user's voice and facial expression data to determine their emotional state.

[1346] A "warning signal" is a warning message that is generated when a measurement exceeds a set threshold.

[1347] "Administrator" means the person or entity responsible for monitoring and managing the System.

[1348] The "regular interval" refers to a predetermined periodic time interval.

[1349] "User's emotional state" refers to the emotional or psychological state that the user is currently experiencing.

[1350] "Notification" means the act or process of communicating warning signals or other important information to management.

[1351] To implement this invention, we first need to build a system that includes a temperature sensor, a water pressure sensor, a sound sensor, and an emotion engine. This system is installed in a factory environment and monitors environmental data and the user's emotional state in real time.

[1352] Natural language description of the program

[1353] The server initializes the temperature sensor (water pressure sensor and sound sensor devices such as DHT11, MPX5010, KY-038, etc.) and emotion engine (such as Amazon's Rekognition). Each sensor acquires the ambient temperature, pressure, and sound level in real time, and the emotion engine analyzes the user's voice and facial expression data.

[1354] This allows the server to continuously collect data from each sensor, and if an abnormality is detected, it will immediately generate a warning signal and notify the administrator. For example, if the temperature sensor detects a temperature above 45°C, or if the emotion engine determines that the user's stress level is high, this information will be sent to the server as a warning signal.

[1355] Upon receiving the warning signal, the server analyzes its contents and takes appropriate action based on the abnormality. For example, it sends a warning message to the administrator via a dedicated notification application. All warning signals are also recorded on the server for later analysis and improvement activities.

[1356] Hardware and software used

[1357] The system uses the following hardware and software:

[1358] Temperature sensors: DHT11, DHT22

[1359] Water pressure sensor: MPX5010, MS5803

[1360] Sound sensor: KY-038, MAX9814

[1361] Emotion engine: Amazon Rekognition, etc.

[1362] Notification system: Twilio, Slack, or a dedicated notification system

[1363] Specific examples

[1364] When a temperature sensor in a factory exceeds 45°C, the server generates a warning signal saying "Temperature warning: 45°C at Site-A" and sends a real-time notification to the administrator via a dedicated notification application. Furthermore, when the emotion engine detects a high user stress level, a warning message saying "User stress detected: High stress at Site-A" is sent to the administrator. In this way, the system supports quick and accurate safety measures.

[1365] Prompt Sentence Examples

[1366] The following prompt sentences can be used by the generative AI model to allow the emotion engine to properly parse the user's emotional state:

[1367] "Analyze the following data to determine the user's emotional state: voice data = some_voice_data, facial expression data = some_expression_data. If the user is stressed, respond with 'high_stress'."

[1368] This allows the emotion engine to determine the user's emotional state in real time based on their voice and facial expression data.

[1369] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1370] Step 1: Initialize your device

[1371] When the device starts up, it initializes the temperature sensor, water pressure sensor, sound sensor, and emotion engine. Specifically, it sets the types and initial measurement values ​​of these sensors and engines. The input of this step is the initial setting data for the sensors and engines, and the output is the initialized sensors and engines.

[1372] Step 2: Acquiring Sensor Data

[1373] The terminal acquires data from the temperature sensor, water pressure sensor, and sound sensor at regular intervals (e.g., every second). It acquires the temperature, pressure, and sound level measurements and records them in a database. The input of this step is real-time data from each sensor, and the output is the acquired temperature, pressure, and sound level data.

[1374] Step 3: Collecting emotion data

[1375] The device collects the user's voice and facial expression data. Specifically, it acquires voice and image data from input devices such as microphones and cameras and sends them to the emotion engine. The input to this step is the user's voice and facial expression data, and the output is the data to be analyzed that is sent to the emotion engine.

[1376] Step 4: Sentiment Analysis

[1377] The emotion engine analyzes the collected voice and facial expression data to determine the user's emotional state. Specifically, it analyzes changes in voice tone and facial expressions to assess stress levels. The input for this step is voice and facial expression data, and the output is a judgment result regarding the user's emotional state.

[1378] Step 5: Validate the data

[1379] The server verifies the data obtained from each sensor and the judgment result from the emotion engine. It compares the temperature, water pressure, and sound sensor measurements and the emotion engine's judgment result (e.g., high stress) with a threshold. The input of this step is the sensor data and emotion judgment result, and the output is the detection of an abnormal state that exceeds the threshold.

[1380] Step 6: Generate a warning signal

[1381] The server generates a warning signal when an abnormality is detected. Specifically, it creates a warning message based on temperature, water pressure, sound level, and abnormal emotional state exceeding the set threshold. The input of this step is the abnormality detection result, and the output is the generated warning signal.

[1382] Step 7: Sending a warning signal

[1383] The server sends the generated warning signal to an administrator, for example, by email or a dedicated notification application, notifying the administrator of the warning message in real time. The input of this step is the warning signal, and the output is the warning message sent to the administrator.

[1384] Step 8: Administrator Action

[1385] The administrator receives the notification from the server, checks the situation on-site, and takes appropriate action, such as activating a cooling device in the case of high temperatures, or instructing workers to take breaks in the case of high stress, etc. The input of this step is the warning message sent to the administrator, and the output is the execution of the response action.

[1386] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1387] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1388] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1389] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1390] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1391] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1392] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1393] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1394] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1395] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1396] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1397] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1398] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1399] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1400] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1401] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1402] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1403] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1404] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1405] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1406] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1407] The following is further disclosed regarding the above embodiment.

[1408] (Claim 1)

[1409] a means for measuring the ambient temperature by a temperature sensor;

[1410] a means for measuring water pressure by a water pressure sensor;

[1411] a means for measuring ambient sounds by a sound sensor;

[1412] means for generating a warning signal if the measured temperature exceeds a certain range;

[1413] means for generating a warning signal if the measured water pressure exceeds a certain range;

[1414] means for generating a warning signal if the measured sound exceeds a certain range;

[1415] means for transmitting the generated alert signal to an administrator;

[1416] A system including:

[1417] (Claim 2)

[1418] The system of claim 1 , further comprising means for acquiring data from the temperature sensor, the water pressure sensor, and the sound sensor at regular intervals.

[1419] (Claim 3)

[1420] 10. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to the administrator.

[1421] "Example 1"

[1422] (Claim 1)

[1423] a means for measuring the ambient temperature by a temperature sensor;

[1424] means for measuring pressure by a water pressure sensor;

[1425] a means for measuring ambient sounds by a sound sensor;

[1426] means for generating a warning signal if the measured temperature exceeds a certain range;

[1427] means for generating a warning signal if the measured pressure exceeds a certain range;

[1428] means for generating a warning signal if the measured sound exceeds a certain range;

[1429] means for transmitting the generated alert signal to a remote server;

[1430] a means for the remote server to analyze the warning signal and notify the appropriate administrator;

[1431] A system including:

[1432] (Claim 2)

[1433] The system of claim 1 , further comprising means for acquiring data from the temperature sensor, the pressure sensor, and the sound sensor at regular intervals.

[1434] (Claim 3)

[1435] 10. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to the administrator.

[1436] "Application Example 1"

[1437] (Claim 1)

[1438] a means for measuring the ambient temperature by a temperature sensor;

[1439] a means for measuring water pressure by a water pressure sensor;

[1440] a means for measuring ambient sounds by a sound sensor;

[1441] means for generating a warning signal if the measured temperature exceeds a certain range;

[1442] means for generating a warning signal if the measured water pressure exceeds a certain range;

[1443] means for generating a warning signal if the measured sound exceeds a certain range;

[1444] means for transmitting the generated alert signal to a server;

[1445] means for analyzing the received warning signal by the server and taking appropriate action;

[1446] a means for notifying an administrator;

[1447] A system including:

[1448] (Claim 2)

[1449] 10. The system of claim 1, further comprising means for periodically acquiring data from the temperature sensor, the water pressure sensor, and the sound sensor.

[1450] (Claim 3)

[1451] 10. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to the server.

[1452] "Example 2: Combining Emotion Engines"

[1453] (Claim 1)

[1454] a means for measuring the ambient temperature by a temperature sensor;

[1455] a means for measuring water pressure by a water pressure sensor;

[1456] a means for measuring ambient sounds by a sound sensor;

[1457] an emotion analysis engine as a means of recognizing user emotions;

[1458] means for generating a warning signal if the measured temperature exceeds a certain range;

[1459] means for generating a warning signal if the measured water pressure exceeds a certain range;

[1460] means for generating a warning signal if the measured sound exceeds a certain range;

[1461] means for generating a warning signal when the emotion analysis engine determines that the user is experiencing abnormal emotions (e.g., high stress);

[1462] means for transmitting the generated alert signal to an administrator;

[1463] A system including:

[1464] (Claim 2)

[1465] The system of claim 1 , further comprising means for acquiring data from the temperature sensor, the water pressure sensor, the sound sensor, and the emotion analysis engine at regular intervals.

[1466] (Claim 3)

[1467] 10. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to the administrator.

[1468] "Application example 2 when combining emotion engines"

[1469] (Claim 1)

[1470] a means for measuring the ambient temperature by a temperature sensor;

[1471] a means for measuring water pressure by a water pressure sensor;

[1472] a means for measuring ambient sounds by a sound sensor;

[1473] means including an emotion engine that analyzes voice and facial expression data of a user to determine the user's emotional state;

[1474] means for generating a warning signal if the measured temperature exceeds a certain range;

[1475] means for generating a warning signal if the measured water pressure exceeds a certain range;

[1476] means for generating a warning signal if the measured sound exceeds a certain range;

[1477] means for generating a warning signal when the user's emotional state is determined to be abnormal;

[1478] means for transmitting the generated alert signal to an administrator;

[1479] A system including:

[1480] (Claim 2)

[1481] 10. The system of claim 1, further comprising means for acquiring data from the temperature sensor, the water pressure sensor, the sound sensor, and the emotion engine at regular intervals.

[1482] (Claim 3)

[1483] 10. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to the administrator. [Explanation of symbols]

[1484] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for measuring the ambient temperature by a temperature sensor; a means for measuring water pressure by a water pressure sensor; a means for measuring ambient sounds by a sound sensor; means for generating a warning signal if the measured temperature exceeds a certain range; means for generating a warning signal if the measured water pressure exceeds a certain range; means for generating a warning signal if the measured sound exceeds a certain range; means for transmitting the generated alert signal to an administrator; A system including:

2. The system of claim 1 further comprising means for periodically acquiring data from said temperature sensor, said water pressure sensor, and said sound sensor.

3. 2. The system of claim 1, further comprising means for notifying an administrator based on the alert signal sent to said administrator.

Images