system

Abstract

We provide the system. [Solution] A means of receiving work plan information and environmental information, and analyzing the data to identify potential hazards before starting work, A means of individually identifying and monitoring workers in real time and providing health management information, A means of detecting hazards based on weather information and work operation data and immediately sending out warnings, A means of providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] At a construction site, ensuring the safety of workers is the most important issue. However, current safety management methods have problems such as human errors and lack of immediacy. In particular, it is difficult to grasp and respond to the state of workers and environmental changes in real time, so unexpected situations are likely to occur. In addition, it is difficult to constantly monitor the health status of individual workers, and there is also a risk of inappropriate work being carried out by unqualified personnel. In order to improve such a current situation and achieve more effective safety management, more advanced technical means are required.

Means for Solving the Problems

[0005] This invention provides a means for analyzing work plan information and environmental information to automatically identify potential hazards before work begins. Furthermore, it provides a safety monitoring system equipped with means for individually identifying and monitoring workers in real time to manage their health status, and means for immediately detecting on-site hazards and sending warnings based on weather information and work movement data. In addition, by linking with workers' wearable devices and collecting and analyzing individual biometric information, it is possible to make an integrated judgment on the safety situation. This makes it possible to significantly improve on-site safety.

[0006] "Work plan information" refers to detailed information about the work plan, including procedures, schedules, and equipment to be used when carrying out on-site work.

[0007] "Environmental information" refers to information about external factors that affect work, such as weather conditions, topography, and surrounding environment at the work site.

[0008] "Real-time monitoring" is the process of continuously monitoring target workers and their work status in real time, acquiring and evaluating data.

[0009] "Health management information" refers to information used to understand the health status of workers, including their vital data, work status, and fatigue levels.

[0010] "Weather information" refers to data on weather forecasts and current weather conditions, including weather changes that may affect work safety.

[0011] "Work motion data" refers to data used to record and analyze the movements and actions of workers, and is information used to identify dangerous movements.

[0012] A "wearable device" is a portable device that workers can wear and that is equipped with various sensors to measure their health status and environmental information.

[0013] "Biometric information" refers to data indicating the physical condition of workers, such as heart rate, body temperature, and blood pressure, and this information is used to manage the workers' health. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The present invention is an advanced safety monitoring system for ensuring the safety of workers at construction sites. Its purpose is to monitor the activities of each worker and changes in the environment in real time, and to prevent potential hazards before they occur.

[0036] This system consists of a central server, terminals used by each worker, and wearable devices worn by the workers. The server integrates and processes work plan information, environmental information, and real-time weather information to perform risk analysis.

[0037] The system's main operation involves the server analyzing TBM-KY information entered via voice input before work begins, identifying work procedures and potential hazards, and notifying the terminal. The user then reviews this information on the terminal and prepares for work.

[0038] During work, the server collects video and biometric information from cameras and wearable devices and analyzes it in real time. If the server determines that a worker is fatigued or needs a break, it sends a break recommendation notification to the device. This process allows users to properly manage their own health.

[0039] The server also monitors weather information in real time and sends immediate warnings in the event of sudden weather changes. For example, if sudden strong winds are predicted, it will send a message to users performing work at heights to warn them.

[0040] Furthermore, the system can improve workplace safety by identifying the risks of unqualified personnel performing tasks and issuing warnings as needed.

[0041] The introduction of this system is expected to ensure worker safety and allow work to proceed more efficiently. This invention is an effective means of comprehensively managing on-site risks and improving safety.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server receives the TBM-KY content, which is entered by the user (worker or supervisor) via voice input into the terminal before work begins. Using speech recognition technology, this information is converted into text data.

[0045] Step 2:

[0046] The server integrates and analyzes the converted text data, work plan information, and weather information. This automatically identifies potential hazards and shortcomings.

[0047] Step 3:

[0048] The server generates a list of identified risks and points of caution and notifies the user's terminal. The user then uses this information to prepare and verify the work before proceeding.

[0049] Step 4:

[0050] Once work begins, the server uses cameras and sensors to identify each worker individually and monitors them in real time.

[0051] Step 5:

[0052] The server analyzes each worker's work time and vital data, and if it determines that a break is necessary, it sends a message to the terminal recommending a break.

[0053] Step 6:

[0054] The server constantly checks real-time weather information and immediately issues warnings if there are sudden changes in the environment. For example, if strong winds or heavy rain are forecast, it will alert the affected users.

[0055] Step 7:

[0056] The server analyzes work actions and credentials to verify that no unauthorized personnel are performing inappropriate tasks. If a violation is detected, an immediate warning is issued.

[0057] Step 8:

[0058] Users receive notifications and warnings from their devices and take appropriate action. They review their work procedures and plans as needed.

[0059] This series of processes is expected to significantly improve safety and efficiency at the work site.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Safety management for workers is extremely important at construction sites. Traditional methods make it difficult to monitor potential hazards and workers' health in real time, resulting in a high risk of accidents and health problems. Furthermore, mechanisms to effectively prevent unqualified workers are insufficient. Solving these problems and improving safety is essential.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for receiving work plan data and environmental data and analyzing the information to identify potential risks before work begins; means for individually identifying workers, monitoring them in real time, and providing health management information; means for detecting risks based on weather data and work condition data and immediately sending warnings; and means for identifying unqualified workers and issuing warnings. This makes it possible to enhance worker safety, identify potential risks early, and prevent accidents and health problems.

[0065] "Work plan data" refers to information used to manage the progress of on-site activities, such as work schedules and site layouts at construction sites.

[0066] "Environmental data" refers to information related to the work environment, such as temperature, humidity, noise level, and air quality at the work site.

[0067] "Workers" refer to individuals who actually perform work at a construction site and are managed based on each individual's identification information.

[0068] "Health management information" refers to the health status of workers based on biometric information such as heart rate, blood pressure, and body temperature.

[0069] "Weather data" refers to information about current weather conditions and predicted weather changes.

[0070] "Work status data" refers to information about ongoing work, such as the location and activity status of workers.

[0071] "Wearable devices" refer to wearable devices that workers wear and use to measure and monitor biometric information.

[0072] "Audio information" refers to the voices emitted by workers, which are analyzed to generate work procedures and precautions.

[0073] An "unqualified person" refers to a person who does not possess the necessary qualifications to perform a particular task.

[0074] This invention relates to a safety monitoring system for enhancing worker safety. The system consists of a server, a terminal for receiving and notifying data, and a wearable device for collecting information.

[0075] The server aggregates business plan data and environmental data using a database management system. For example, PostgreSQL can be used as the database management system. In addition, the server implements speech recognition software to analyze speech information and converts speech data into text by utilizing cloud-based APIs. Google® Cloud Speech-to-Text API is one example.

[0076] The terminals serve to relay warnings and notifications sent from the server to workers, allowing them to monitor work status in real time. For example, if a worker's health management information exceeds a certain threshold, a warning will be displayed on the terminal.

[0077] The wearable devices are designed to monitor the health of workers and collect biometric data such as heart rate and body temperature. This information is transmitted to a server via Bluetooth and analyzed by an AI model.

[0078] For example, a server can detect a sudden change in weather and send a warning message to a terminal such as, "Strong winds are predicted. Please temporarily suspend work at heights and ensure your safety." This allows users to take appropriate action in a timely manner.

[0079] An example of a prompt for a generated AI model is, "Please tell me about the effects of strong winds and safety measures during work at heights on construction sites."

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The server retrieves work plan data and environmental data from the database. This input data includes work schedules and environmental conditions. The server uses data analysis software to analyze the data to identify potential risks. The analysis results in a list of risky processes and insufficient safety equipment.

[0083] Step 2:

[0084] The server receives voice information transmitted from the user via the terminal. This voice information includes TBM-KY information, which is converted into text data by speech recognition software. This converted text data is then analyzed using natural language processing (NLP) techniques to identify work procedures and potential hazards. The server sends this identified information to the terminal as an output list.

[0085] Step 3:

[0086] The server receives biometric data from wearable devices via Bluetooth. This biometric data includes health information such as heart rate and body temperature. The server analyzes the data using a generative AI model and compares it to calculated thresholds. If an abnormality in the health condition is detected, the server sends a rest recommendation notification to the terminal.

[0087] Step 4:

[0088] The server obtains weather information in real time from a weather data API. This input data includes current weather conditions and near-future weather forecasts. Based on this data, the server uses an AI model to analyze and identify risky weather conditions (e.g., strong winds and thunderstorms). It immediately generates a warning message and notifies the terminal as output.

[0089] Step 5:

[0090] The server accesses the worker qualification database to verify that no unqualified individuals are performing work. The input data is the workers' qualification information. The server checks this information and performs comparison and analysis to identify unqualified individuals. If a violation is detected, a warning message is sent to the terminal as output.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] Effective safety management at work sites requires immediate understanding of individual worker conditions and changes in the external environment, and the implementation of appropriate countermeasures. However, conventional safety monitoring systems have limitations in providing information efficiently in real time, making it difficult for workers to quickly understand and respond to on-site conditions. Therefore, a new solution is needed that simultaneously improves worker safety and work efficiency.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information; means for detecting hazards based on weather information and work operation data and immediately transmitting warnings; and means for providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site. This makes it possible for workers to carry out their work efficiently while ensuring safety.

[0096] "Work plan information" is a systematic compilation of information regarding the procedures, schedules, materials, and equipment necessary to carry out a task.

[0097] "Environmental information" refers to data that indicates external conditions that may affect work, such as temperature, humidity, illuminance, and noise levels at the work site.

[0098] "Means of analyzing data to identify potential hazards" refers to the process of identifying and evaluating predicted risks and hazardous factors based on various information collected before the start of work.

[0099] "A means of individually identifying and monitoring workers in real time and providing health management information" refers to a system that identifies each worker, measures their current health status and workload, and provides advice and instructions based on the results.

[0100] "A means of detecting hazards based on weather information and work movement data and immediately sending warnings" refers to a method of monitoring current weather conditions and worker movements in real time and promptly issuing warnings when abnormalities or hazards are detected.

[0101] "Means of providing information in real time using visual display devices, enabling workers to quickly grasp the situation on site" refers to a function that instantly transmits important work-related information to workers via visual devices to support situational judgment.

[0102] A system implementing this invention includes a server, a terminal, and a visual display device such as smart glasses. The server plays a central role, receiving work plan information, environmental information, and real-time weather information to identify potential hazards. The server analyzes this data and uses advanced algorithms to detect high-risk situations.

[0103] The server also collects biometric data from wearable devices to monitor individual workers in real time. This data includes heart rate, movement patterns, and location information, which is used to assess the workers' health and safety status.

[0104] The terminal (smart glasses) receives analysis results transmitted from the server and provides workers with real-time warnings and health management information. The terminal uses a visual user interface to instantly communicate changes in the situation, enabling workers to respond quickly.

[0105] For example, if a worker is working at a height and a sudden strong wind is detected approaching, the server analyzes the situation and instantly sends a warning to the smart glasses. This allows the worker to quickly evacuate to a safe location. Additionally, if the heart rate exceeds a certain level, a visual notification prompting a break is automatically displayed.

[0106] Furthermore, this system can also analyze information received via voice input, allowing users to review work procedures and precautions in advance. For example, pre-start inspection procedures can be reviewed by voice to prevent oversights and misunderstandings.

[0107] An example of a prompt would be, "To brainstorm ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers using smart glasses to improve safety?" This prompt allows the generated AI model to suggest more specific advice and functions.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The server receives work plan information, environmental information, and real-time weather information. Based on this information, it builds an initial dataset and begins analysis to identify potential risks. Statistical models and AI algorithms are used in the analysis to determine the priority of risk factors. The input data mainly consists of sensor data and output from predictive models, and the output generates a list of high-risk scenarios.

[0111] Step 2:

[0112] The server acquires and analyzes real-time biometric data for each worker collected from wearable devices. The main input data includes heart rate, activity level, and geographical location information. Using this data, the server evaluates the worker's physical condition and safety risks, and sets a warning level if an abnormality is detected. The analysis results output an evaluation of the worker's safety status.

[0113] Step 3:

[0114] The server evaluates the dynamic conditions at the site based on weather information and work operation data. If a sudden weather change or abnormal operation pattern is detected, the server immediately sends a warning. Weather sensor data and camera images are used as input, and weather simulation and operation analysis algorithms are applied. The output includes specific warning content and recommended actions.

[0115] Step 4:

[0116] The terminal receives information transmitted from the server and displays it in real time on a visual display device. The terminal visually informs the worker of recognized hazards and health management information. The input information consists of analyzed data and notification content, and the output is visualized warning messages and instructions. Based on this, the worker immediately selects a response action.

[0117] Step 5:

[0118] Users interact with the system through voice input. The server analyzes the voice data and provides feedback to the user regarding relevant work procedures and points to note. This process utilizes speech recognition technology, where the input is the voice data itself, and the output is specific work instructions and improvement suggestions.

[0119] Step 6:

[0120] During the process, the server utilizes an AI model to generate reactions to dynamic changes in the situation on-site. The prompt used for the model is, "To consider ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers via smart glasses to improve safety?" This question then generates specific preventative measures. The input here is the prompt statement, and the output is a guideline for preventative action.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention combines an emotional engine with a safety monitoring system designed to thoroughly ensure the safety of workers at construction sites. This system can consider not only the physical safety of workers but also their mental health.

[0123] The system consists of a server, terminals used by each worker, and wearable devices worn by the workers. The server analyzes work plan information and environmental information to identify risks before work begins, and also identifies and monitors individual workers in real time during work. Specifically, it uses video and audio data acquired from cameras and wearables to identify the emotional state of the users.

[0124] The emotion engine analyzes changes in voice tone and facial expressions to determine whether the user is stressed or focused, and the server generates appropriate feedback based on this. For example, if the emotion engine detects "fatigue" from the user's voice, the server sends a message to the device recommending a break.

[0125] Furthermore, the server can use sentiment data to perform risk assessments and recommend interrupting work if it is deemed dangerous. In this way, it helps maintain and improve team dynamics on-site while ensuring user safety.

[0126] For example, if a worker appears to be experiencing stress while working at height, the system can detect that emotion and send a notification such as, "We recommend taking a break for a few minutes," thereby protecting the worker's safety. By utilizing the emotion engine, it is also possible to address new risks on the job site.

[0127] This invention makes it possible to comprehensively manage on-site risks from multiple perspectives and improve safety and efficiency.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] Before starting work, the user inputs the content of the TBM-KY (Toolbox Meeting / Hazard Prediction) into a terminal via voice input and sends it to the server. The server receives this voice data.

[0131] Step 2:

[0132] The server converts the received audio data into text data using speech recognition technology and analyzes it along with work plan information and environmental information. This allows it to identify potential hazards before work begins and send notifications of any deficiencies to the user's terminal.

[0133] Step 3:

[0134] Once work begins, the server collects real-time video and audio data from cameras and wearable devices. This allows for the identification of individual workers and the monitoring of their movements and biometric information.

[0135] Step 4:

[0136] The server applies an emotion engine to the collected video and audio data, analyzing the user's emotional state from their voice tone and facial expressions. For example, it can detect "fatigue" or "stress" from their voice tone.

[0137] Step 5:

[0138] Based on the analysis results of the emotion engine, the server evaluates the user's physical and mental state and sends a message to the terminal recommending a break if necessary.

[0139] Step 6:

[0140] The server analyzes weather information and work operation data, and if it detects a danger in real time, it immediately sends a warning to the terminal. Sentiment analysis results are also taken into consideration during this process.

[0141] Step 7:

[0142] The device displays received notifications and warnings to the user and provides an interface for the user to take appropriate action. The user reviews this and adjusts or interrupts their work as needed.

[0143] This series of processes enables the system to achieve comprehensive safety management and reduce risks at the work site.

[0144] (Example 2)

[0145] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0146] On-site work requires not only physical safety but also the mental health of workers. However, currently, there is a lack of systems to comprehensively manage these factors and intervene at the appropriate time. In particular, detecting workers' stress levels and decreased attention in real time and taking appropriate action based on that is a challenge. Furthermore, making decisions about work interruptions or proposing improvement measures based on emotional states is also difficult.

[0147] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0148] In this invention, the server includes means for receiving work information and environmental information and analyzing the information to identify risks before work begins; means for individually identifying and monitoring workers in real time and evaluating their health status; means for analyzing emotional data, determining stress and attention levels, and generating optimal feedback; and means for recommending work interruptions based on the results of the emotional analysis. This enables comprehensive management of the physical and mental safety of workers and provides an efficient work environment while reducing risks at the worksite.

[0149] "Work information" refers to all information related to the work, such as work plans and progress.

[0150] "Environmental information" refers to information related to the work environment, such as temperature, humidity, and noise levels at the work site.

[0151] "Risk" refers to the dangers and problems that may arise during the work.

[0152] "Means of analysis" refers to the techniques and methods used to analyze received information and derive meaningful conclusions or judgments from it.

[0153] "Workers" refers to the personnel who are actually performing the work on-site.

[0154] "Monitoring methods" refer to technologies and devices for observing and recording the condition of workers and the work environment in real time.

[0155] "Health status" refers to the overall physical and mental condition of a worker.

[0156] "Emotional data" refers to information about emotions obtained from the worker's voice tone, facial expressions, and other similar data.

[0157] "Feedback" refers to responses that provide workers with appropriate solutions or warnings for improvement.

[0158] "Means of recommending interruption" refers to methods or techniques for suggesting to workers to temporarily suspend their work in specific situations.

[0159] The embodiment for carrying out the invention is a safety monitoring system for ensuring the safety of workers at a construction site. This system consists of a server, a worker terminal, and a wearable terminal.

[0160] The server receives integrated work and environmental information and analyzes this data to identify risks before work begins. The server is equipped with high-performance computing devices, which are used to analyze video and audio data to assess workers' health and stress levels. Specific software examples include emotion recognition engines specialized in audio analysis and facial expression recognition technology based on image analysis.

[0161] Each worker's device receives feedback from the server and recommends the most appropriate action for the worker. For example, if the emotion engine detects a "fatigue state," the device will display "We suggest a break" and prompt the worker to take a break.

[0162] The user, a worker, wears a wearable device that transmits data from heart rate and motion detection sensors to a server. Based on this information, the server monitors the worker's stress level in real time.

[0163] For example, if a worker's stress level is detected to be rising while working at a high position, the emotion engine will determine that "rest is needed." This will then display a message on the worker's device saying, "We recommend taking a break for a few minutes."

[0164] Furthermore, as an example of a prompt, the question, "What emotional factors should be considered to reduce risks in the work environment?" can be input into the generating AI model to find its own solutions. In this way, the invention provides a system that manages not only physical safety but also mental health, achieving both work efficiency and safety.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The server receives work information and environmental information. Input information includes work plans, weather conditions in the surrounding environment, and noise levels. This data is analyzed to identify potential risks. Pattern recognition algorithms are used in the data analysis to identify conditions deemed particularly dangerous. Specifically, the server uses statistical models to detect anomalies in weather conditions and generates a risk list that proposes preventative measures.

[0168] Step 2:

[0169] The server acquires data to individually identify workers. Inputs include camera footage and biometric information from wearables. Based on this information, the server monitors the workers' health in real time and detects abnormal heart rates and behavioral patterns. Machine learning algorithms are used for data processing. Specifically, when the server detects an anomaly, it sends a warning message to the worker's device.

[0170] Step 3:

[0171] The server analyzes emotional data. Input data includes voice tone and facial expression data. It activates an emotion engine to determine stress levels and concentration levels. The resulting output is an evaluation of the worker's emotional state. Specifically, the server analyzes changes in pitch and speed of the voice data to determine if the user is tense.

[0172] Step 4:

[0173] The server generates feedback based on the sentiment analysis results. The evaluation results from step 3 are used as input. In the feedback generation process, a generative AI model is used to formulate the optimal response and message. The output is a specific action instruction delivered to the worker's terminal. For example, a message such as "Fatigue has been detected. Taking a break is recommended" is generated.

[0174] Step 5:

[0175] The server continuously monitors the data and verifies the effectiveness of the feedback. New data is used as input, and the process from steps 2 to 4 is repeated to confirm improvements in the worker's condition. Specifically, the server periodically resumes data collection and performs recalculations to mitigate risks.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] Ensuring worker safety at construction sites requires not only monitoring physical hazards but also considering the mental health of workers. However, conventional safety monitoring systems lacked the means to accurately assess workers' emotions and stress levels and provide appropriate feedback, making it difficult to comprehensively improve worker safety and efficiency.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information and emotional status; and means for analyzing changes in voice tone and facial expressions to determine the emotional status of workers and generate appropriate feedback based on that. This enables comprehensive safety management that takes into account the mental health of workers while ensuring their physical safety.

[0181] "Work plan information" refers to data on the processes and schedules necessary for the progress of a construction site, and is used as the basis for safety monitoring.

[0182] "Environmental information" refers to data on weather conditions such as temperature, humidity, atmospheric pressure, and wind speed at the work site, and is an important element for evaluating safety.

[0183] "Emotional state" refers to the mental state of a worker, encompassing emotional factors such as stress levels and concentration levels.

[0184] "Real-time monitoring" is a process of instantly monitoring and analyzing data, and is a technology for continuously understanding the latest status of workers.

[0185] "Feedback" refers to information and instructions provided to workers based on the analysis results performed by the system, with the aim of improving work safety and efficiency.

[0186] A "wearable device" is an electronic device that can be worn by workers and is used to collect biometric information and emotional data in real time.

[0187] A "safety monitoring system" is a comprehensive system designed to ensure the safety of workers and support efficient work, and it evaluates and manages risks based on the results of analysis of various data.

[0188] The system that realizes this invention is an integrated safety monitoring system designed to ensure the safety and mental health of workers at construction sites. The system's main hardware includes a server, terminals used by workers (such as smartphones and smart glasses), and wearable devices.

[0189] The server first receives work plan and environmental information, and identifies potential hazards through data analysis. This enables risk assessment before work begins. The server also individually identifies each worker, monitors them in real time, and provides health management information and emotional status. Specifically, emotion analysis libraries such as EmotionEngine are used to analyze changes in voice tone and facial expressions to determine the worker's stress level and concentration level.

[0190] The terminal functions as a device that displays server-generated feedback to the worker. For example, if a worker is feeling anxious while working at height, the terminal will display a message such as, "Take a deep breath to ease your tension."

[0191] Wearable devices collect real-time biometric information from workers and integrate it with a server to determine the overall safety situation. This allows for the sending of warnings when immediate action is required, facilitating a rapid response.

[0192] For example, when a worker is working under harsh weather conditions, the system can assess the risks based on environmental information and issue instructions such as, "We recommend taking a break for a few minutes."

[0193] An example of a prompt for a generated AI model would be: "Please provide a detailed scenario for a system that combines safety monitoring and sentiment analysis at a construction site. Explain how the emotional state of workers affects safety assessments." Using this prompt, detailed recommended strategies and operational scenarios for the system can be generated.

[0194] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0195] Step 1:

[0196] The server receives work plan information and environmental information. Based on this information, it performs data analysis to identify potential hazards. Inputs include work process data and weather data. Outputs include a hazard assessment report. These analyses utilize statistical methods and machine learning models to comprehensively evaluate on-site risks.

[0197] Step 2:

[0198] The server receives real-time biometric and emotional data from wearable devices. Based on this data, it assesses the worker's stress level and concentration level, and monitors their health. Inputs include heart rate, body temperature, and voice tone, and output is a health report for each worker. The server uses software such as EmotionEngine to perform emotional analysis.

[0199] Step 3:

[0200] The user (worker) receives feedback on a terminal. The server generates feedback and sends it to the terminal, where it is displayed on the worker's smart glasses or smartphone. The input is the generated feedback message, and the output is visual or auditory instructions. Examples include recommendations for breaks or instructions on when to resume work.

[0201] Step 4:

[0202] The server detects danger in real time and immediately sends a warning to the terminal. Inputs are continuously acquired biometric and environmental information, and outputs are warning messages and instructions. This process uses algorithms such as decision trees to make dynamic decisions. For example, if weather conditions rapidly deteriorate, it will notify the user with an instruction such as, "Please temporarily suspend work."

[0203] Step 5:

[0204] The user selects the next action based on system feedback. The input is the instruction received from the terminal, and the output is the action the user actually takes. Here, the user's judgment is a crucial factor in safety. Specific examples include actions such as the user taking a recommended break for safety reasons.

[0205] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0206] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0207] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0208] [Second Embodiment]

[0209] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0210] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0211] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0212] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0213] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0214] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0215] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0216] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0217] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0218] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0219] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0220] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0221] The present invention is an advanced safety monitoring system for ensuring the safety of workers at construction sites. Its purpose is to monitor the activities of each worker and changes in the environment in real time, and to prevent potential hazards before they occur.

[0222] This system consists of a central server, terminals used by each worker, and wearable devices worn by the workers. The server integrates and processes work plan information, environmental information, and real-time weather information to perform risk analysis.

[0223] The system's main operation involves the server analyzing TBM-KY information entered via voice input before work begins, identifying work procedures and potential hazards, and notifying the terminal. The user then reviews this information on the terminal and prepares for work.

[0224] During work, the server collects video and biometric information from cameras and wearable devices and analyzes it in real time. If the server determines that a worker is fatigued or needs a break, it sends a break recommendation notification to the device. This process allows users to properly manage their own health.

[0225] The server also monitors weather information in real time and sends immediate warnings in the event of sudden weather changes. For example, if sudden strong winds are predicted, it will send a message to users performing work at heights to warn them.

[0226] Furthermore, the system can improve workplace safety by identifying the risks of unqualified personnel performing tasks and issuing warnings as needed.

[0227] The introduction of this system is expected to ensure worker safety and allow work to proceed more efficiently. This invention is an effective means of comprehensively managing on-site risks and improving safety.

[0228] The following describes the processing flow.

[0229] Step 1:

[0230] The server receives the TBM-KY content, which is entered by the user (worker or supervisor) via voice input into the terminal before work begins. Using speech recognition technology, this information is converted into text data.

[0231] Step 2:

[0232] The server integrates and analyzes the converted text data, work plan information, and weather information. This automatically identifies potential hazards and shortcomings.

[0233] Step 3:

[0234] The server generates a list of identified risks and points of caution and notifies the user's terminal. The user then uses this information to prepare and verify the work before proceeding.

[0235] Step 4:

[0236] Once work begins, the server uses cameras and sensors to identify each worker individually and monitors them in real time.

[0237] Step 5:

[0238] The server analyzes each worker's work time and vital data, and if it determines that a break is necessary, it sends a message to the terminal recommending a break.

[0239] Step 6:

[0240] The server constantly checks real-time weather information and immediately issues warnings if there are sudden changes in the environment. For example, if strong winds or heavy rain are forecast, it will alert the affected users.

[0241] Step 7:

[0242] The server analyzes work actions and credentials to verify that no unauthorized personnel are performing inappropriate tasks. If a violation is detected, an immediate warning is issued.

[0243] Step 8:

[0244] Users receive notifications and warnings from their devices and take appropriate action. They review their work procedures and plans as needed.

[0245] This series of processes is expected to significantly improve safety and efficiency at the work site.

[0246] (Example 1)

[0247] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0248] Safety management for workers is extremely important at construction sites. Traditional methods make it difficult to monitor potential hazards and workers' health in real time, resulting in a high risk of accidents and health problems. Furthermore, mechanisms to effectively prevent unqualified workers are insufficient. Solving these problems and improving safety is essential.

[0249] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0250] In this invention, the server includes means for receiving work plan data and environmental data and analyzing the information to identify potential risks before work begins; means for individually identifying workers, monitoring them in real time, and providing health management information; means for detecting risks based on weather data and work condition data and immediately sending warnings; and means for identifying unqualified workers and issuing warnings. This makes it possible to enhance worker safety, identify potential risks early, and prevent accidents and health problems.

[0251] "Work plan data" refers to information used to manage the progress of on-site activities, such as work schedules and site layouts at construction sites.

[0252] "Environmental data" refers to information related to the work environment, such as temperature, humidity, noise level, and air quality at the work site.

[0253] "Workers" refer to individuals who actually perform work at a construction site and are managed based on each individual's identification information.

[0254] "Health management information" refers to the health status of workers based on biometric information such as heart rate, blood pressure, and body temperature.

[0255] "Weather data" refers to information about current weather conditions and predicted weather changes.

[0256] "Work status data" refers to information about ongoing work, such as the location and activity status of workers.

[0257] "Wearable devices" refer to wearable devices that workers wear and use to measure and monitor biometric information.

[0258] "Audio information" refers to the voices emitted by workers, which are analyzed to generate work procedures and precautions.

[0259] An "unqualified person" refers to a person who does not possess the necessary qualifications to perform a particular task.

[0260] This invention relates to a safety monitoring system for enhancing worker safety. The system consists of a server, a terminal for receiving and notifying data, and a wearable device for collecting information.

[0261] The server aggregates business plan data and environmental data using a database management system. For example, PostgreSQL can be used as the database management system. Furthermore, the server implements speech recognition software to analyze audio information and converts audio data to text using cloud-based APIs. Google Cloud Speech-to-Text API is one example.

[0262] The terminals serve to relay warnings and notifications sent from the server to workers, allowing them to monitor work status in real time. For example, if a worker's health management information exceeds a certain threshold, a warning will be displayed on the terminal.

[0263] The wearable devices are designed to monitor the health of workers and collect biometric data such as heart rate and body temperature. This information is transmitted to a server via Bluetooth and analyzed by an AI model.

[0264] For example, a server can detect a sudden change in weather and send a warning message to a terminal such as, "Strong winds are predicted. Please temporarily suspend work at heights and ensure your safety." This allows users to take appropriate action in a timely manner.

[0265] An example of a prompt for a generated AI model is, "Please tell me about the effects of strong winds and safety measures during work at heights on construction sites."

[0266] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0267] Step 1:

[0268] The server retrieves work plan data and environmental data from the database. This input data includes work schedules and environmental conditions. The server uses data analysis software to analyze the data to identify potential risks. The analysis results in a list of risky processes and insufficient safety equipment.

[0269] Step 2:

[0270] The server receives voice information transmitted from the user via the terminal. This voice information includes TBM-KY information, which is converted into text data by speech recognition software. This converted text data is then analyzed using natural language processing (NLP) techniques to identify work procedures and potential hazards. The server sends this identified information to the terminal as an output list.

[0271] Step 3:

[0272] The server receives biometric data from wearable devices via Bluetooth. This biometric data includes health information such as heart rate and body temperature. The server analyzes the data using a generative AI model and compares it to calculated thresholds. If an abnormality in the health condition is detected, the server sends a rest recommendation notification to the terminal.

[0273] Step 4:

[0274] The server obtains weather information in real time from a weather data API. This input data includes current weather conditions and near-future weather forecasts. Based on this data, the server uses an AI model to analyze and identify risky weather conditions (e.g., strong winds and thunderstorms). It immediately generates a warning message and notifies the terminal as output.

[0275] Step 5:

[0276] The server accesses the worker qualification database to verify that no unqualified individuals are performing work. The input data is the workers' qualification information. The server checks this information and performs comparison and analysis to identify unqualified individuals. If a violation is detected, a warning message is sent to the terminal as output.

[0277] (Application Example 1)

[0278] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] Effective safety management at work sites requires immediate understanding of individual worker conditions and changes in the external environment, and the implementation of appropriate countermeasures. However, conventional safety monitoring systems have limitations in providing information efficiently in real time, making it difficult for workers to quickly understand and respond to on-site conditions. Therefore, a new solution is needed that simultaneously improves worker safety and work efficiency.

[0280] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0281] In this invention, the server includes means for receiving work plan information and environmental information, analyzing data to identify potential risks before work starts, means for individually identifying workers, monitoring them in real time, and providing health management information, means for detecting risks based on weather information and work operation data and immediately sending warnings, and means for providing information in real time using a visual display device to enable workers to quickly grasp the on-site situation. As a result, it becomes possible for workers to efficiently carry out their work while ensuring safety.

[0282] "Work plan information" is a systematic compilation of information on the procedures, schedules, materials, and equipment required to perform work.

[0283] "Environmental information" is data indicating external conditions that may affect work, such as the temperature, humidity, illuminance, and noise level at the work site.

[0284] "Means for analyzing data to identify potential risks" refers to a process for identifying and evaluating predicted risks and risk factors based on various information collected before work starts.

[0285] "Means for individually identifying workers, monitoring them in real time, and providing health management information" is a mechanism for identifying each worker, measuring their current health status and work load, and providing advice and instructions based on the results.

[0286] "Means for detecting risks based on weather information and work operation data and immediately sending warnings" is a method for monitoring current weather conditions and the movements of workers in real time and quickly prompting attention when abnormalities or risks are detected.

[0287] "Means of providing information in real time using visual display devices, enabling workers to quickly grasp the situation on site" refers to a function that instantly transmits important work-related information to workers via visual devices to support situational judgment.

[0288] A system implementing this invention includes a server, a terminal, and a visual display device such as smart glasses. The server plays a central role, receiving work plan information, environmental information, and real-time weather information to identify potential hazards. The server analyzes this data and uses advanced algorithms to detect high-risk situations.

[0289] The server also collects biometric data from wearable devices to monitor individual workers in real time. This data includes heart rate, movement patterns, and location information, which is used to assess the workers' health and safety status.

[0290] The terminal (smart glasses) receives analysis results transmitted from the server and provides workers with real-time warnings and health management information. The terminal uses a visual user interface to instantly communicate changes in the situation, enabling workers to respond quickly.

[0291] For example, if a worker is working at a height and a sudden strong wind is detected approaching, the server analyzes the situation and instantly sends a warning to the smart glasses. This allows the worker to quickly evacuate to a safe location. Additionally, if the heart rate exceeds a certain level, a visual notification prompting a break is automatically displayed.

[0292] Furthermore, this system can also analyze information received via voice input, allowing users to review work procedures and precautions in advance. For example, pre-start inspection procedures can be reviewed by voice to prevent oversights and misunderstandings.

[0293] An example of a prompt would be, "To brainstorm ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers using smart glasses to improve safety?" This prompt allows the generated AI model to suggest more specific advice and functions.

[0294] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0295] Step 1:

[0296] The server receives work plan information, environmental information, and real-time weather information. Based on this information, it builds an initial dataset and begins analysis to identify potential risks. Statistical models and AI algorithms are used in the analysis to determine the priority of risk factors. The input data mainly consists of sensor data and output from predictive models, and the output generates a list of high-risk scenarios.

[0297] Step 2:

[0298] The server acquires and analyzes real-time biometric data for each worker collected from wearable devices. The main input data includes heart rate, activity level, and geographical location information. Using this data, the server evaluates the worker's physical condition and safety risks, and sets a warning level if an abnormality is detected. The analysis results output an evaluation of the worker's safety status.

[0299] Step 3:

[0300] The server evaluates the dynamic conditions at the site based on weather information and work operation data. If a sudden weather change or abnormal operation pattern is detected, the server immediately sends a warning. Weather sensor data and camera images are used as input, and weather simulation and operation analysis algorithms are applied. The output includes specific warning content and recommended actions.

[0301] Step 4:

[0302] The terminal receives the information sent from the server and presents it to the visual display device in real time. The terminal visually notifies the operator of the recognized danger and health management information. The input information is the analyzed data and notification content, and the output is the visualized warning message and instruction. Based on this, the operator immediately selects a corresponding action.

[0303] Step 5:

[0304] The user interacts with the system through voice input. The server analyzes the voice data and feedbacks the relevant work procedures and precautions to the user. In this process, voice recognition technology is used. The input is the voice data itself, and specific work instructions and improvement suggestions are generated as the output.

[0305] Step 6:

[0306] During the operation, the server utilizes the AI model to generate reactions to the dynamic situation changes on-site. As a prompt to the model, "To consider the idea of a real-time safety monitoring system at the construction site, what information should be provided to the operator by smart glasses to contribute to the improvement of safety?" is used, and specific preventive measures are generated from this question. Here, the input is the prompt text, and the output is the guideline for preventive actions.

[0307] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0308] The present invention combines an emotion engine with a safety monitoring system for thoroughly ensuring the safety of workers at the construction site. This system can consider not only the physical safety of workers but also their mental health status.

[0309] The system consists of a server, terminals used by each worker, and wearable devices worn by the workers. The server analyzes work plan information and environmental information to identify risks before work begins, and also identifies and monitors individual workers in real time during work. Specifically, it uses video and audio data acquired from cameras and wearables to identify the emotional state of the users.

[0310] The emotion engine analyzes changes in voice tone and facial expressions to determine whether the user is stressed or focused, and the server generates appropriate feedback based on this. For example, if the emotion engine detects "fatigue" from the user's voice, the server sends a message to the device recommending a break.

[0311] Furthermore, the server can use sentiment data to perform risk assessments and recommend interrupting work if it is deemed dangerous. In this way, it helps maintain and improve team dynamics on-site while ensuring user safety.

[0312] For example, if a worker appears to be experiencing stress while working at height, the system can detect that emotion and send a notification such as, "We recommend taking a break for a few minutes," thereby protecting the worker's safety. By utilizing the emotion engine, it is also possible to address new risks on the job site.

[0313] This invention makes it possible to comprehensively manage on-site risks from multiple perspectives and improve safety and efficiency.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] Before starting work, the user inputs the content of the TBM-KY (Toolbox Meeting / Hazard Prediction) into a terminal via voice input and sends it to the server. The server receives this voice data.

[0317] Step 2:

[0318] The server converts the received audio data into text data using speech recognition technology and analyzes it along with work plan information and environmental information. This allows it to identify potential hazards before work begins and send notifications of any deficiencies to the user's terminal.

[0319] Step 3:

[0320] Once work begins, the server collects real-time video and audio data from cameras and wearable devices. This allows for the identification of individual workers and the monitoring of their movements and biometric information.

[0321] Step 4:

[0322] The server applies an emotion engine to the collected video and audio data, analyzing the user's emotional state from their voice tone and facial expressions. For example, it can detect "fatigue" or "stress" from their voice tone.

[0323] Step 5:

[0324] Based on the analysis results of the emotion engine, the server evaluates the user's physical and mental state and sends a message to the terminal recommending a break if necessary.

[0325] Step 6:

[0326] The server analyzes weather information and work operation data, and if it detects a danger in real time, it immediately sends a warning to the terminal. Sentiment analysis results are also taken into consideration during this process.

[0327] Step 7:

[0328] The device displays received notifications and warnings to the user and provides an interface for the user to take appropriate action. The user reviews this and adjusts or interrupts their work as needed.

[0329] This series of processes enables the system to achieve comprehensive safety management and reduce risks at the work site.

[0330] (Example 2)

[0331] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0332] On-site work requires not only physical safety but also the mental health of workers. However, currently, there is a lack of systems to comprehensively manage these factors and intervene at the appropriate time. In particular, detecting workers' stress levels and decreased attention in real time and taking appropriate action based on that is a challenge. Furthermore, making decisions about work interruptions or proposing improvement measures based on emotional states is also difficult.

[0333] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0334] In this invention, the server includes means for receiving work information and environmental information and analyzing the information to identify risks before work begins; means for individually identifying and monitoring workers in real time and evaluating their health status; means for analyzing emotional data, determining stress and attention levels, and generating optimal feedback; and means for recommending work interruptions based on the results of the emotional analysis. This enables comprehensive management of the physical and mental safety of workers and provides an efficient work environment while reducing risks at the worksite.

[0335] "Work information" refers to all information related to the work, such as work plans and progress.

[0336] "Environmental information" refers to information related to the work environment, such as temperature, humidity, and noise levels at the work site.

[0337] "Risk" refers to the dangers and problems that may arise during the work.

[0338] "Means of analysis" refers to the techniques and methods used to analyze received information and derive meaningful conclusions or judgments from it.

[0339] "Workers" refers to the personnel who are actually performing the work on-site.

[0340] "Monitoring methods" refer to technologies and devices for observing and recording the condition of workers and the work environment in real time.

[0341] "Health status" refers to the overall physical and mental condition of a worker.

[0342] "Emotional data" refers to information about emotions obtained from the worker's voice tone, facial expressions, and other similar data.

[0343] "Feedback" refers to responses that provide workers with appropriate solutions or warnings for improvement.

[0344] "Means of recommending interruption" refers to methods or techniques for suggesting to workers to temporarily suspend their work in specific situations.

[0345] The embodiment for carrying out the invention is a safety monitoring system for ensuring the safety of workers at a construction site. This system consists of a server, a worker terminal, and a wearable terminal.

[0346] The server receives integrated work and environmental information and analyzes this data to identify risks before work begins. The server is equipped with high-performance computing devices, which are used to analyze video and audio data to assess workers' health and stress levels. Specific software examples include emotion recognition engines specialized in audio analysis and facial expression recognition technology based on image analysis.

[0347] Each worker's device receives feedback from the server and recommends the most appropriate action for the worker. For example, if the emotion engine detects a "fatigue state," the device will display "We suggest a break" and prompt the worker to take a break.

[0348] The user, a worker, wears a wearable device that transmits data from heart rate and motion detection sensors to a server. Based on this information, the server monitors the worker's stress level in real time.

[0349] For example, if a worker's stress level is detected to be rising while working at a high position, the emotion engine will determine that "rest is needed." This will then display a message on the worker's device saying, "We recommend taking a break for a few minutes."

[0350] Furthermore, as an example of a prompt, the question, "What emotional factors should be considered to reduce risks in the work environment?" can be input into the generating AI model to find its own solutions. In this way, the invention provides a system that manages not only physical safety but also mental health, achieving both work efficiency and safety.

[0351] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0352] Step 1:

[0353] The server receives work information and environmental information. Input information includes work plans, weather conditions in the surrounding environment, and noise levels. This data is analyzed to identify potential risks. Pattern recognition algorithms are used in the data analysis to identify conditions deemed particularly dangerous. Specifically, the server uses statistical models to detect anomalies in weather conditions and generates a risk list that proposes preventative measures.

[0354] Step 2:

[0355] The server acquires data to individually identify workers. Inputs include camera footage and biometric information from wearables. Based on this information, the server monitors the workers' health in real time and detects abnormal heart rates and behavioral patterns. Machine learning algorithms are used for data processing. Specifically, when the server detects an anomaly, it sends a warning message to the worker's device.

[0356] Step 3:

[0357] The server analyzes emotional data. Input data includes voice tone and facial expression data. It activates an emotion engine to determine stress levels and concentration levels. The resulting output is an evaluation of the worker's emotional state. Specifically, the server analyzes changes in pitch and speed of the voice data to determine if the user is tense.

[0358] Step 4:

[0359] The server generates feedback based on the sentiment analysis results. The evaluation results from step 3 are used as input. In the feedback generation process, a generative AI model is used to formulate the optimal response and message. The output is a specific action instruction delivered to the worker's terminal. For example, a message such as "Fatigue has been detected. Taking a break is recommended" is generated.

[0360] Step 5:

[0361] The server continuously monitors the data and verifies the effectiveness of the feedback. New data is used as input, and the process from steps 2 to 4 is repeated to confirm improvements in the worker's condition. Specifically, the server periodically resumes data collection and performs recalculations to mitigate risks.

[0362] (Application Example 2)

[0363] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0364] Ensuring worker safety at construction sites requires not only monitoring physical hazards but also considering the mental health of workers. However, conventional safety monitoring systems lacked the means to accurately assess workers' emotions and stress levels and provide appropriate feedback, making it difficult to comprehensively improve worker safety and efficiency.

[0365] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0366] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information and emotional status; and means for analyzing changes in voice tone and facial expressions to determine the emotional status of workers and generate appropriate feedback based on that. This enables comprehensive safety management that takes into account the mental health of workers while ensuring their physical safety.

[0367] "Work plan information" refers to data on the processes and schedules necessary for the progress of a construction site, and is used as the basis for safety monitoring.

[0368] "Environmental information" refers to data on weather conditions such as temperature, humidity, atmospheric pressure, and wind speed at the work site, and is an important element for evaluating safety.

[0369] "Emotional state" refers to the mental state of a worker, encompassing emotional factors such as stress levels and concentration levels.

[0370] "Real-time monitoring" is a process of instantly monitoring and analyzing data, and is a technology for continuously understanding the latest status of workers.

[0371] "Feedback" refers to information and instructions provided to workers based on the analysis results performed by the system, with the aim of improving work safety and efficiency.

[0372] A "wearable device" is an electronic device that can be worn by workers and is used to collect biometric information and emotional data in real time.

[0373] A "safety monitoring system" is a comprehensive system designed to ensure the safety of workers and support efficient work, and it evaluates and manages risks based on the results of analysis of various data.

[0374] The system that realizes this invention is an integrated safety monitoring system designed to ensure the safety and mental health of workers at construction sites. The system's main hardware includes a server, terminals used by workers (such as smartphones and smart glasses), and wearable devices.

[0375] The server first receives work plan and environmental information, and identifies potential hazards through data analysis. This enables risk assessment before work begins. The server also individually identifies each worker, monitors them in real time, and provides health management information and emotional status. Specifically, emotion analysis libraries such as EmotionEngine are used to analyze changes in voice tone and facial expressions to determine the worker's stress level and concentration level.

[0376] The terminal functions as a device that displays server-generated feedback to the worker. For example, if a worker is feeling anxious while working at height, the terminal will display a message such as, "Take a deep breath to ease your tension."

[0377] Wearable devices collect real-time biometric information from workers and integrate it with a server to determine the overall safety situation. This allows for the sending of warnings when immediate action is required, facilitating a rapid response.

[0378] For example, when a worker is working under harsh weather conditions, the system can assess the risks based on environmental information and issue instructions such as, "We recommend taking a break for a few minutes."

[0379] An example of a prompt for a generated AI model would be: "Please provide a detailed scenario for a system that combines safety monitoring and sentiment analysis at a construction site. Explain how the emotional state of workers affects safety assessments." Using this prompt, detailed recommended strategies and operational scenarios for the system can be generated.

[0380] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0381] Step 1:

[0382] The server receives work plan information and environmental information. Based on this information, it performs data analysis to identify potential hazards. Inputs include work process data and weather data. Outputs include a hazard assessment report. These analyses utilize statistical methods and machine learning models to comprehensively evaluate on-site risks.

[0383] Step 2:

[0384] The server receives real-time biometric and emotional data from wearable devices. Based on this data, it assesses the worker's stress level and concentration level, and monitors their health. Inputs include heart rate, body temperature, and voice tone, and output is a health report for each worker. The server uses software such as EmotionEngine to perform emotional analysis.

[0385] Step 3:

[0386] The user (worker) receives feedback on a terminal. The server generates feedback and sends it to the terminal, where it is displayed on the worker's smart glasses or smartphone. The input is the generated feedback message, and the output is visual or auditory instructions. Examples include recommendations for breaks or instructions on when to resume work.

[0387] Step 4:

[0388] The server detects danger in real time and immediately sends a warning to the terminal. Inputs are continuously acquired biometric and environmental information, and outputs are warning messages and instructions. This process uses algorithms such as decision trees to make dynamic decisions. For example, if weather conditions rapidly deteriorate, it will notify the user with an instruction such as, "Please temporarily suspend work."

[0389] Step 5:

[0390] The user selects the next action based on system feedback. The input is the instruction received from the terminal, and the output is the action the user actually takes. Here, the user's judgment is a crucial factor in safety. Specific examples include actions such as the user taking a recommended break for safety reasons.

[0391] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0392] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0393] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0394] [Third Embodiment]

[0395] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0396] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0397] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0398] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0399] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0401] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0402] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0403] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0404] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0405] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0406] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0407] The present invention is an advanced safety monitoring system for ensuring the safety of workers at construction sites. Its purpose is to monitor the activities of each worker and changes in the environment in real time, and to prevent potential hazards before they occur.

[0408] This system consists of a central server, terminals used by each worker, and wearable devices worn by the workers. The server integrates and processes work plan information, environmental information, and real-time weather information to perform risk analysis.

[0409] The system's main operation involves the server analyzing TBM-KY information entered via voice input before work begins, identifying work procedures and potential hazards, and notifying the terminal. The user then reviews this information on the terminal and prepares for work.

[0410] During work, the server collects video and biometric information from cameras and wearable devices and analyzes it in real time. If the server determines that a worker is fatigued or needs a break, it sends a break recommendation notification to the device. This process allows users to properly manage their own health.

[0411] The server also monitors weather information in real time and sends immediate warnings in the event of sudden weather changes. For example, if sudden strong winds are predicted, it will send a message to users performing work at heights to warn them.

[0412] Furthermore, the system can improve workplace safety by identifying the risks of unqualified personnel performing tasks and issuing warnings as needed.

[0413] The introduction of this system is expected to ensure worker safety and allow work to proceed more efficiently. This invention is an effective means of comprehensively managing on-site risks and improving safety.

[0414] The following describes the processing flow.

[0415] Step 1:

[0416] The server receives the TBM-KY content, which is entered by the user (worker or supervisor) via voice input into the terminal before work begins. Using speech recognition technology, this information is converted into text data.

[0417] Step 2:

[0418] The server integrates and analyzes the converted text data, work plan information, and weather information. This automatically identifies potential hazards and shortcomings.

[0419] Step 3:

[0420] The server generates a list of identified risks and points of caution and notifies the user's terminal. The user then uses this information to prepare and verify the work before proceeding.

[0421] Step 4:

[0422] Once work begins, the server uses cameras and sensors to identify each worker individually and monitors them in real time.

[0423] Step 5:

[0424] The server analyzes each worker's work time and vital data, and if it determines that a break is necessary, it sends a message to the terminal recommending a break.

[0425] Step 6:

[0426] The server constantly checks real-time weather information and immediately issues warnings if there are sudden changes in the environment. For example, if strong winds or heavy rain are forecast, it will alert the affected users.

[0427] Step 7:

[0428] The server analyzes work actions and credentials to verify that no unauthorized personnel are performing inappropriate tasks. If a violation is detected, an immediate warning is issued.

[0429] Step 8:

[0430] Users receive notifications and warnings from their devices and take appropriate action. They review their work procedures and plans as needed.

[0431] This series of processes is expected to significantly improve safety and efficiency at the work site.

[0432] (Example 1)

[0433] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0434] Safety management for workers is extremely important at construction sites. Traditional methods make it difficult to monitor potential hazards and workers' health in real time, resulting in a high risk of accidents and health problems. Furthermore, mechanisms to effectively prevent unqualified workers are insufficient. Solving these problems and improving safety is essential.

[0435] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0436] In this invention, the server includes means for receiving work plan data and environmental data and analyzing the information to identify potential risks before work begins; means for individually identifying workers, monitoring them in real time, and providing health management information; means for detecting risks based on weather data and work condition data and immediately sending warnings; and means for identifying unqualified workers and issuing warnings. This makes it possible to enhance worker safety, identify potential risks early, and prevent accidents and health problems.

[0437] "Work plan data" refers to information used to manage the progress of on-site activities, such as work schedules and site layouts at construction sites.

[0438] "Environmental data" refers to information related to the work environment, such as temperature, humidity, noise level, and air quality at the work site.

[0439] "Workers" refer to individuals who actually perform work at a construction site and are managed based on each individual's identification information.

[0440] "Health management information" refers to the health status of workers based on biometric information such as heart rate, blood pressure, and body temperature.

[0441] "Weather data" refers to information about current weather conditions and predicted weather changes.

[0442] "Work status data" refers to information about ongoing work, such as the location and activity status of workers.

[0443] "Wearable devices" refer to wearable devices that workers wear and use to measure and monitor biometric information.

[0444] "Audio information" refers to the voices emitted by workers, which are analyzed to generate work procedures and precautions.

[0445] An "unqualified person" refers to a person who does not possess the necessary qualifications to perform a particular task.

[0446] This invention relates to a safety monitoring system for enhancing worker safety. The system consists of a server, a terminal for receiving and notifying data, and a wearable device for collecting information.

[0447] The server aggregates business plan data and environmental data using a database management system. For example, PostgreSQL can be used as the database management system. Furthermore, the server implements speech recognition software to analyze audio information and converts audio data to text using cloud-based APIs. Google Cloud Speech-to-Text API is one example.

[0448] The terminals serve to relay warnings and notifications sent from the server to workers, allowing them to monitor work status in real time. For example, if a worker's health management information exceeds a certain threshold, a warning will be displayed on the terminal.

[0449] The wearable devices are designed to monitor the health of workers and collect biometric data such as heart rate and body temperature. This information is transmitted to a server via Bluetooth and analyzed by an AI model.

[0450] For example, a server can detect a sudden change in weather and send a warning message to a terminal such as, "Strong winds are predicted. Please temporarily suspend work at heights and ensure your safety." This allows users to take appropriate action in a timely manner.

[0451] An example of a prompt for a generated AI model is, "Please tell me about the effects of strong winds and safety measures during work at heights on construction sites."

[0452] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0453] Step 1:

[0454] The server retrieves work plan data and environmental data from the database. This input data includes work schedules and environmental conditions. The server uses data analysis software to analyze the data to identify potential risks. The analysis results in a list of risky processes and insufficient safety equipment.

[0455] Step 2:

[0456] The server receives voice information transmitted from the user via the terminal. This voice information includes TBM-KY information, which is converted into text data by speech recognition software. This converted text data is then analyzed using natural language processing (NLP) techniques to identify work procedures and potential hazards. The server sends this identified information to the terminal as an output list.

[0457] Step 3:

[0458] The server receives biometric data from wearable devices via Bluetooth. This biometric data includes health information such as heart rate and body temperature. The server analyzes the data using a generative AI model and compares it to calculated thresholds. If an abnormality in the health condition is detected, the server sends a rest recommendation notification to the terminal.

[0459] Step 4:

[0460] The server obtains weather information in real time from a weather data API. This input data includes current weather conditions and near-future weather forecasts. Based on this data, the server uses an AI model to analyze and identify risky weather conditions (e.g., strong winds and thunderstorms). It immediately generates a warning message and notifies the terminal as output.

[0461] Step 5:

[0462] The server accesses the worker qualification database to verify that no unqualified individuals are performing work. The input data is the workers' qualification information. The server checks this information and performs comparison and analysis to identify unqualified individuals. If a violation is detected, a warning message is sent to the terminal as output.

[0463] (Application Example 1)

[0464] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0465] Effective safety management at work sites requires immediate understanding of individual worker conditions and changes in the external environment, and the implementation of appropriate countermeasures. However, conventional safety monitoring systems have limitations in providing information efficiently in real time, making it difficult for workers to quickly understand and respond to on-site conditions. Therefore, a new solution is needed that simultaneously improves worker safety and work efficiency.

[0466] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0467] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information; means for detecting hazards based on weather information and work operation data and immediately transmitting warnings; and means for providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site. This makes it possible for workers to carry out their work efficiently while ensuring safety.

[0468] "Work plan information" is a systematic compilation of information regarding the procedures, schedules, materials, and equipment necessary to carry out a task.

[0469] "Environmental information" refers to data that indicates external conditions that may affect work, such as temperature, humidity, illuminance, and noise levels at the work site.

[0470] "Means of analyzing data to identify potential hazards" refers to the process of identifying and evaluating predicted risks and hazardous factors based on various information collected before the start of work.

[0471] "A means of individually identifying and monitoring workers in real time and providing health management information" refers to a system that identifies each worker, measures their current health status and workload, and provides advice and instructions based on the results.

[0472] "A means of detecting hazards based on weather information and work movement data and immediately sending warnings" refers to a method of monitoring current weather conditions and worker movements in real time and promptly issuing warnings when abnormalities or hazards are detected.

[0473] "Means of providing information in real time using visual display devices, enabling workers to quickly grasp the situation on site" refers to a function that instantly transmits important work-related information to workers via visual devices to support situational judgment.

[0474] A system implementing this invention includes a server, a terminal, and a visual display device such as smart glasses. The server plays a central role, receiving work plan information, environmental information, and real-time weather information to identify potential hazards. The server analyzes this data and uses advanced algorithms to detect high-risk situations.

[0475] The server also collects biometric data from wearable devices to monitor individual workers in real time. This data includes heart rate, movement patterns, and location information, which is used to assess the workers' health and safety status.

[0476] The terminal (smart glasses) receives analysis results transmitted from the server and provides workers with real-time warnings and health management information. The terminal uses a visual user interface to instantly communicate changes in the situation, enabling workers to respond quickly.

[0477] For example, if a worker is working at a height and a sudden strong wind is detected approaching, the server analyzes the situation and instantly sends a warning to the smart glasses. This allows the worker to quickly evacuate to a safe location. Additionally, if the heart rate exceeds a certain level, a visual notification prompting a break is automatically displayed.

[0478] Furthermore, this system can also analyze information received via voice input, allowing users to review work procedures and precautions in advance. For example, pre-start inspection procedures can be reviewed by voice to prevent oversights and misunderstandings.

[0479] An example of a prompt would be, "To brainstorm ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers using smart glasses to improve safety?" This prompt allows the generated AI model to suggest more specific advice and functions.

[0480] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0481] Step 1:

[0482] The server receives work plan information, environmental information, and real-time weather information. Based on this information, it builds an initial dataset and begins analysis to identify potential risks. Statistical models and AI algorithms are used in the analysis to determine the priority of risk factors. The input data mainly consists of sensor data and output from predictive models, and the output generates a list of high-risk scenarios.

[0483] Step 2:

[0484] The server acquires and analyzes real-time biometric data for each worker collected from wearable devices. The main input data includes heart rate, activity level, and geographical location information. Using this data, the server evaluates the worker's physical condition and safety risks, and sets a warning level if an abnormality is detected. The analysis results output an evaluation of the worker's safety status.

[0485] Step 3:

[0486] The server evaluates the dynamic conditions at the site based on weather information and work operation data. If a sudden weather change or abnormal operation pattern is detected, the server immediately sends a warning. Weather sensor data and camera images are used as input, and weather simulation and operation analysis algorithms are applied. The output includes specific warning content and recommended actions.

[0487] Step 4:

[0488] The terminal receives information transmitted from the server and displays it in real time on a visual display device. The terminal visually informs the worker of recognized hazards and health management information. The input information consists of analyzed data and notification content, and the output is visualized warning messages and instructions. Based on this, the worker immediately selects a response action.

[0489] Step 5:

[0490] Users interact with the system through voice input. The server analyzes the voice data and provides feedback to the user regarding relevant work procedures and points to note. This process utilizes speech recognition technology, where the input is the voice data itself, and the output is specific work instructions and improvement suggestions.

[0491] Step 6:

[0492] During the process, the server utilizes an AI model to generate reactions to dynamic changes in the situation on-site. The prompt used for the model is, "To consider ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers via smart glasses to improve safety?" This question then generates specific preventative measures. The input here is the prompt statement, and the output is a guideline for preventative action.

[0493] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0494] This invention combines an emotional engine with a safety monitoring system designed to thoroughly ensure the safety of workers at construction sites. This system can consider not only the physical safety of workers but also their mental health.

[0495] The system consists of a server, terminals used by each worker, and wearable devices worn by the workers. The server analyzes work plan information and environmental information to identify risks before work begins, and also identifies and monitors individual workers in real time during work. Specifically, it uses video and audio data acquired from cameras and wearables to identify the emotional state of the users.

[0496] The emotion engine analyzes changes in voice tone and facial expressions to determine whether the user is stressed or focused, and the server generates appropriate feedback based on this. For example, if the emotion engine detects "fatigue" from the user's voice, the server sends a message to the device recommending a break.

[0497] Furthermore, the server can use sentiment data to perform risk assessments and recommend interrupting work if it is deemed dangerous. In this way, it helps maintain and improve team dynamics on-site while ensuring user safety.

[0498] For example, if a worker appears to be experiencing stress while working at height, the system can detect that emotion and send a notification such as, "We recommend taking a break for a few minutes," thereby protecting the worker's safety. By utilizing the emotion engine, it is also possible to address new risks on the job site.

[0499] This invention makes it possible to comprehensively manage on-site risks from multiple perspectives and improve safety and efficiency.

[0500] The following describes the processing flow.

[0501] Step 1:

[0502] Before starting work, the user inputs the content of the TBM-KY (Toolbox Meeting / Hazard Prediction) into a terminal via voice input and sends it to the server. The server receives this voice data.

[0503] Step 2:

[0504] The server converts the received audio data into text data using speech recognition technology and analyzes it along with work plan information and environmental information. This allows it to identify potential hazards before work begins and send notifications of any deficiencies to the user's terminal.

[0505] Step 3:

[0506] Once work begins, the server collects real-time video and audio data from cameras and wearable devices. This allows for the identification of individual workers and the monitoring of their movements and biometric information.

[0507] Step 4:

[0508] The server applies an emotion engine to the collected video and audio data, analyzing the user's emotional state from their voice tone and facial expressions. For example, it can detect "fatigue" or "stress" from their voice tone.

[0509] Step 5:

[0510] Based on the analysis results of the emotion engine, the server evaluates the user's physical and mental state and sends a message to the terminal recommending a break if necessary.

[0511] Step 6:

[0512] The server analyzes weather information and work operation data, and if it detects a danger in real time, it immediately sends a warning to the terminal. Sentiment analysis results are also taken into consideration during this process.

[0513] Step 7:

[0514] The device displays received notifications and warnings to the user and provides an interface for the user to take appropriate action. The user reviews this and adjusts or interrupts their work as needed.

[0515] This series of processes enables the system to achieve comprehensive safety management and reduce risks at the work site.

[0516] (Example 2)

[0517] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0518] On-site work requires not only physical safety but also the mental health of workers. However, currently, there is a lack of systems to comprehensively manage these factors and intervene at the appropriate time. In particular, detecting workers' stress levels and decreased attention in real time and taking appropriate action based on that is a challenge. Furthermore, making decisions about work interruptions or proposing improvement measures based on emotional states is also difficult.

[0519] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0520] In this invention, the server includes means for receiving work information and environmental information and analyzing the information to identify risks before work begins; means for individually identifying and monitoring workers in real time and evaluating their health status; means for analyzing emotional data, determining stress and attention levels, and generating optimal feedback; and means for recommending work interruptions based on the results of the emotional analysis. This enables comprehensive management of the physical and mental safety of workers and provides an efficient work environment while reducing risks at the worksite.

[0521] "Work information" refers to all information related to the work, such as work plans and progress.

[0522] "Environmental information" refers to information related to the work environment, such as temperature, humidity, and noise levels at the work site.

[0523] "Risk" refers to the dangers and problems that may arise during the work.

[0524] "Means of analysis" refers to the techniques and methods used to analyze received information and derive meaningful conclusions or judgments from it.

[0525] "Workers" refers to the personnel who are actually performing the work on-site.

[0526] "Monitoring methods" refer to technologies and devices for observing and recording the condition of workers and the work environment in real time.

[0527] "Health status" refers to the overall physical and mental condition of a worker.

[0528] "Emotional data" refers to information about emotions obtained from the worker's voice tone, facial expressions, and other similar data.

[0529] "Feedback" refers to responses that provide workers with appropriate solutions or warnings for improvement.

[0530] "Means of recommending interruption" refers to methods or techniques for suggesting to workers to temporarily suspend their work in specific situations.

[0531] The embodiment for carrying out the invention is a safety monitoring system for ensuring the safety of workers at a construction site. This system consists of a server, a worker terminal, and a wearable terminal.

[0532] The server receives integrated work and environmental information and analyzes this data to identify risks before work begins. The server is equipped with high-performance computing devices, which are used to analyze video and audio data to assess workers' health and stress levels. Specific software examples include emotion recognition engines specialized in audio analysis and facial expression recognition technology based on image analysis.

[0533] Each worker's device receives feedback from the server and recommends the most appropriate action for the worker. For example, if the emotion engine detects a "fatigue state," the device will display "We suggest a break" and prompt the worker to take a break.

[0534] The user, a worker, wears a wearable device that transmits data from heart rate and motion detection sensors to a server. Based on this information, the server monitors the worker's stress level in real time.

[0535] For example, if a worker's stress level is detected to be rising while working at a high position, the emotion engine will determine that "rest is needed." This will then display a message on the worker's device saying, "We recommend taking a break for a few minutes."

[0536] Furthermore, as an example of a prompt, the question, "What emotional factors should be considered to reduce risks in the work environment?" can be input into the generating AI model to find its own solutions. In this way, the invention provides a system that manages not only physical safety but also mental health, achieving both work efficiency and safety.

[0537] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0538] Step 1:

[0539] The server receives work information and environmental information. Input information includes work plans, weather conditions in the surrounding environment, and noise levels. This data is analyzed to identify potential risks. Pattern recognition algorithms are used in the data analysis to identify conditions deemed particularly dangerous. Specifically, the server uses statistical models to detect anomalies in weather conditions and generates a risk list that proposes preventative measures.

[0540] Step 2:

[0541] The server acquires data to individually identify workers. Inputs include camera footage and biometric information from wearables. Based on this information, the server monitors the workers' health in real time and detects abnormal heart rates and behavioral patterns. Machine learning algorithms are used for data processing. Specifically, when the server detects an anomaly, it sends a warning message to the worker's device.

[0542] Step 3:

[0543] The server analyzes emotional data. Input data includes voice tone and facial expression data. It activates an emotion engine to determine stress levels and concentration levels. The resulting output is an evaluation of the worker's emotional state. Specifically, the server analyzes changes in pitch and speed of the voice data to determine if the user is tense.

[0544] Step 4:

[0545] The server generates feedback based on the sentiment analysis results. The evaluation results from step 3 are used as input. In the feedback generation process, a generative AI model is used to formulate the optimal response and message. The output is a specific action instruction delivered to the worker's terminal. For example, a message such as "Fatigue has been detected. Taking a break is recommended" is generated.

[0546] Step 5:

[0547] The server continuously monitors the data and verifies the effectiveness of the feedback. New data is used as input, and the process from steps 2 to 4 is repeated to confirm improvements in the worker's condition. Specifically, the server periodically resumes data collection and performs recalculations to mitigate risks.

[0548] (Application Example 2)

[0549] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0550] Ensuring worker safety at construction sites requires not only monitoring physical hazards but also considering the mental health of workers. However, conventional safety monitoring systems lacked the means to accurately assess workers' emotions and stress levels and provide appropriate feedback, making it difficult to comprehensively improve worker safety and efficiency.

[0551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0552] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information and emotional status; and means for analyzing changes in voice tone and facial expressions to determine the emotional status of workers and generate appropriate feedback based on that. This enables comprehensive safety management that takes into account the mental health of workers while ensuring their physical safety.

[0553] "Work plan information" refers to data on the processes and schedules necessary for the progress of a construction site, and is used as the basis for safety monitoring.

[0554] "Environmental information" refers to data on weather conditions such as temperature, humidity, atmospheric pressure, and wind speed at the work site, and is an important element for evaluating safety.

[0555] "Emotional state" refers to the mental state of a worker, encompassing emotional factors such as stress levels and concentration levels.

[0556] "Real-time monitoring" is a process of instantly monitoring and analyzing data, and is a technology for continuously understanding the latest status of workers.

[0557] "Feedback" refers to information and instructions provided to workers based on the analysis results performed by the system, with the aim of improving work safety and efficiency.

[0558] A "wearable device" is an electronic device that can be worn by workers and is used to collect biometric information and emotional data in real time.

[0559] A "safety monitoring system" is a comprehensive system designed to ensure the safety of workers and support efficient work, and it evaluates and manages risks based on the results of analysis of various data.

[0560] The system that realizes this invention is an integrated safety monitoring system designed to ensure the safety and mental health of workers at construction sites. The system's main hardware includes a server, terminals used by workers (such as smartphones and smart glasses), and wearable devices.

[0561] The server first receives work plan and environmental information, and identifies potential hazards through data analysis. This enables risk assessment before work begins. The server also individually identifies each worker, monitors them in real time, and provides health management information and emotional status. Specifically, emotion analysis libraries such as EmotionEngine are used to analyze changes in voice tone and facial expressions to determine the worker's stress level and concentration level.

[0562] The terminal functions as a device that displays server-generated feedback to the worker. For example, if a worker is feeling anxious while working at height, the terminal will display a message such as, "Take a deep breath to ease your tension."

[0563] Wearable devices collect real-time biometric information from workers and integrate it with a server to determine the overall safety situation. This allows for the sending of warnings when immediate action is required, facilitating a rapid response.

[0564] For example, when a worker is working under harsh weather conditions, the system can assess the risks based on environmental information and issue instructions such as, "We recommend taking a break for a few minutes."

[0565] An example of a prompt for a generated AI model would be: "Please provide a detailed scenario for a system that combines safety monitoring and sentiment analysis at a construction site. Explain how the emotional state of workers affects safety assessments." Using this prompt, detailed recommended strategies and operational scenarios for the system can be generated.

[0566] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0567] Step 1:

[0568] The server receives work plan information and environmental information. Based on this information, it performs data analysis to identify potential hazards. Inputs include work process data and weather data. Outputs include a hazard assessment report. These analyses utilize statistical methods and machine learning models to comprehensively evaluate on-site risks.

[0569] Step 2:

[0570] The server receives real-time biometric and emotional data from wearable devices. Based on this data, it assesses the worker's stress level and concentration level, and monitors their health. Inputs include heart rate, body temperature, and voice tone, and output is a health report for each worker. The server uses software such as EmotionEngine to perform emotional analysis.

[0571] Step 3:

[0572] The user (worker) receives feedback on a terminal. The server generates feedback and sends it to the terminal, where it is displayed on the worker's smart glasses or smartphone. The input is the generated feedback message, and the output is visual or auditory instructions. Examples include recommendations for breaks or instructions on when to resume work.

[0573] Step 4:

[0574] The server detects danger in real time and immediately sends a warning to the terminal. Inputs are continuously acquired biometric and environmental information, and outputs are warning messages and instructions. This process uses algorithms such as decision trees to make dynamic decisions. For example, if weather conditions rapidly deteriorate, it will notify the user with an instruction such as, "Please temporarily suspend work."

[0575] Step 5:

[0576] The user selects the next action based on system feedback. The input is the instruction received from the terminal, and the output is the action the user actually takes. Here, the user's judgment is a crucial factor in safety. Specific examples include actions such as the user taking a recommended break for safety reasons.

[0577] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0578] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0579] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0580] [Fourth Embodiment]

[0581] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0582] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0583] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0584] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0585] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0586] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0587] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0588] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0589] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0590] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0593] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0594] The present invention is an advanced safety monitoring system for ensuring the safety of workers at construction sites. Its purpose is to monitor the activities of each worker and changes in the environment in real time, and to prevent potential hazards before they occur.

[0595] This system consists of a central server, terminals used by each worker, and wearable devices worn by the workers. The server integrates and processes work plan information, environmental information, and real-time weather information to perform risk analysis.

[0596] The system's main operation involves the server analyzing TBM-KY information entered via voice input before work begins, identifying work procedures and potential hazards, and notifying the terminal. The user then reviews this information on the terminal and prepares for work.

[0597] During work, the server collects video and biometric information from cameras and wearable devices and analyzes it in real time. If the server determines that a worker is fatigued or needs a break, it sends a break recommendation notification to the device. This process allows users to properly manage their own health.

[0598] The server also monitors weather information in real time and sends immediate warnings in the event of sudden weather changes. For example, if sudden strong winds are predicted, it will send a message to users performing work at heights to warn them.

[0599] Furthermore, the system can improve workplace safety by identifying the risks of unqualified personnel performing tasks and issuing warnings as needed.

[0600] The introduction of this system is expected to ensure worker safety and allow work to proceed more efficiently. This invention is an effective means of comprehensively managing on-site risks and improving safety.

[0601] The following describes the processing flow.

[0602] Step 1:

[0603] The server receives the TBM-KY content, which is entered by the user (worker or supervisor) via voice input into the terminal before work begins. Using speech recognition technology, this information is converted into text data.

[0604] Step 2:

[0605] The server integrates and analyzes the converted text data, work plan information, and weather information. This automatically identifies potential hazards and shortcomings.

[0606] Step 3:

[0607] The server generates a list of identified risks and points of caution and notifies the user's terminal. The user then uses this information to prepare and verify the work before proceeding.

[0608] Step 4:

[0609] Once work begins, the server uses cameras and sensors to identify each worker individually and monitors them in real time.

[0610] Step 5:

[0611] The server analyzes each worker's work time and vital data, and if it determines that a break is necessary, it sends a message to the terminal recommending a break.

[0612] Step 6:

[0613] The server constantly checks real-time weather information and immediately issues warnings if there are sudden changes in the environment. For example, if strong winds or heavy rain are forecast, it will alert the affected users.

[0614] Step 7:

[0615] The server analyzes work actions and credentials to verify that no unauthorized personnel are performing inappropriate tasks. If a violation is detected, an immediate warning is issued.

[0616] Step 8:

[0617] Users receive notifications and warnings from their devices and take appropriate action. They review their work procedures and plans as needed.

[0618] This series of processes is expected to significantly improve safety and efficiency at the work site.

[0619] (Example 1)

[0620] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] Safety management for workers is extremely important at construction sites. Traditional methods make it difficult to monitor potential hazards and workers' health in real time, resulting in a high risk of accidents and health problems. Furthermore, mechanisms to effectively prevent unqualified workers are insufficient. Solving these problems and improving safety is essential.

[0622] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0623] In this invention, the server includes means for receiving work plan data and environmental data and analyzing the information to identify potential risks before work begins; means for individually identifying workers, monitoring them in real time, and providing health management information; means for detecting risks based on weather data and work condition data and immediately sending warnings; and means for identifying unqualified workers and issuing warnings. This makes it possible to enhance worker safety, identify potential risks early, and prevent accidents and health problems.

[0624] "Work plan data" refers to information used to manage the progress of on-site activities, such as work schedules and site layouts at construction sites.

[0625] "Environmental data" refers to information related to the work environment, such as temperature, humidity, noise level, and air quality at the work site.

[0626] "Workers" refer to individuals who actually perform work at a construction site and are managed based on each individual's identification information.

[0627] "Health management information" refers to the health status of workers based on biometric information such as heart rate, blood pressure, and body temperature.

[0628] "Weather data" refers to information about current weather conditions and predicted weather changes.

[0629] "Work status data" refers to information about ongoing work, such as the location and activity status of workers.

[0630] "Wearable devices" refer to wearable devices that workers wear and use to measure and monitor biometric information.

[0631] "Audio information" refers to the voices emitted by workers, which are analyzed to generate work procedures and precautions.

[0632] An "unqualified person" refers to a person who does not possess the necessary qualifications to perform a particular task.

[0633] This invention relates to a safety monitoring system for enhancing worker safety. The system consists of a server, a terminal for receiving and notifying data, and a wearable device for collecting information.

[0634] The server aggregates business plan data and environmental data using a database management system. For example, PostgreSQL can be used as the database management system. Furthermore, the server implements speech recognition software to analyze audio information and converts audio data to text using cloud-based APIs. Google Cloud Speech-to-Text API is one example.

[0635] The terminals serve to relay warnings and notifications sent from the server to workers, allowing them to monitor work status in real time. For example, if a worker's health management information exceeds a certain threshold, a warning will be displayed on the terminal.

[0636] The wearable devices are designed to monitor the health of workers and collect biometric data such as heart rate and body temperature. This information is transmitted to a server via Bluetooth and analyzed by an AI model.

[0637] For example, a server can detect a sudden change in weather and send a warning message to a terminal such as, "Strong winds are predicted. Please temporarily suspend work at heights and ensure your safety." This allows users to take appropriate action in a timely manner.

[0638] An example of a prompt for a generated AI model is, "Please tell me about the effects of strong winds and safety measures during work at heights on construction sites."

[0639] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0640] Step 1:

[0641] The server retrieves work plan data and environmental data from the database. This input data includes work schedules and environmental conditions. The server uses data analysis software to analyze the data to identify potential risks. The analysis results in a list of risky processes and insufficient safety equipment.

[0642] Step 2:

[0643] The server receives voice information transmitted from the user via the terminal. This voice information includes TBM-KY information, which is converted into text data by speech recognition software. This converted text data is then analyzed using natural language processing (NLP) techniques to identify work procedures and potential hazards. The server sends this identified information to the terminal as an output list.

[0644] Step 3:

[0645] The server receives biometric data from wearable devices via Bluetooth. This biometric data includes health information such as heart rate and body temperature. The server analyzes the data using a generative AI model and compares it to calculated thresholds. If an abnormality in the health condition is detected, the server sends a rest recommendation notification to the terminal.

[0646] Step 4:

[0647] The server obtains weather information in real time from a weather data API. This input data includes current weather conditions and near-future weather forecasts. Based on this data, the server uses an AI model to analyze and identify risky weather conditions (e.g., strong winds and thunderstorms). It immediately generates a warning message and notifies the terminal as output.

[0648] Step 5:

[0649] The server accesses the worker qualification database to verify that no unqualified individuals are performing work. The input data is the workers' qualification information. The server checks this information and performs comparison and analysis to identify unqualified individuals. If a violation is detected, a warning message is sent to the terminal as output.

[0650] (Application Example 1)

[0651] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0652] Effective safety management at work sites requires immediate understanding of individual worker conditions and changes in the external environment, and the implementation of appropriate countermeasures. However, conventional safety monitoring systems have limitations in providing information efficiently in real time, making it difficult for workers to quickly understand and respond to on-site conditions. Therefore, a new solution is needed that simultaneously improves worker safety and work efficiency.

[0653] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0654] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information; means for detecting hazards based on weather information and work operation data and immediately transmitting warnings; and means for providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site. This makes it possible for workers to carry out their work efficiently while ensuring safety.

[0655] "Work plan information" is a systematic compilation of information regarding the procedures, schedules, materials, and equipment necessary to carry out a task.

[0656] "Environmental information" refers to data that indicates external conditions that may affect work, such as temperature, humidity, illuminance, and noise levels at the work site.

[0657] "Means of analyzing data to identify potential hazards" refers to the process of identifying and evaluating predicted risks and hazardous factors based on various information collected before the start of work.

[0658] "A means of individually identifying and monitoring workers in real time and providing health management information" refers to a system that identifies each worker, measures their current health status and workload, and provides advice and instructions based on the results.

[0659] "A means of detecting hazards based on weather information and work movement data and immediately sending warnings" refers to a method of monitoring current weather conditions and worker movements in real time and promptly issuing warnings when abnormalities or hazards are detected.

[0660] "Means of providing information in real time using visual display devices, enabling workers to quickly grasp the situation on site" refers to a function that instantly transmits important work-related information to workers via visual devices to support situational judgment.

[0661] A system implementing this invention includes a server, a terminal, and a visual display device such as smart glasses. The server plays a central role, receiving work plan information, environmental information, and real-time weather information to identify potential hazards. The server analyzes this data and uses advanced algorithms to detect high-risk situations.

[0662] The server also collects biometric data from wearable devices to monitor individual workers in real time. This data includes heart rate, movement patterns, and location information, which is used to assess the workers' health and safety status.

[0663] The terminal (smart glasses) receives analysis results transmitted from the server and provides workers with real-time warnings and health management information. The terminal uses a visual user interface to instantly communicate changes in the situation, enabling workers to respond quickly.

[0664] For example, if a worker is working at a height and a sudden strong wind is detected approaching, the server analyzes the situation and instantly sends a warning to the smart glasses. This allows the worker to quickly evacuate to a safe location. Additionally, if the heart rate exceeds a certain level, a visual notification prompting a break is automatically displayed.

[0665] Furthermore, this system can also analyze information received via voice input, allowing users to review work procedures and precautions in advance. For example, pre-start inspection procedures can be reviewed by voice to prevent oversights and misunderstandings.

[0666] An example of a prompt would be, "To brainstorm ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers using smart glasses to improve safety?" This prompt allows the generated AI model to suggest more specific advice and functions.

[0667] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0668] Step 1:

[0669] The server receives work plan information, environmental information, and real-time weather information. Based on this information, it builds an initial dataset and begins analysis to identify potential risks. Statistical models and AI algorithms are used in the analysis to determine the priority of risk factors. The input data mainly consists of sensor data and output from predictive models, and the output generates a list of high-risk scenarios.

[0670] Step 2:

[0671] The server acquires and analyzes real-time biometric data for each worker collected from wearable devices. The main input data includes heart rate, activity level, and geographical location information. Using this data, the server evaluates the worker's physical condition and safety risks, and sets a warning level if an abnormality is detected. The analysis results output an evaluation of the worker's safety status.

[0672] Step 3:

[0673] The server evaluates the dynamic conditions at the site based on weather information and work operation data. If a sudden weather change or abnormal operation pattern is detected, the server immediately sends a warning. Weather sensor data and camera images are used as input, and weather simulation and operation analysis algorithms are applied. The output includes specific warning content and recommended actions.

[0674] Step 4:

[0675] The terminal receives information transmitted from the server and displays it in real time on a visual display device. The terminal visually informs the worker of recognized hazards and health management information. The input information consists of analyzed data and notification content, and the output is visualized warning messages and instructions. Based on this, the worker immediately selects a response action.

[0676] Step 5:

[0677] Users interact with the system through voice input. The server analyzes the voice data and provides feedback to the user regarding relevant work procedures and points to note. This process utilizes speech recognition technology, where the input is the voice data itself, and the output is specific work instructions and improvement suggestions.

[0678] Step 6:

[0679] During the process, the server utilizes an AI model to generate reactions to dynamic changes in the situation on-site. The prompt used for the model is, "To consider ideas for a real-time safety monitoring system at a construction site, what information should be provided to workers via smart glasses to improve safety?" This question then generates specific preventative measures. The input here is the prompt statement, and the output is a guideline for preventative action.

[0680] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0681] This invention combines an emotional engine with a safety monitoring system designed to thoroughly ensure the safety of workers at construction sites. This system can consider not only the physical safety of workers but also their mental health.

[0682] The system consists of a server, terminals used by each worker, and wearable devices worn by the workers. The server analyzes work plan information and environmental information to identify risks before work begins, and also identifies and monitors individual workers in real time during work. Specifically, it uses video and audio data acquired from cameras and wearables to identify the emotional state of the users.

[0683] The emotion engine analyzes changes in voice tone and facial expressions to determine whether the user is stressed or focused, and the server generates appropriate feedback based on this. For example, if the emotion engine detects "fatigue" from the user's voice, the server sends a message to the device recommending a break.

[0684] Furthermore, the server can use sentiment data to perform risk assessments and recommend interrupting work if it is deemed dangerous. In this way, it helps maintain and improve team dynamics on-site while ensuring user safety.

[0685] For example, if a worker appears to be experiencing stress while working at height, the system can detect that emotion and send a notification such as, "We recommend taking a break for a few minutes," thereby protecting the worker's safety. By utilizing the emotion engine, it is also possible to address new risks on the job site.

[0686] This invention makes it possible to comprehensively manage on-site risks from multiple perspectives and improve safety and efficiency.

[0687] The following describes the processing flow.

[0688] Step 1:

[0689] Before starting work, the user inputs the content of the TBM-KY (Toolbox Meeting / Hazard Prediction) into a terminal via voice input and sends it to the server. The server receives this voice data.

[0690] Step 2:

[0691] The server converts the received audio data into text data using speech recognition technology and analyzes it along with work plan information and environmental information. This allows it to identify potential hazards before work begins and send notifications of any deficiencies to the user's terminal.

[0692] Step 3:

[0693] Once work begins, the server collects real-time video and audio data from cameras and wearable devices. This allows for the identification of individual workers and the monitoring of their movements and biometric information.

[0694] Step 4:

[0695] The server applies an emotion engine to the collected video and audio data, analyzing the user's emotional state from their voice tone and facial expressions. For example, it can detect "fatigue" or "stress" from their voice tone.

[0696] Step 5:

[0697] Based on the analysis results of the emotion engine, the server evaluates the user's physical and mental state and sends a message to the terminal recommending a break if necessary.

[0698] Step 6:

[0699] The server analyzes weather information and work operation data, and if it detects a danger in real time, it immediately sends a warning to the terminal. Sentiment analysis results are also taken into consideration during this process.

[0700] Step 7:

[0701] The device displays received notifications and warnings to the user and provides an interface for the user to take appropriate action. The user reviews this and adjusts or interrupts their work as needed.

[0702] This series of processes enables the system to achieve comprehensive safety management and reduce risks at the work site.

[0703] (Example 2)

[0704] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] On-site work requires not only physical safety but also the mental health of workers. However, currently, there is a lack of systems to comprehensively manage these factors and intervene at the appropriate time. In particular, detecting workers' stress levels and decreased attention in real time and taking appropriate action based on that is a challenge. Furthermore, making decisions about work interruptions or proposing improvement measures based on emotional states is also difficult.

[0706] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0707] In this invention, the server includes means for receiving work information and environmental information and analyzing the information to identify risks before work begins; means for individually identifying and monitoring workers in real time and evaluating their health status; means for analyzing emotional data, determining stress and attention levels, and generating optimal feedback; and means for recommending work interruptions based on the results of the emotional analysis. This enables comprehensive management of the physical and mental safety of workers and provides an efficient work environment while reducing risks at the worksite.

[0708] "Work information" refers to all information related to the work, such as work plans and progress.

[0709] "Environmental information" refers to information related to the work environment, such as temperature, humidity, and noise levels at the work site.

[0710] "Risk" refers to the dangers and problems that may arise during the work.

[0711] "Means of analysis" refers to the techniques and methods used to analyze received information and derive meaningful conclusions or judgments from it.

[0712] "Workers" refers to the personnel who are actually performing the work on-site.

[0713] "Monitoring methods" refer to technologies and devices for observing and recording the condition of workers and the work environment in real time.

[0714] "Health status" refers to the overall physical and mental condition of a worker.

[0715] "Emotional data" refers to information about emotions obtained from the worker's voice tone, facial expressions, and other similar data.

[0716] "Feedback" refers to responses that provide workers with appropriate solutions or warnings for improvement.

[0717] "Means of recommending interruption" refers to methods or techniques for suggesting to workers to temporarily suspend their work in specific situations.

[0718] The embodiment for carrying out the invention is a safety monitoring system for ensuring the safety of workers at a construction site. This system consists of a server, a worker terminal, and a wearable terminal.

[0719] The server receives integrated work and environmental information and analyzes this data to identify risks before work begins. The server is equipped with high-performance computing devices, which are used to analyze video and audio data to assess workers' health and stress levels. Specific software examples include emotion recognition engines specialized in audio analysis and facial expression recognition technology based on image analysis.

[0720] Each worker's device receives feedback from the server and recommends the most appropriate action for the worker. For example, if the emotion engine detects a "fatigue state," the device will display "We suggest a break" and prompt the worker to take a break.

[0721] The user, a worker, wears a wearable device that transmits data from heart rate and motion detection sensors to a server. Based on this information, the server monitors the worker's stress level in real time.

[0722] For example, if a worker's stress level is detected to be rising while working at a high position, the emotion engine will determine that "rest is needed." This will then display a message on the worker's device saying, "We recommend taking a break for a few minutes."

[0723] Furthermore, as an example of a prompt, the question, "What emotional factors should be considered to reduce risks in the work environment?" can be input into the generating AI model to find its own solutions. In this way, the invention provides a system that manages not only physical safety but also mental health, achieving both work efficiency and safety.

[0724] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0725] Step 1:

[0726] The server receives work information and environmental information. Input information includes work plans, weather conditions in the surrounding environment, and noise levels. This data is analyzed to identify potential risks. Pattern recognition algorithms are used in the data analysis to identify conditions deemed particularly dangerous. Specifically, the server uses statistical models to detect anomalies in weather conditions and generates a risk list that proposes preventative measures.

[0727] Step 2:

[0728] The server acquires data to individually identify workers. Inputs include camera footage and biometric information from wearables. Based on this information, the server monitors the workers' health in real time and detects abnormal heart rates and behavioral patterns. Machine learning algorithms are used for data processing. Specifically, when the server detects an anomaly, it sends a warning message to the worker's device.

[0729] Step 3:

[0730] The server analyzes emotional data. Input data includes voice tone and facial expression data. It activates an emotion engine to determine stress levels and concentration levels. The resulting output is an evaluation of the worker's emotional state. Specifically, the server analyzes changes in pitch and speed of the voice data to determine if the user is tense.

[0731] Step 4:

[0732] The server generates feedback based on the sentiment analysis results. The evaluation results from step 3 are used as input. In the feedback generation process, a generative AI model is used to formulate the optimal response and message. The output is a specific action instruction delivered to the worker's terminal. For example, a message such as "Fatigue has been detected. Taking a break is recommended" is generated.

[0733] Step 5:

[0734] The server continuously monitors the data and verifies the effectiveness of the feedback. New data is used as input, and the process from steps 2 to 4 is repeated to confirm improvements in the worker's condition. Specifically, the server periodically resumes data collection and performs recalculations to mitigate risks.

[0735] (Application Example 2)

[0736] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] Ensuring worker safety at construction sites requires not only monitoring physical hazards but also considering the mental health of workers. However, conventional safety monitoring systems lacked the means to accurately assess workers' emotions and stress levels and provide appropriate feedback, making it difficult to comprehensively improve worker safety and efficiency.

[0738] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0739] In this invention, the server includes means for receiving work plan information and environmental information and analyzing the data to identify potential hazards before work begins; means for individually identifying and monitoring workers in real time and providing health management information and emotional status; and means for analyzing changes in voice tone and facial expressions to determine the emotional status of workers and generate appropriate feedback based on that. This enables comprehensive safety management that takes into account the mental health of workers while ensuring their physical safety.

[0740] "Work plan information" refers to data on the processes and schedules necessary for the progress of a construction site, and is used as the basis for safety monitoring.

[0741] "Environmental information" refers to data on weather conditions such as temperature, humidity, atmospheric pressure, and wind speed at the work site, and is an important element for evaluating safety.

[0742] "Emotional state" refers to the mental state of a worker, encompassing emotional factors such as stress levels and concentration levels.

[0743] "Real-time monitoring" is a process of instantly monitoring and analyzing data, and is a technology for continuously understanding the latest status of workers.

[0744] "Feedback" refers to information and instructions provided to workers based on the analysis results performed by the system, with the aim of improving work safety and efficiency.

[0745] A "wearable device" is an electronic device that can be worn by workers and is used to collect biometric information and emotional data in real time.

[0746] A "safety monitoring system" is a comprehensive system designed to ensure the safety of workers and support efficient work, and it evaluates and manages risks based on the results of analysis of various data.

[0747] The system that realizes this invention is an integrated safety monitoring system designed to ensure the safety and mental health of workers at construction sites. The system's main hardware includes a server, terminals used by workers (such as smartphones and smart glasses), and wearable devices.

[0748] The server first receives work plan and environmental information, and identifies potential hazards through data analysis. This enables risk assessment before work begins. The server also individually identifies each worker, monitors them in real time, and provides health management information and emotional status. Specifically, emotion analysis libraries such as EmotionEngine are used to analyze changes in voice tone and facial expressions to determine the worker's stress level and concentration level.

[0749] The terminal functions as a device that displays server-generated feedback to the worker. For example, if a worker is feeling anxious while working at height, the terminal will display a message such as, "Take a deep breath to ease your tension."

[0750] Wearable devices collect real-time biometric information from workers and integrate it with a server to determine the overall safety situation. This allows for the sending of warnings when immediate action is required, facilitating a rapid response.

[0751] For example, when a worker is working under harsh weather conditions, the system can assess the risks based on environmental information and issue instructions such as, "We recommend taking a break for a few minutes."

[0752] An example of a prompt for a generated AI model would be: "Please provide a detailed scenario for a system that combines safety monitoring and sentiment analysis at a construction site. Explain how the emotional state of workers affects safety assessments." Using this prompt, detailed recommended strategies and operational scenarios for the system can be generated.

[0753] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0754] Step 1:

[0755] The server receives work plan information and environmental information. Based on this information, it performs data analysis to identify potential hazards. Inputs include work process data and weather data. Outputs include a hazard assessment report. These analyses utilize statistical methods and machine learning models to comprehensively evaluate on-site risks.

[0756] Step 2:

[0757] The server receives real-time biometric and emotional data from wearable devices. Based on this data, it assesses the worker's stress level and concentration level, and monitors their health. Inputs include heart rate, body temperature, and voice tone, and output is a health report for each worker. The server uses software such as EmotionEngine to perform emotional analysis.

[0758] Step 3:

[0759] The user (worker) receives feedback on a terminal. The server generates feedback and sends it to the terminal, where it is displayed on the worker's smart glasses or smartphone. The input is the generated feedback message, and the output is visual or auditory instructions. Examples include recommendations for breaks or instructions on when to resume work.

[0760] Step 4:

[0761] The server detects danger in real time and immediately sends a warning to the terminal. Inputs are continuously acquired biometric and environmental information, and outputs are warning messages and instructions. This process uses algorithms such as decision trees to make dynamic decisions. For example, if weather conditions rapidly deteriorate, it will notify the user with an instruction such as, "Please temporarily suspend work."

[0762] Step 5:

[0763] The user selects the next action based on system feedback. The input is the instruction received from the terminal, and the output is the action the user actually takes. Here, the user's judgment is a crucial factor in safety. Specific examples include actions such as the user taking a recommended break for safety reasons.

[0764] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0765] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0766] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0767] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0768] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0769] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0770] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0771] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0772] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0773] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0774] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0775] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0776] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0777] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0778] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0779] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0780] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0781] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0782] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0783] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0784] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0785] The following is further disclosed regarding the embodiments described above.

[0786] (Claim 1)

[0787] A means of receiving work plan information and environmental information, and analyzing the data to identify potential hazards before starting work,

[0788] A means of individually identifying and monitoring workers in real time and providing health management information,

[0789] A means of detecting hazards based on weather information and work operation data and immediately sending out warnings,

[0790] A safety monitoring system including a safety monitoring system.

[0791] (Claim 2)

[0792] The safety monitoring system according to claim 1, comprising means for collecting real-time biometric information for each worker in conjunction with a wearable device and for integrating and determining the safety situation.

[0793] (Claim 3)

[0794] The safety monitoring system according to claim 1, comprising means for analyzing audio data, automatically generating work procedures and precautions, and pointing out any shortcomings in advance.

[0795] "Example 1"

[0796] (Claim 1)

[0797] A means of receiving business plan data and environmental data and analyzing the information to identify potential risks before commencing operations,

[0798] A means of individually identifying workers, monitoring them in real time, and providing health management information,

[0799] A means of detecting risks based on weather data and work status data and immediately sending out warnings,

[0800] A means to identify and warn about work being performed by unqualified individuals,

[0801] A system that includes this.

[0802] (Claim 2)

[0803] The system according to claim 1, comprising means for collecting real-time biometric data for each worker in conjunction with wearable devices and for integrating and determining the safety situation.

[0804] (Claim 3)

[0805] The system according to claim 1, comprising means for analyzing voice information, automatically generating work procedures and points to note, and pointing out any shortcomings in advance.

[0806] "Application Example 1"

[0807] (Claim 1)

[0808] A means of receiving work plan information and environmental information, and analyzing the data to identify potential hazards before starting work,

[0809] A means of individually identifying and monitoring workers in real time and providing health management information,

[0810] A means of detecting hazards based on weather information and work operation data and immediately sending out warnings,

[0811] A means of providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, comprising means for collecting real-time biometric data for each worker in conjunction with a wearable device and for integrating and determining the safety situation.

[0815] (Claim 3)

[0816] The system according to claim 1, comprising means for analyzing audio data, automatically generating work procedures and precautions, and pointing out any shortcomings in advance.

[0817] "Example 2 of combining an emotion engine"

[0818] (Claim 1)

[0819] A means of receiving work information and environmental information, and analyzing the information to identify risks before starting work,

[0820] A means of individually identifying workers, monitoring them in real time, and evaluating their health status,

[0821] A means of analyzing emotional data, determining stress and attention levels, and generating optimal feedback,

[0822] A means of recommending work interruption based on the results of sentiment analysis,

[0823] A system that includes this.

[0824] (Claim 2)

[0825] The system according to claim 1, comprising means for collecting real-time biometric data for each worker in cooperation with a wearable device and for comprehensively evaluating safety.

[0826] (Claim 3)

[0827] The system according to claim 1, comprising means for analyzing voice information, automatically generating work procedures and warnings, and identifying items for improvement in advance.

[0828] "Application example 2 when combining with an emotional engine"

[0829] (Claim 1)

[0830] A means of receiving work plan information and environmental information, and analyzing the data to identify potential hazards before starting work,

[0831] A means of individually identifying and monitoring workers in real time and providing health management information,

[0832] A means for analyzing changes in voice tone and facial expressions to determine the emotional state of a worker and generate appropriate feedback based on that,

[0833] A means of detecting hazards based on weather information and work operation data and immediately sending out warnings,

[0834] A means of providing feedback to workers and supporting them in maintaining work efficiency and safety,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, comprising means for collecting real-time biometric and emotional information for each worker in conjunction with a wearable device and for integrating and determining the safety situation.

[0838] (Claim 3)

[0839] The system according to claim 1, comprising means for analyzing voice data, automatically generating work procedures and precautions, pointing out any shortcomings in advance, and utilizing an emotion engine to evaluate the mental health of workers and recommend appropriate breaks or work interruptions. [Explanation of symbols]

[0840] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving work plan information and environmental information, and analyzing the data to identify potential hazards before starting work, A means of individually identifying and monitoring workers in real time and providing health management information, A means of detecting hazards based on weather information and work operation data and immediately sending out warnings, A means of providing information in real time using a visual display device, enabling workers to quickly grasp the situation on site, A system that includes this.

2. The system according to claim 1, comprising means for collecting real-time biometric data for each worker in conjunction with a wearable device and for integrating and determining the safety situation.

3. The system according to claim 1, comprising means for analyzing audio data, automatically generating work procedures and precautions, and pointing out any shortcomings in advance.

Images