system

Abstract

We provide the system. [Solution] A means of collecting past project data and safety records and storing them in a database, A means of providing an interface for inputting specific information from the field, A means for generating a risk scenario using artificial intelligence based on the aforementioned data and input information, A means for presenting the generated risk scenario to the user and receiving input for countermeasures, The aforementioned countermeasures and their results are stored in a database and analyzed by means of a database. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is aimed at solving the problem that safety activities become routine and accidents are likely to occur due to insufficient recognition of risks in on-site work. Also, it is necessary to address the issue that there is a lack of a mechanism for on-site workers to effectively grasp risks that are difficult to predict and take appropriate countermeasures.

Means for Solving the Problems

[0005] This invention provides a means for collecting past project data and safety records and storing them in a database, and for collecting on-site data through an interface for inputting specific information from the field. It also includes a means for generating hazardous scenarios using artificial intelligence based on this data, presenting them to the user, and prompting the input of countermeasures. Furthermore, it enhances the learning effect by accumulating and analyzing the corresponding countermeasures and their results in the database. In addition, it introduces means for updating the risk assessment algorithm to improve the accuracy of subsequent analyses, and means for encouraging user participation and improving safety awareness by incorporating a point system and award system.

[0006] "Project data" refers to detailed information about past construction and work projects, including risk factors and accident records.

[0007] A "safety record" is a collection of data related to safety at a work site, including information on past incidents and safety measures.

[0008] A "database" is an information management system that systematically organizes and stores information, making it searchable and usable as needed.

[0009] An "interface" is a component that provides display and input means to enable the exchange of information between a user and a system.

[0010] "Artificial intelligence" is a technology that uses computers to mimic human intelligence, possessing the ability to make sophisticated decisions through data analysis and learning processes.

[0011] A "hazard scenario" is a simulation that specifically depicts situations that anticipate potential risks occurring in the field.

[0012] A "countermeasure" is a specific action plan formulated to deal with anticipated risks.

[0013] A "risk assessment algorithm" is a computational method for numerically evaluating the probability of risk occurrence and its impact based on collected data and generated risk scenarios.

[0014] The "point system" is a mechanism that quantifies users' contributions and proactiveness in risk prediction activities and reflects this in rewards and evaluations.

[0015] An "award system" is a system for officially recognizing and commending users who have demonstrated outstanding safety awareness and implemented excellent countermeasures. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system that supports hazard prediction activities to improve safety awareness at construction sites, and analyzes risks using artificial intelligence based on past project data and safety records. A specific embodiment of this system is described below.

[0038] First, the server collects past project data and safety records from the database. This allows for a comprehensive understanding of past cases and key safety points at construction sites. The server then formats the collected information for analysis, making it usable by AI models.

[0039] Next, the terminal provides an interface that allows field workers to input information. The user (worker) uses this interface to input specific information such as the site's location, weather conditions, and work status. This field information then becomes an important data source for the AI.

[0040] The server uses artificial intelligence to generate hazard scenarios based on this information. The generated hazard scenarios take into account different risk factors and serve to attract the user's attention and raise safety awareness by visualizing potential dangers.

[0041] Next, the device presents the user with this risk scenario. Based on the presented scenario, the user considers appropriate countermeasures and records a specific action plan on the device. This enables the user to independently consider and implement risk mitigation measures.

[0042] Subsequently, the server stores the entered countermeasures and their implementation results in a database. This data will be used to improve future AI models and develop new risk assessment algorithms, enabling more accurate risk prediction.

[0043] For example, if work is being done at a high altitude at a certain site, the server uses data from past fall accidents during high-altitude work to generate a dangerous scenario where "the scaffolding becomes unstable in strong winds." The user reviews this scenario, devises countermeasures such as installing safety devices according to the wind speed or temporarily suspending work, and inputs them into the terminal. The server records this information and uses it for future risk assessments.

[0044] In this way, this system provides effective measures to prevent accidents at construction sites while raising safety awareness.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The server collects historical project data and safety records from the database. It then formats this data and converts it into a format that the AI ​​model can use for analysis.

[0048] Step 2:

[0049] The terminal provides the user with an interface for inputting specific information at the work site. This interface allows input of location information, weather conditions for the day, and the health status of the workers.

[0050] Step 3:

[0051] Users input information about the field situation through the terminal interface. They meticulously record any unusual circumstances or important information at the site.

[0052] Step 4:

[0053] The server receives information entered by the user, integrates it with historical data, and analyzes it. Using an AI model, it generates hazard scenarios based on the actual situation on site.

[0054] Step 5:

[0055] The device presents the generated risk scenario to the user. The user reviews the risk information and its background described in the displayed scenario.

[0056] Step 6:

[0057] Based on the presented hazard scenarios, the user considers safety measures and decides on specific countermeasures. The decided countermeasures are then entered into the terminal and recorded.

[0058] Step 7:

[0059] The server stores the countermeasures entered by users and the results of their implementation in a database. This accumulated data will be used to improve future AI models and enhance the accuracy of risk assessments.

[0060] Step 8:

[0061] The server regularly updates its risk assessment algorithm to achieve more accurate hazard prediction. It also encourages user participation through a points system and awards program, aiming to improve safety awareness.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] Ensuring safety at construction sites is a crucial issue, but traditional hazard prediction activities often rely on experience and intuition, making it easy to overlook risk factors. Furthermore, the lack of concrete, systematic support to effectively improve workers' safety awareness has resulted in limited improvements in site safety. Therefore, it is necessary to effectively utilize past data and automatically predict potential risks to achieve effective risk management and improve workers' safety awareness.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for collecting past similar activity data and safety records and storing them in an information storage device; means for providing an interface that enables the input of situational information from a specific site; and means for generating hazard prediction scenarios using a generative AI model based on the information and the inputted situation. This enables immediate and effective risk management and improved safety awareness by automatically predicting potential risks at the work site and presenting them visually to workers.

[0067] "Similar activity data" refers to records of similar projects or activities carried out in the past, and includes information such as the work performed, the duration, the equipment used, and any problems that occurred.

[0068] "Safety records" refer to records of safety-related incidents in past projects, including details of accidents, causes, countermeasures, and lessons learned.

[0069] An "information storage device" refers to a device that can store data for a long period of time and retrieve it as needed, and this includes hard disks, flash memory, and other similar devices.

[0070] An "interface" refers to a mechanism that provides a means for a user to operate or input information into a system, and this includes graphical user interfaces.

[0071] A "generative AI model" refers to a model that uses machine learning techniques to learn patterns from specific data and then makes predictions and decisions based on new data.

[0072] A "risk prediction scenario" refers to a presentation format that summarizes potential future risks, generated based on past data and current circumstances.

[0073] "Worker" refers to an individual responsible for a specific task or project at a construction site, and is responsible for implementing safety measures and providing input information.

[0074] This invention is a risk prediction support system aimed at improving safety at construction sites. Based on past similar activity data and safety records, this system utilizes a generated AI model to predict potential risks and provides workers with specific hazard prediction scenarios, thereby raising safety awareness and preventing accidents.

[0075] The server first collects historical data on similar activities and safety records from the database. This allows for a comprehensive understanding of problems in past operations and the effectiveness of safety measures. Database management and data processing tools such as SQL queries and Pandas are used for data collection and formatting.

[0076] Next, the terminal provides a user interface for field workers to input situation information. This interface provides workers with a means to input specific information such as location information, weather conditions, and work status at the work site using a tablet or smartphone.

[0077] Users can input data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" using their terminal. This information is collected by the server and used as input for the generated AI model.

[0078] The generating AI model uses machine learning libraries such as TENSORFLOW® and PyTorch to generate hazard prediction scenarios based on data provided by the server. These scenarios include important warnings for field workers, such as predicting the risk of scaffolding becoming unstable as wind speed increases.

[0079] As a concrete example, based on the prompt message, "Use past data on falls during work at heights to generate potential hazard scenarios and suggest risk mitigation measures," the AI ​​model can generate and present specific safety suggestions. This allows workers to immediately take countermeasures against risks.

[0080] This system allows on-site workers to independently devise and implement safety measures, resulting in a reduction in the risk of accidents at construction sites and an improvement in safety awareness.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] The server retrieves historical similar activity data and safety records from the database. This process uses SQL queries to extract the necessary data and store it in the information storage device. The input is raw data from the database, and the output is formatted structured data. Specifically, the server executes a query like "SELECT FROM safety_records WHERE date >= '2020-01-01'".

[0084] Step 2:

[0085] The server formats the acquired data into a format that the AI ​​model can process. This process uses the Pandas library to clean the data and transform it into a format suitable for machine learning models. The input is the raw dataset, and the output is an input format optimized for the AI ​​model. Specifically, it performs tasks such as imputing missing values ​​and converting categorical data to numerical values.

[0086] Step 3:

[0087] The terminal provides an interface that allows field workers to input situation information. The user interface allows for the input of specific information such as location, weather, and work status. Input is field information from the worker, and output is a new information packet sent to the server. Specifically, data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" is input.

[0088] Step 4:

[0089] The server uses a generative AI model to generate hazard prediction scenarios, taking on-site information and formatted data received from the terminal as input. TensorFlow or PyTorch are used to run the model. The input is on-site information and formatted historical data, and the output is potential hazard scenarios. Specifically, the server makes predictions such as "the scaffolding may become unstable during strong winds."

[0090] Step 5:

[0091] The terminal presents the user with generated hazard scenarios. The user interface visually displays the scenarios, and the user devises countermeasures based on them. The input is scenario data from the server, and the output is the user's confirmation and subsequent input of countermeasures. Specifically, countermeasures such as "installing safety devices" are devised for the presented scenarios.

[0092] Step 6:

[0093] The server saves the user's input response and its result to a database. This process records the data as new information, which is then used for later analysis and improvement of the AI ​​model. The input is the response and result entered by the user on the terminal, and the output is the updated database record. Specifically, it executes an SQL query such as "INSERT INTO response_log (date, location, action, result) VALUES ('2023-10-01', 'Site A', 'Safety device installed', 'Success')".

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] Improving safety awareness within factories and ensuring a safe working environment are extremely important. However, in many factories today, workers have limited means of receiving safety-related information in real time, making it difficult to immediately identify and respond to potential risks. Furthermore, the lack of well-established methods for risk prediction using historical safety data hinders the improvement of workers' safety awareness.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes means for collecting past work data and safety records and storing them in an information source; means for providing a user interface for inputting location information and environmental conditions from the work environment; means for generating hazard scenarios using a machine learning model based on the data and input information; and means for presenting hazard information to workers in real time via a visual device and prompting them to pay attention. This enables workers to immediately grasp potential risks in the work environment and take appropriate safety measures. Furthermore, by updating the risk assessment algorithm using past safety data, the accuracy of subsequent analyses can be improved, leading to a further enhancement of safety awareness.

[0099] "Work data" refers to information about the work performed within the factory and the activities of the workers, and includes data from past cases.

[0100] A "safety record" is a record of data related to past safety incidents and countermeasures.

[0101] "Information sources" refer to databases or storage systems used to store and manage collected work data and safety records.

[0102] A "user interface" is the operating environment that allows workers to input information or receive presented information.

[0103] A "machine learning model" is a program that includes algorithms for analyzing accumulated data and predicting risky scenarios.

[0104] A "risk scenario" is an indicator that shows potential risks and related situations, generated based on past and on-site information.

[0105] "Visual devices" are devices used to visually display information, and include head-mounted displays and smart glasses worn and used by workers on site.

[0106] The system that realizes this invention functions primarily through the interaction between a server, a terminal, and a user. The server first collects past work data and safety records as information sources and stores them in a database. The database can be a relational database such as PostgreSQL.

[0107] Users input location information and environmental conditions from their work environment via terminals within the factory. The terminals are equipped with an interface that allows for real-time input and verification of work status and environmental data. The terminals also transmit information to a server via an internet connection.

[0108] The server generates risk scenarios using machine learning models based on collected data and real-time input information. Machine learning frameworks such as TensorFlow and PyTorch can be used for this process. The generative AI model assesses and predicts potential risks in the work environment based on historical data and current conditions.

[0109] Users can wear visual devices to view these hazardous scenarios in real time. These devices often include smart glasses or head-mounted displays. These devices provide hazard information and countermeasures, enhancing the user's safety awareness. This allows users to take appropriate action immediately.

[0110] A concrete example would be a scenario where the system detects the risk of a worker accidentally entering a forklift aisle, displays an alarm via smart glasses, and prompts appropriate avoidance action. An example of a prompt message would be, "Predict entry into the forklift aisle and display a warning alert to the worker."

[0111] Thus, the present invention provides concrete means for improving safety within a factory, thereby enhancing workers' safety awareness and ensuring a safe working environment.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] The server collects historical work data and safety records. This information is entered from external databases and internal record systems. After collecting the data, data cleaning is performed and the format is standardized before storing it in the database. This creates a consistent information source and facilitates subsequent data analysis.

[0115] Step 2:

[0116] The user inputs location information and environmental conditions from the work environment via a terminal. Location information is obtained from a GPS module, and environmental conditions are obtained from sensors. The terminal aggregates this real-time data and converts it into a digital format. The collected information is transmitted to a server via the internet.

[0117] Step 3:

[0118] The server generates risk scenarios using a generative AI model based on historical data stored in the database and real-time data received from the terminal. The machine learning model performs risk analysis on the input data and predicts potential hazards. The generated risk scenarios are then prepared for presentation to the user.

[0119] Step 4:

[0120] The server transmits generated hazard scenarios to terminals connected to the visual device. In response, the visual device displays hazard information in real time, prompting the user to take immediate action. The user then takes safety measures based on the presented information. Specifically, smart glasses display alerts and guidance within the user's field of vision.

[0121] Step 5:

[0122] Users review hazardous scenarios presented through visual devices and take safety measures as needed. Users can input their safety measures into a terminal, which records their actions. This input is sent to a server and used to predict future risks.

[0123] Step 6:

[0124] The server stores the safety measures information received from users and their results in a database. This information is used to update algorithms and optimize risk assessments in order to improve the accuracy of the generated AI model. This information will be important data for improving the accuracy of future analyses.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention is a system that incorporates an emotion engine to recognize user emotions in order to facilitate hazard prediction activities at construction sites. Specific embodiments of this system are described below.

[0127] First, the server collects past project data and safety records from the database. This information contributes to improving the user's risk perception through the emotion engine and enables the generation of more sophisticated risk scenarios.

[0128] Next, the terminal provides an interface for field workers to input information. This interface incorporates input fields for the field situation, as well as a module for recognizing the user's emotions in real time. The emotion engine analyzes the user's facial expressions, tone of voice, and other factors to evaluate the user's psychological state.

[0129] Users input detailed on-site information via their devices and also review their own emotional information, which is recognized by the emotion engine. This information becomes a factor influencing on-site decision-making.

[0130] The server uses an AI model to generate sophisticated hazard scenarios based on on-site information and emotional data entered by users. Emotional data is a crucial element in scenario generation, considering the impact of specific emotional states on risk perception. The server also adjusts the severity of the risks according to the emotional data, providing optimal feedback to individual workers.

[0131] The device presents this generated scenario to the user and suggests specific safety measures based on their emotions. For example, if the emotion engine assesses the user's stress level, adjustments are made, such as suggesting additional warnings or taking a break from work.

[0132] Subsequently, the user formulates individual safety measures based on the presented scenario and proposed countermeasures, and records them on their device. These countermeasures and their results are stored in a database by the server and used to improve the AI ​​model and enhance the accuracy of the emotion engine for future use.

[0133] For example, if the emotion engine detects a worker is under stress while performing work at height, the server will suggest a hazard scenario such as "increasing the frequency of warnings when the wind speed exceeds safety limits." Psychological support information based on the emotion data will also be provided simultaneously.

[0134] This system allows workers to make safety decisions that take into account not only the specific risks at the site but also their emotional state. This helps prevent accidents at construction sites and improves safety awareness.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] The server collects past project data and safety records from the database. This allows it to extract information on past accident history and risk factors, preparing it for future risk assessments.

[0138] Step 2:

[0139] The terminal displays an interface for field workers to operate. This interface includes an emotion recognition module for recognizing the user's emotions, along with inputting field information.

[0140] Step 3:

[0141] The user inputs specific information collected on-site (location, weather conditions, work details, etc.) into the terminal. Additionally, an emotion recognition module automatically analyzes the user's emotional state through facial expressions and voice.

[0142] Step 4:

[0143] The server receives input field data and sentiment data, and uses AI to generate risk scenarios based on this data. Sentiment data is used to weight the scenarios and influence the perception of risk.

[0144] Step 5:

[0145] The device presents the user with risk scenarios generated from the server and specific safety measures based on emotional data. During this process, feedback and warnings are provided according to the user's psychological state.

[0146] Step 6:

[0147] Based on the presented scenario, the user considers safety measures and decides on the necessary actions. They record the decided measures on their device to prepare for the next task.

[0148] Step 7:

[0149] The server stores user feedback and implemented countermeasures in a database. This data is used for the continuous learning and improvement of the AI ​​model and emotion engine.

[0150] Step 8:

[0151] The server uses the accumulated data to refine the risk assessment algorithm, improving the accuracy of future analyses, while simultaneously optimizing project planning to take sentiment data into consideration.

[0152] (Example 2)

[0153] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0154] Improving safety management at construction sites is often difficult due to a lack of safety awareness among workers and inadequate risk perception. In such situations, conventional safety measures struggle to accurately predict potential hazards and provide optimal countermeasures for individual workers. Furthermore, the influence of workers' psychological states on safety decisions is often insufficient, resulting in many cases where accidents are not adequately prevented. This invention aims to solve these problems and achieve effective safety management.

[0155] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0156] In this invention, the server includes means for collecting past work data and safety history and storing it in an information management device, means for providing information input for inputting user information from the work site, and means for creating risk prediction information using an information processing device based on the information and input data. This makes it possible to recognize risks that take into account the individual emotional data of users and to provide appropriate feedback based on that.

[0157] "Past work data" refers to information about work performed in the past, and specifically includes records of the work content, working conditions, and results.

[0158] "Safety history" refers to records of past safety-related events and countermeasures, including accidents and near misses in the workplace, as well as the associated countermeasures and evaluations.

[0159] An "information management device" refers to a system or hardware that stores data and allows access to that data as needed, and is primarily a device with database functionality.

[0160] "Information input volume" refers to the interfaces necessary for users to input data, including text input fields, voice input devices, cameras, etc.

[0161] An "information processing device" refers to a computer system or software that analyzes and processes collected data to derive necessary information, and is the environment in which artificial intelligence algorithms are executed.

[0162] "Risk prediction information" refers to predictive information created based on collected data, which includes potential risk factors and appropriate countermeasures for those risks.

[0163] "Feedback" refers to advice, warnings, or information provided by the system to the user, including guidance to help the user take appropriate action.

[0164] This invention is a system for improving safety at construction sites, utilizing an AI model generated based on past work data and safety history. This allows users to consider their own psychological state and work environment and take appropriate risk mitigation measures.

[0165] First, the server collects past work data and safety history from the information management device and stores this information in a database. This may involve using a cloud-based database system or a local server. This accumulated information forms the basis for creating hazard prediction information using a generative AI model.

[0166] Next, the terminal provides information input to users at the work site. This interface allows workers to input the situation at the site and operates on devices such as tablets and smartphones. Input is done via text or voice, and the terminal is equipped with a camera and microphone, and implements emotion recognition functionality that analyzes the user's psychological state through their facial expressions and tone of voice.

[0167] This recognized emotion data is sent to an information processing unit on the server. The information processing unit uses a generative AI model to create risk prediction information. The AI ​​model predicts typical risk factors under specific circumstances and analyzes how those risks affect the user's psychological state.

[0168] For example, if the emotion recognition function detects a user experiencing stress while working at height, the server will generate a danger scenario such as "strengthen countermeasures in case the wind speed exceeds safety standards." Psychological support information based on emotion data will also be provided.

[0169] In this way, users can formulate individual safety measures based on the presented risk prediction information and feedback, and record them on their devices. This improves the overall accuracy of the system and the safety awareness of the users.

[0170] As an example of a prompt, the AI ​​model can be input with a message like, "Please provide the psychological state of a worker when strong winds blow while they are working at height, and the associated risk scenarios," to obtain appropriate risk prediction information.

[0171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0172] Step 1:

[0173] The server collects historical work data and safety history from the database. It takes work details and safety-related information from the existing database as input and processes it into a dataset. The output is a data file containing aggregated information necessary for risk analysis. This data is then used for subsequent analysis by an AI model.

[0174] Step 2:

[0175] The terminal provides field workers with a means of inputting information. Users input information about the surrounding environment and work details. Inputs include real-time data from the site and a brief health status report of the worker. Output is a data stream in which the input information is sent to the server in an appropriate format. The terminal also utilizes information from the camera and microphone as needed.

[0176] Step 3:

[0177] The emotion recognition function built into the device evaluates the user's psychological state. This includes acquiring facial expression data from the camera and analyzing voice tone from the microphone. The input is real-time visual and auditory data from the user. The output is evaluation data indicating the user's emotional state, which is sent to the server.

[0178] Step 4:

[0179] The server generates hazard prediction information using an AI model based on acquired field data and sentiment data. Input information includes user information and sentiment recognition data transmitted from the terminal, as well as past work data. Statistical methods and machine learning algorithms are used for data processing to predict specific risk factors. The output consists of the generated hazard prediction scenario and its supporting data, which is then returned to the terminal.

[0180] Step 5:

[0181] The terminal presents the user with generated risk prediction information and safety measures. The user is required to review the proposed countermeasures and consider additional safety measures based on their own judgment. The input is the prediction information provided by the server. The output is the countermeasure information that the user reviews and, if necessary, enters into the terminal. The terminal then sends this information back to the server.

[0182] Step 6:

[0183] The server stores newly entered countermeasures in a database and uses them to improve the AI ​​model for future analyses. This enables the provision of even more accurate risk prediction scenarios in subsequent analyses. The input consists of user feedback and countermeasure data, and the output is the result of analyzing and appropriately storing this data.

[0184] (Application Example 2)

[0185] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0186] In modern work environments, it is essential to properly manage the interaction between workers' emotions and safety measures. However, conventional safety management systems have struggled to recognize workers' psychological states in real time and provide immediate feedback accordingly. As a result, there is a challenge in implementing effective safety measures that take into account fluctuations in risk due to workers' emotional states.

[0187] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0188] In this invention, the server includes means for collecting past work data and safety records and storing them in an information storage means, means for providing an information input device for inputting relevant information from the work site, and means for recognizing the emotional state of technicians in real time and generating psychological feedback. This enables the immediate presentation of hazard scenarios based on the worker's psychological state and personalized safety feedback based on emotions.

[0189] "Past operational data" refers to the history of work performed at the work site, details of the work, and related information, and is an information resource that forms the basis for system improvement and risk assessment.

[0190] "Safety records" refer to documented information about past safety-related incidents and the measures taken to address them, and serve as important reference information for future safety management.

[0191] "Information storage means" refers to digital or physical storage devices used to collect and store various data and information, such as databases.

[0192] An "information input device" is a device equipped with a user interface for workers to input on-site conditions and specific data into a system, and this includes computer terminals and tablets.

[0193] Machine learning is a technology that improves the accuracy of specific tasks by using large amounts of data to enable computers to self-improve.

[0194] A "risk scenario" refers to a plan or set of countermeasures that include predictive information about dangerous events that may occur under specific conditions, and plays an important role in ensuring safety during work.

[0195] "The emotional state of engineers" refers to the psychological and emotional state that workers experience while performing their tasks, and appropriately understanding this state contributes to improving workplace safety.

[0196] "Psychological feedback" refers to providing appropriate advice and information based on the worker's psychological state, with the aim of avoiding risks and improving safety.

[0197] This invention is a system that improves workplace safety by providing immediate and appropriate feedback according to the worker's emotional state and work environment. The server collects past work data and safety records and stores them in an information storage means. This prepares the basic information necessary for data analysis. The server receives relevant information from workers on site through an information input device. The information input device includes smart glasses and mobile terminals, which workers can use to input their current work situation and psychological state.

[0198] The server uses Google Cloud's Vision API and Speech-to-Text API to recognize the worker's emotional state in real time. This allows for the evaluation of their psychological state through facial expression analysis and voice analysis. Furthermore, machine learning technology is used to generate risk scenarios based on this data. These generated risk scenarios are presented to the worker as psychological feedback, further enhancing workplace safety.

[0199] For example, if a worker experiences a problem while performing maintenance on complex machinery and feels anxious, the system recognizes this state through its emotion engine and displays feedback on the worker's display such as, "Take a deep breath and take a break." Similarly, when a worker experiences tension when performing a new work procedure for the first time, advice such as, "Work at your own pace," is displayed to reduce stress.

[0200] An example of a prompt message for a generative AI model is: "Use the worker's facial image and voice data to evaluate their psychological state (stress, anxiety, tension, etc.) and create and provide appropriate feedback."

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The server collects historical operational data and safety records from information storage systems. This provides the basic information necessary for data analysis. This data is used for risk assessment and training machine learning models.

[0204] Step 2:

[0205] The terminal receives information about the worker's work status through an input interface. This includes data related to the work content, location, and current psychological state. The terminal sends this information to a server, which is then prepared for further analysis.

[0206] Step 3:

[0207] The server uses received work information and machine learning techniques to recognize the worker's emotional state in real time. It performs facial expression analysis using Google Cloud's Vision API and speech analysis using the Speech-to-Text API. This identifies the worker's psychological state, such as stress levels and anxiety.

[0208] Step 4:

[0209] The server combines emotional data with historical operational data and uses a generative AI model to generate risk scenarios. Based on the results of the emotional analysis, the optimal risk scenario is created, and suggestions such as warnings during work or the need for rest are made.

[0210] Step 5:

[0211] The terminal displays risk scenarios and feedback sent from the server to the worker. The worker reviews this information directly on the display and applies it as needed. The feedback includes specific safety actions and psychological support information.

[0212] Step 6:

[0213] Users implement safety measures based on the feedback provided and record the results from their device to the server. This verifies the effectiveness of the countermeasures and contributes to improving future risk assessments and the accuracy of feedback.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention is a system that supports hazard prediction activities to improve safety awareness at construction sites, and analyzes risks using artificial intelligence based on past project data and safety records. A specific embodiment of this system is described below.

[0231] First, the server collects past project data and safety records from the database. This allows for a comprehensive understanding of past cases and key safety points at construction sites. The server then formats the collected information for analysis, making it usable by AI models.

[0232] Next, the terminal provides an interface that allows field workers to input information. The user (worker) uses this interface to input specific information such as the site's location, weather conditions, and work status. This field information then becomes an important data source for the AI.

[0233] The server uses artificial intelligence to generate hazard scenarios based on this information. The generated hazard scenarios take into account different risk factors and serve to attract the user's attention and raise safety awareness by visualizing potential dangers.

[0234] Next, the device presents the user with this risk scenario. Based on the presented scenario, the user considers appropriate countermeasures and records a specific action plan on the device. This enables the user to independently consider and implement risk mitigation measures.

[0235] Subsequently, the server stores the entered countermeasures and their implementation results in a database. This data will be used to improve future AI models and develop new risk assessment algorithms, enabling more accurate risk prediction.

[0236] For example, if work is being done at a high altitude at a certain site, the server uses data from past fall accidents during high-altitude work to generate a dangerous scenario where "the scaffolding becomes unstable in strong winds." The user reviews this scenario, devises countermeasures such as installing safety devices according to the wind speed or temporarily suspending work, and inputs them into the terminal. The server records this information and uses it for future risk assessments.

[0237] In this way, this system provides effective measures to prevent accidents at construction sites while raising safety awareness.

[0238] The following describes the processing flow.

[0239] Step 1:

[0240] The server collects historical project data and safety records from the database. It then formats this data and converts it into a format that the AI ​​model can use for analysis.

[0241] Step 2:

[0242] The terminal provides the user with an interface for inputting specific information at the work site. This interface allows input of location information, weather conditions for the day, and the health status of the workers.

[0243] Step 3:

[0244] Users input information about the field situation through the terminal interface. They meticulously record any unusual circumstances or important information at the site.

[0245] Step 4:

[0246] The server receives information entered by the user, integrates it with historical data, and analyzes it. Using an AI model, it generates hazard scenarios based on the actual situation on site.

[0247] Step 5:

[0248] The device presents the generated risk scenario to the user. The user reviews the risk information and its background described in the displayed scenario.

[0249] Step 6:

[0250] Based on the presented hazard scenarios, the user considers safety measures and decides on specific countermeasures. The decided countermeasures are then entered into the terminal and recorded.

[0251] Step 7:

[0252] The server stores the countermeasures entered by users and the results of their implementation in a database. This accumulated data will be used to improve future AI models and enhance the accuracy of risk assessments.

[0253] Step 8:

[0254] The server regularly updates its risk assessment algorithm to achieve more accurate hazard prediction. It also encourages user participation through a points system and awards program, aiming to improve safety awareness.

[0255] (Example 1)

[0256] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0257] Ensuring safety at construction sites is a crucial issue, but traditional hazard prediction activities often rely on experience and intuition, making it easy to overlook risk factors. Furthermore, the lack of concrete, systematic support to effectively improve workers' safety awareness has resulted in limited improvements in site safety. Therefore, it is necessary to effectively utilize past data and automatically predict potential risks to achieve effective risk management and improve workers' safety awareness.

[0258] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0259] In this invention, the server includes means for collecting past similar activity data and safety records and storing them in an information storage device; means for providing an interface that enables the input of situational information from a specific site; and means for generating hazard prediction scenarios using a generative AI model based on the information and the inputted situation. This enables immediate and effective risk management and improved safety awareness by automatically predicting potential risks at the work site and presenting them visually to workers.

[0260] "Similar activity data" refers to records of similar projects or activities carried out in the past, and includes information such as the work performed, the duration, the equipment used, and any problems that occurred.

[0261] "Safety records" refer to records of safety-related incidents in past projects, including details of accidents, causes, countermeasures, and lessons learned.

[0262] An "information storage device" refers to a device that can store data for a long period of time and retrieve it as needed, and this includes hard disks, flash memory, and other similar devices.

[0263] An "interface" refers to a mechanism that provides a means for a user to operate or input information into a system, and this includes graphical user interfaces.

[0264] A "generative AI model" refers to a model that uses machine learning techniques to learn patterns from specific data and then makes predictions and decisions based on new data.

[0265] A "risk prediction scenario" refers to a presentation format that summarizes potential future risks, generated based on past data and current circumstances.

[0266] "Worker" refers to an individual responsible for a specific task or project at a construction site, and is responsible for implementing safety measures and providing input information.

[0267] This invention is a risk prediction support system aimed at improving safety at construction sites. Based on past similar activity data and safety records, this system utilizes a generated AI model to predict potential risks and provides workers with specific hazard prediction scenarios, thereby raising safety awareness and preventing accidents.

[0268] The server first collects historical data on similar activities and safety records from the database. This allows for a comprehensive understanding of problems in past operations and the effectiveness of safety measures. Database management and data processing tools such as SQL queries and Pandas are used for data collection and formatting.

[0269] Next, the terminal provides a user interface for field workers to input situation information. This interface provides workers with a means to input specific information such as location information, weather conditions, and work status at the work site using a tablet or smartphone.

[0270] Users can input data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" using their terminal. This information is collected by the server and used as input for the generated AI model.

[0271] The generative AI model uses machine learning libraries such as TensorFlow and PyTorch to generate hazard prediction scenarios based on data provided by the server. These scenarios include important warnings for field workers, such as predicting the risk of scaffolding becoming unstable as wind speed increases.

[0272] As a concrete example, based on the prompt message, "Use past data on falls during work at heights to generate potential hazard scenarios and suggest risk mitigation measures," the AI ​​model can generate and present specific safety suggestions. This allows workers to immediately take countermeasures against risks.

[0273] This system allows on-site workers to independently devise and implement safety measures, resulting in a reduction in the risk of accidents at construction sites and an improvement in safety awareness.

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] The server retrieves historical similar activity data and safety records from the database. This process uses SQL queries to extract the necessary data and store it in the information storage device. The input is raw data from the database, and the output is formatted structured data. Specifically, the server executes a query like "SELECT FROM safety_records WHERE date >= '2020-01-01'".

[0277] Step 2:

[0278] The server formats the acquired data into a format that the AI ​​model can process. This process uses the Pandas library to clean the data and transform it into a format suitable for machine learning models. The input is the raw dataset, and the output is an input format optimized for the AI ​​model. Specifically, it performs tasks such as imputing missing values ​​and converting categorical data to numerical values.

[0279] Step 3:

[0280] The terminal provides an interface that allows field workers to input situation information. The user interface allows for the input of specific information such as location, weather, and work status. Input is field information from the worker, and output is a new information packet sent to the server. Specifically, data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" is input.

[0281] Step 4:

[0282] The server uses the generated AI model to generate a risk prediction scenario with the on-site information received from the terminal and the formatted data as inputs. TensorFlow or PyTorch is used for model execution. The inputs are the on-site information and the formatted historical data, and the output is the potential risk scenario. As a specific operation, the server makes predictions such as "The scaffolding may become unstable during strong winds."

[0283] Step 5:

[0284] The terminal presents the generated risk scenario to the user. The scenario is visually displayed on the user interface, and the user devises countermeasures based on this. The input is the scenario data from the server, and the output is the user's confirmation and subsequent input of countermeasures. Specifically, countermeasures such as "Installation of safety devices" for the presented scenario are devised.

[0285] Step 6:

[0286] The server saves the countermeasures and their results input by the user in the database. In this process, it records as new data for later analysis and improvement of the AI model. The input is the countermeasures and results input by the user to the terminal, and the output is the updated database record. As a specific operation, it executes an SQL query such as "INSERT INTO response_log (date, location, action, result) VALUES ('2023-10-01', 'Site A','safety device installation','success')."

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server," and the smart glasses 214 are referred to as the "terminal."

[0289] Improving safety awareness within factories and ensuring a safe working environment are extremely important. However, in many factories today, workers have limited means of receiving safety-related information in real time, making it difficult to immediately identify and respond to potential risks. Furthermore, the lack of well-established methods for risk prediction using historical safety data hinders the improvement of workers' safety awareness.

[0290] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0291] In this invention, the server includes means for collecting past work data and safety records and storing them in an information source; means for providing a user interface for inputting location information and environmental conditions from the work environment; means for generating hazard scenarios using a machine learning model based on the data and input information; and means for presenting hazard information to workers in real time via a visual device and prompting them to pay attention. This enables workers to immediately grasp potential risks in the work environment and take appropriate safety measures. Furthermore, by updating the risk assessment algorithm using past safety data, the accuracy of subsequent analyses can be improved, leading to a further enhancement of safety awareness.

[0292] "Work data" refers to information about the work performed within the factory and the activities of the workers, and includes data from past cases.

[0293] A "safety record" is a record of data related to past safety incidents and countermeasures.

[0294] "Information sources" refer to databases or storage systems used to store and manage collected work data and safety records.

[0295] A "user interface" is the operating environment that allows workers to input information or receive presented information.

[0296] A "machine learning model" is a program that includes algorithms for analyzing accumulated data and predicting risky scenarios.

[0297] A "risk scenario" is an indicator that shows potential risks and related situations, generated based on past and on-site information.

[0298] "Visual devices" are devices used to visually display information, and include head-mounted displays and smart glasses worn and used by workers on site.

[0299] The system that realizes this invention functions primarily through the interaction between a server, a terminal, and a user. The server first collects past work data and safety records as information sources and stores them in a database. The database can be a relational database such as PostgreSQL.

[0300] Users input location information and environmental conditions from their work environment via terminals within the factory. The terminals are equipped with an interface that allows for real-time input and verification of work status and environmental data. The terminals also transmit information to a server via an internet connection.

[0301] The server generates risk scenarios using machine learning models based on collected data and real-time input information. Machine learning frameworks such as TensorFlow and PyTorch can be used for this process. The generative AI model assesses and predicts potential risks in the work environment based on historical data and current conditions.

[0302] The user can wear a visual device and check this dangerous scenario in real time. As the visual device, smart glasses or head-mounted displays are often used. This visual device plays a role in presenting danger information and countermeasures and enhancing the user's safety awareness. As a result, the user can immediately take appropriate countermeasures.

[0303] As a specific example, a scenario can be considered where the system detects the danger that an operator accidentally enters the forklift passage, displays an alarm through smart glasses, and prompts appropriate avoidance actions. An example of a prompt sentence is "Predict entry into the forklift passage and display an alert to arouse the operator's attention."

[0304] In this way, the present invention provides specific means for improving the safety in the factory, enhances the safety awareness of the operator, and ensures the safety of the working environment.

[0305] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0306] Step 1:

[0307] The server collects past work data and safety records. This information is input from an external database or an in-house record system. After collecting the data, before storing it in the database, data cleaning is performed to unify the format. As a result, an information source is formed, making subsequent data analysis easier.

[0308] Step 2:

[0309] The user inputs position information and environmental conditions from the working environment through a terminal. The position information is obtained from a GPS module, and the environmental conditions are inputs obtained from sensors. The terminal aggregates these real-time data and converts the data into a digital format. The collected information is transmitted to the server via the Internet.

[0310] Step 3:

[0311] The server generates risk scenarios using a generative AI model based on historical data stored in the database and real-time data received from the terminal. The machine learning model performs risk analysis on the input data and predicts potential hazards. The generated risk scenarios are then prepared for presentation to the user.

[0312] Step 4:

[0313] The server transmits generated hazard scenarios to terminals connected to the visual device. In response, the visual device displays hazard information in real time, prompting the user to take immediate action. The user then takes safety measures based on the presented information. Specifically, smart glasses display alerts and guidance within the user's field of vision.

[0314] Step 5:

[0315] Users review hazardous scenarios presented through visual devices and take safety measures as needed. Users can input their safety measures into a terminal, which records their actions. This input is sent to a server and used to predict future risks.

[0316] Step 6:

[0317] The server stores the safety measures information received from users and their results in a database. This information is used to update algorithms and optimize risk assessments in order to improve the accuracy of the generated AI model. This information will be important data for improving the accuracy of future analyses.

[0318] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0319] This invention is a system that incorporates an emotion engine to recognize user emotions in order to facilitate hazard prediction activities at construction sites. Specific embodiments of this system are described below.

[0320] First, the server collects past project data and safety records from the database. This information contributes to improving the user's risk perception through the emotion engine and enables the generation of more sophisticated risk scenarios.

[0321] Next, the terminal provides an interface for field workers to input information. This interface incorporates input fields for the field situation, as well as a module for recognizing the user's emotions in real time. The emotion engine analyzes the user's facial expressions, tone of voice, and other factors to evaluate the user's psychological state.

[0322] Users input detailed on-site information via their devices and also review their own emotional information, which is recognized by the emotion engine. This information becomes a factor influencing on-site decision-making.

[0323] The server uses an AI model to generate sophisticated hazard scenarios based on on-site information and emotional data entered by users. Emotional data is a crucial element in scenario generation, considering the impact of specific emotional states on risk perception. The server also adjusts the severity of the risks according to the emotional data, providing optimal feedback to individual workers.

[0324] The device presents this generated scenario to the user and suggests specific safety measures based on their emotions. For example, if the emotion engine assesses the user's stress level, adjustments are made, such as suggesting additional warnings or taking a break from work.

[0325] Subsequently, the user formulates individual safety measures based on the presented scenario and proposed countermeasures, and records them on their device. These countermeasures and their results are stored in a database by the server and used to improve the AI ​​model and enhance the accuracy of the emotion engine for future use.

[0326] For example, if the emotion engine detects a worker is under stress while performing work at height, the server will suggest a hazard scenario such as "increasing the frequency of warnings when the wind speed exceeds safety limits." Psychological support information based on the emotion data will also be provided simultaneously.

[0327] This system allows workers to make safety decisions that take into account not only the specific risks at the site but also their emotional state. This helps prevent accidents at construction sites and improves safety awareness.

[0328] The following describes the processing flow.

[0329] Step 1:

[0330] The server collects past project data and safety records from the database. This allows it to extract information on past accident history and risk factors, preparing it for future risk assessments.

[0331] Step 2:

[0332] The terminal displays an interface for field workers to operate. This interface includes an emotion recognition module for recognizing the user's emotions, along with inputting field information.

[0333] Step 3:

[0334] The user inputs specific information collected on-site (location, weather conditions, work details, etc.) into the terminal. Additionally, an emotion recognition module automatically analyzes the user's emotional state through facial expressions and voice.

[0335] Step 4:

[0336] The server receives input field data and sentiment data, and uses AI to generate risk scenarios based on this data. Sentiment data is used to weight the scenarios and influence the perception of risk.

[0337] Step 5:

[0338] The device presents the user with risk scenarios generated from the server and specific safety measures based on emotional data. During this process, feedback and warnings are provided according to the user's psychological state.

[0339] Step 6:

[0340] Based on the presented scenario, the user considers safety measures and decides on the necessary actions. They record the decided measures on their device to prepare for the next task.

[0341] Step 7:

[0342] The server stores user feedback and implemented countermeasures in a database. This data is used for the continuous learning and improvement of the AI ​​model and emotion engine.

[0343] Step 8:

[0344] The server uses the accumulated data to refine the risk assessment algorithm, improving the accuracy of future analyses, while simultaneously optimizing project planning to take sentiment data into consideration.

[0345] (Example 2)

[0346] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0347] Improving safety management at construction sites is often difficult due to a lack of safety awareness among workers and inadequate risk perception. In such situations, conventional safety measures struggle to accurately predict potential hazards and provide optimal countermeasures for individual workers. Furthermore, the influence of workers' psychological states on safety decisions is often insufficient, resulting in many cases where accidents are not adequately prevented. This invention aims to solve these problems and achieve effective safety management.

[0348] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0349] In this invention, the server includes means for collecting past work data and safety history and storing it in an information management device, means for providing information input for inputting user information from the work site, and means for creating risk prediction information using an information processing device based on the information and input data. This makes it possible to recognize risks that take into account the individual emotional data of users and to provide appropriate feedback based on that.

[0350] "Past work data" refers to information about work performed in the past, and specifically includes records of the work content, working conditions, and results.

[0351] "Safety history" refers to records of past safety-related events and countermeasures, including accidents and near misses in the workplace, as well as the associated countermeasures and evaluations.

[0352] An "information management device" refers to a system or hardware that stores data and allows access to that data as needed, and is primarily a device with database functionality.

[0353] "Information input volume" refers to the interfaces necessary for users to input data, including text input fields, voice input devices, cameras, etc.

[0354] An "information processing device" refers to a computer system or software that analyzes and processes collected data to derive necessary information, and is the environment in which artificial intelligence algorithms are executed.

[0355] "Risk prediction information" refers to predictive information created based on collected data, which includes potential risk factors and appropriate countermeasures for those risks.

[0356] "Feedback" refers to advice, warnings, or information provided by the system to the user, including guidance to help the user take appropriate action.

[0357] This invention is a system for improving safety at construction sites, utilizing an AI model generated based on past work data and safety history. This allows users to consider their own psychological state and work environment and take appropriate risk mitigation measures.

[0358] First, the server collects past work data and safety history from the information management device and stores this information in a database. This may involve using a cloud-based database system or a local server. This accumulated information forms the basis for creating hazard prediction information using a generative AI model.

[0359] Next, the terminal provides information input to users at the work site. This interface allows workers to input the situation at the site and operates on devices such as tablets and smartphones. Input is done via text or voice, and the terminal is equipped with a camera and microphone, and implements emotion recognition functionality that analyzes the user's psychological state through their facial expressions and tone of voice.

[0360] This recognized emotion data is sent to an information processing unit on the server. The information processing unit uses a generative AI model to create risk prediction information. The AI ​​model predicts typical risk factors under specific circumstances and analyzes how those risks affect the user's psychological state.

[0361] For example, if the emotion recognition function detects a user experiencing stress while working at height, the server will generate a danger scenario such as "strengthen countermeasures in case the wind speed exceeds safety standards." Psychological support information based on emotion data will also be provided.

[0362] In this way, users can formulate individual safety measures based on the presented risk prediction information and feedback, and record them on their devices. This improves the overall accuracy of the system and the safety awareness of the users.

[0363] As an example of a prompt, the AI ​​model can be input with a message like, "Please provide the psychological state of a worker when strong winds blow while they are working at height, and the associated risk scenarios," to obtain appropriate risk prediction information.

[0364] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0365] Step 1:

[0366] The server collects historical work data and safety history from the database. It takes work details and safety-related information from the existing database as input and processes it into a dataset. The output is a data file containing aggregated information necessary for risk analysis. This data is then used for subsequent analysis by an AI model.

[0367] Step 2:

[0368] The terminal provides field workers with a means of inputting information. Users input information about the surrounding environment and work details. Inputs include real-time data from the site and a brief health status report of the worker. Output is a data stream in which the input information is sent to the server in an appropriate format. The terminal also utilizes information from the camera and microphone as needed.

[0369] Step 3:

[0370] The emotion recognition function built into the device evaluates the user's psychological state. This includes acquiring facial expression data from the camera and analyzing voice tone from the microphone. The input is real-time visual and auditory data from the user. The output is evaluation data indicating the user's emotional state, which is sent to the server.

[0371] Step 4:

[0372] The server generates hazard prediction information using an AI model based on acquired field data and sentiment data. Input information includes user information and sentiment recognition data transmitted from the terminal, as well as past work data. Statistical methods and machine learning algorithms are used for data processing to predict specific risk factors. The output consists of the generated hazard prediction scenario and its supporting data, which is then returned to the terminal.

[0373] Step 5:

[0374] The terminal presents the user with generated risk prediction information and safety measures. The user is required to review the proposed countermeasures and consider additional safety measures based on their own judgment. The input is the prediction information provided by the server. The output is the countermeasure information that the user reviews and, if necessary, enters into the terminal. The terminal then sends this information back to the server.

[0375] Step 6:

[0376] The server stores newly entered countermeasures in a database and uses them to improve the AI ​​model for future analyses. This enables the provision of even more accurate risk prediction scenarios in subsequent analyses. The input consists of user feedback and countermeasure data, and the output is the result of analyzing and appropriately storing this data.

[0377] (Application Example 2)

[0378] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0379] In modern work environments, it is essential to properly manage the interaction between workers' emotions and safety measures. However, conventional safety management systems have struggled to recognize workers' psychological states in real time and provide immediate feedback accordingly. As a result, there is a challenge in implementing effective safety measures that take into account fluctuations in risk due to workers' emotional states.

[0380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0381] In this invention, the server includes means for collecting past work data and safety records and storing them in an information storage means, means for providing an information input device for inputting relevant information from the work site, and means for recognizing the emotional state of technicians in real time and generating psychological feedback. This enables the immediate presentation of hazard scenarios based on the worker's psychological state and personalized safety feedback based on emotions.

[0382] "Past operational data" refers to the history of work performed at the work site, details of the work, and related information, and is an information resource that forms the basis for system improvement and risk assessment.

[0383] "Safety records" refer to documented information about past safety-related incidents and the measures taken to address them, and serve as important reference information for future safety management.

[0384] "Information storage means" refers to digital or physical storage devices used to collect and store various data and information, such as databases.

[0385] An "information input device" is a device equipped with a user interface for workers to input on-site conditions and specific data into a system, and this includes computer terminals and tablets.

[0386] Machine learning is a technology that improves the accuracy of specific tasks by using large amounts of data to enable computers to self-improve.

[0387] A "risk scenario" refers to a plan or set of countermeasures that include predictive information about dangerous events that may occur under specific conditions, and plays an important role in ensuring safety during work.

[0388] "The emotional state of engineers" refers to the psychological and emotional state that workers experience while performing their tasks, and appropriately understanding this state contributes to improving workplace safety.

[0389] "Psychological feedback" refers to providing appropriate advice and information based on the worker's psychological state, with the aim of avoiding risks and improving safety.

[0390] This invention is a system that improves workplace safety by providing immediate and appropriate feedback according to the worker's emotional state and work environment. The server collects past work data and safety records and stores them in an information storage means. This prepares the basic information necessary for data analysis. The server receives relevant information from workers on site through an information input device. The information input device includes smart glasses and mobile terminals, which workers can use to input their current work situation and psychological state.

[0391] The server uses Google Cloud's Vision API and Speech-to-Text API to recognize the emotional state of workers in real time. This allows for the evaluation of their psychological state through facial expression and voice analysis. Furthermore, machine learning technology is used to generate risk scenarios based on this data. These generated risk scenarios are presented to workers as psychological feedback, further enhancing workplace safety.

[0392] For example, if a worker experiences a problem while performing maintenance on complex machinery and feels anxious, the system recognizes this state through its emotion engine and displays feedback on the worker's display such as, "Take a deep breath and take a break." Similarly, when a worker experiences tension when performing a new work procedure for the first time, advice such as, "Work at your own pace," is displayed to reduce stress.

[0393] An example of a prompt message for a generative AI model is: "Use the worker's facial image and voice data to evaluate their psychological state (stress, anxiety, tension, etc.) and create and provide appropriate feedback."

[0394] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0395] Step 1:

[0396] The server collects historical operational data and safety records from information storage systems. This provides the basic information necessary for data analysis. This data is used for risk assessment and training machine learning models.

[0397] Step 2:

[0398] The terminal receives information about the worker's work status through an input interface. This includes data related to the work content, location, and current psychological state. The terminal sends this information to a server, which is then prepared for further analysis.

[0399] Step 3:

[0400] The server uses received work information and machine learning techniques to recognize the worker's emotional state in real time. It performs facial expression analysis using Google Cloud's Vision API and speech analysis using the Speech-to-Text API. This identifies the worker's psychological state, such as stress levels and anxiety.

[0401] Step 4:

[0402] The server combines emotional data with historical operational data and uses a generative AI model to generate risk scenarios. Based on the results of the emotional analysis, the optimal risk scenario is created, and suggestions such as warnings during work or the need for rest are made.

[0403] Step 5:

[0404] The terminal displays risk scenarios and feedback sent from the server to the worker. The worker reviews this information directly on the display and applies it as needed. The feedback includes specific safety actions and psychological support information.

[0405] Step 6:

[0406] Users implement safety measures based on the feedback provided and record the results from their device to the server. This verifies the effectiveness of the countermeasures and contributes to improving future risk assessments and the accuracy of feedback.

[0407] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0408] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0409] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0410] [Third Embodiment]

[0411] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0412] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0413] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0414] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0415] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0416] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0417] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0418] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0419] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0420] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0421] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0422] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0423] This invention is a system that supports hazard prediction activities to improve safety awareness at construction sites, and analyzes risks using artificial intelligence based on past project data and safety records. A specific embodiment of this system is described below.

[0424] First, the server collects past project data and safety records from the database. This allows for a comprehensive understanding of past cases and key safety points at construction sites. The server then formats the collected information for analysis, making it usable by AI models.

[0425] Next, the terminal provides an interface that allows field workers to input information. The user (worker) uses this interface to input specific information such as the site's location, weather conditions, and work status. This field information then becomes an important data source for the AI.

[0426] The server uses artificial intelligence to generate hazard scenarios based on this information. The generated hazard scenarios take into account different risk factors and serve to attract the user's attention and raise safety awareness by visualizing potential dangers.

[0427] Next, the device presents the user with this risk scenario. Based on the presented scenario, the user considers appropriate countermeasures and records a specific action plan on the device. This enables the user to independently consider and implement risk mitigation measures.

[0428] Subsequently, the server stores the entered countermeasures and their implementation results in a database. This data will be used to improve future AI models and develop new risk assessment algorithms, enabling more accurate risk prediction.

[0429] For example, if work is being done at a high altitude at a certain site, the server uses data from past fall accidents during high-altitude work to generate a dangerous scenario where "the scaffolding becomes unstable in strong winds." The user reviews this scenario, devises countermeasures such as installing safety devices according to the wind speed or temporarily suspending work, and inputs them into the terminal. The server records this information and uses it for future risk assessments.

[0430] In this way, this system provides effective measures to prevent accidents at construction sites while raising safety awareness.

[0431] The following describes the processing flow.

[0432] Step 1:

[0433] The server collects historical project data and safety records from the database. It then formats this data and converts it into a format that the AI ​​model can use for analysis.

[0434] Step 2:

[0435] The terminal provides the user with an interface for inputting specific information at the work site. This interface allows input of location information, weather conditions for the day, and the health status of the workers.

[0436] Step 3:

[0437] Users input information about the field situation through the terminal interface. They meticulously record any unusual circumstances or important information at the site.

[0438] Step 4:

[0439] The server receives information entered by the user, integrates it with historical data, and analyzes it. Using an AI model, it generates hazard scenarios based on the actual situation on site.

[0440] Step 5:

[0441] The device presents the generated risk scenario to the user. The user reviews the risk information and its background described in the displayed scenario.

[0442] Step 6:

[0443] Based on the presented hazard scenarios, the user considers safety measures and decides on specific countermeasures. The decided countermeasures are then entered into the terminal and recorded.

[0444] Step 7:

[0445] The server stores the countermeasures entered by users and the results of their implementation in a database. This accumulated data will be used to improve future AI models and enhance the accuracy of risk assessments.

[0446] Step 8:

[0447] The server regularly updates its risk assessment algorithm to achieve more accurate hazard prediction. It also encourages user participation through a points system and awards program, aiming to improve safety awareness.

[0448] (Example 1)

[0449] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0450] Ensuring safety at construction sites is a crucial issue, but traditional hazard prediction activities often rely on experience and intuition, making it easy to overlook risk factors. Furthermore, the lack of concrete, systematic support to effectively improve workers' safety awareness has resulted in limited improvements in site safety. Therefore, it is necessary to effectively utilize past data and automatically predict potential risks to achieve effective risk management and improve workers' safety awareness.

[0451] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0452] In this invention, the server includes means for collecting past similar activity data and safety records and storing them in an information storage device; means for providing an interface that enables the input of situational information from a specific site; and means for generating hazard prediction scenarios using a generative AI model based on the information and the inputted situation. This enables immediate and effective risk management and improved safety awareness by automatically predicting potential risks at the work site and presenting them visually to workers.

[0453] "Similar activity data" refers to records of similar projects or activities carried out in the past, and includes information such as the work performed, the duration, the equipment used, and any problems that occurred.

[0454] "Safety records" refer to records of safety-related incidents in past projects, including details of accidents, causes, countermeasures, and lessons learned.

[0455] An "information storage device" refers to a device that can store data for a long period of time and retrieve it as needed, and this includes hard disks, flash memory, and other similar devices.

[0456] An "interface" refers to a mechanism that provides a means for a user to operate or input information into a system, and this includes graphical user interfaces.

[0457] A "generative AI model" refers to a model that uses machine learning techniques to learn patterns from specific data and then makes predictions and decisions based on new data.

[0458] A "risk prediction scenario" refers to a presentation format that summarizes potential future risks, generated based on past data and current circumstances.

[0459] "Worker" refers to an individual responsible for a specific task or project at a construction site, and is responsible for implementing safety measures and providing input information.

[0460] This invention is a risk prediction support system aimed at improving safety at construction sites. Based on past similar activity data and safety records, this system utilizes a generated AI model to predict potential risks and provides workers with specific hazard prediction scenarios, thereby raising safety awareness and preventing accidents.

[0461] The server first collects historical data on similar activities and safety records from the database. This allows for a comprehensive understanding of problems in past operations and the effectiveness of safety measures. Database management and data processing tools such as SQL queries and Pandas are used for data collection and formatting.

[0462] Next, the terminal provides a user interface for field workers to input situation information. This interface provides workers with a means to input specific information such as location information, weather conditions, and work status at the work site using a tablet or smartphone.

[0463] Users can input data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" using their terminal. This information is collected by the server and used as input for the generated AI model.

[0464] The generative AI model uses machine learning libraries such as TensorFlow and PyTorch to generate hazard prediction scenarios based on data provided by the server. These scenarios include important warnings for field workers, such as predicting the risk of scaffolding becoming unstable as wind speed increases.

[0465] As a concrete example, based on the prompt message, "Use past data on falls during work at heights to generate potential hazard scenarios and suggest risk mitigation measures," the AI ​​model can generate and present specific safety suggestions. This allows workers to immediately take countermeasures against risks.

[0466] This system allows on-site workers to independently devise and implement safety measures, resulting in a reduction in the risk of accidents at construction sites and an improvement in safety awareness.

[0467] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0468] Step 1:

[0469] The server retrieves historical similar activity data and safety records from the database. This process uses SQL queries to extract the necessary data and store it in the information storage device. The input is raw data from the database, and the output is formatted structured data. Specifically, the server executes a query like "SELECT FROM safety_records WHERE date >= '2020-01-01'".

[0470] Step 2:

[0471] The server formats the acquired data into a format that the AI ​​model can process. This process uses the Pandas library to clean the data and transform it into a format suitable for machine learning models. The input is the raw dataset, and the output is an input format optimized for the AI ​​model. Specifically, it performs tasks such as imputing missing values ​​and converting categorical data to numerical values.

[0472] Step 3:

[0473] The terminal provides an interface that allows field workers to input situation information. The user interface allows for the input of specific information such as location, weather, and work status. Input is field information from the worker, and output is a new information packet sent to the server. Specifically, data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" is input.

[0474] Step 4:

[0475] The server uses a generative AI model to generate hazard prediction scenarios, taking on-site information and formatted data received from the terminal as input. TensorFlow or PyTorch are used to run the model. The input is on-site information and formatted historical data, and the output is potential hazard scenarios. Specifically, the server makes predictions such as "the scaffolding may become unstable during strong winds."

[0476] Step 5:

[0477] The terminal presents the user with generated hazard scenarios. The user interface visually displays the scenarios, and the user devises countermeasures based on them. The input is scenario data from the server, and the output is the user's confirmation and subsequent input of countermeasures. Specifically, countermeasures such as "installing safety devices" are devised for the presented scenarios.

[0478] Step 6:

[0479] The server saves the user's input response and its result to a database. This process records the data as new information, which is then used for later analysis and improvement of the AI ​​model. The input is the response and result entered by the user on the terminal, and the output is the updated database record. Specifically, it executes an SQL query such as "INSERT INTO response_log (date, location, action, result) VALUES ('2023-10-01', 'Site A', 'Safety device installed', 'Success')".

[0480] (Application Example 1)

[0481] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0482] Improving safety awareness within factories and ensuring a safe working environment are extremely important. However, in many factories today, workers have limited means of receiving safety-related information in real time, making it difficult to immediately identify and respond to potential risks. Furthermore, the lack of well-established methods for risk prediction using historical safety data hinders the improvement of workers' safety awareness.

[0483] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0484] In this invention, the server includes means for collecting past work data and safety records and storing them in an information source; means for providing a user interface for inputting location information and environmental conditions from the work environment; means for generating hazard scenarios using a machine learning model based on the data and input information; and means for presenting hazard information to workers in real time via a visual device and prompting them to pay attention. This enables workers to immediately grasp potential risks in the work environment and take appropriate safety measures. Furthermore, by updating the risk assessment algorithm using past safety data, the accuracy of subsequent analyses can be improved, leading to a further enhancement of safety awareness.

[0485] "Work data" refers to information about the work performed within the factory and the activities of the workers, and includes data from past cases.

[0486] A "safety record" is a record of data related to past safety incidents and countermeasures.

[0487] "Information sources" refer to databases or storage systems used to store and manage collected work data and safety records.

[0488] A "user interface" is the operating environment that allows workers to input information or receive presented information.

[0489] A "machine learning model" is a program that includes algorithms for analyzing accumulated data and predicting risky scenarios.

[0490] A "risk scenario" is an indicator that shows potential risks and related situations, generated based on past and on-site information.

[0491] "Visual devices" are devices used to visually display information, and include head-mounted displays and smart glasses worn and used by workers on site.

[0492] The system that realizes this invention functions primarily through the interaction between a server, a terminal, and a user. The server first collects past work data and safety records as information sources and stores them in a database. The database can be a relational database such as PostgreSQL.

[0493] Users input location information and environmental conditions from their work environment via terminals within the factory. The terminals are equipped with an interface that allows for real-time input and verification of work status and environmental data. The terminals also transmit information to a server via an internet connection.

[0494] The server generates risk scenarios using machine learning models based on collected data and real-time input information. Machine learning frameworks such as TensorFlow and PyTorch can be used for this process. The generative AI model assesses and predicts potential risks in the work environment based on historical data and current conditions.

[0495] Users can wear visual devices to view these hazardous scenarios in real time. These devices often include smart glasses or head-mounted displays. These devices provide hazard information and countermeasures, enhancing the user's safety awareness. This allows users to take appropriate action immediately.

[0496] A concrete example would be a scenario where the system detects the risk of a worker accidentally entering a forklift aisle, displays an alarm via smart glasses, and prompts appropriate avoidance action. An example of a prompt message would be, "Predict entry into the forklift aisle and display a warning alert to the worker."

[0497] Thus, the present invention provides concrete means for improving safety within a factory, thereby enhancing workers' safety awareness and ensuring a safe working environment.

[0498] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0499] Step 1:

[0500] The server collects historical work data and safety records. This information is entered from external databases and internal record systems. After collecting the data, data cleaning is performed and the format is standardized before storing it in the database. This creates a consistent information source and facilitates subsequent data analysis.

[0501] Step 2:

[0502] The user inputs location information and environmental conditions from the work environment via a terminal. Location information is obtained from a GPS module, and environmental conditions are obtained from sensors. The terminal aggregates this real-time data and converts it into a digital format. The collected information is transmitted to a server via the internet.

[0503] Step 3:

[0504] The server generates risk scenarios using a generative AI model based on historical data stored in the database and real-time data received from the terminal. The machine learning model performs risk analysis on the input data and predicts potential hazards. The generated risk scenarios are then prepared for presentation to the user.

[0505] Step 4:

[0506] The server transmits generated hazard scenarios to terminals connected to the visual device. In response, the visual device displays hazard information in real time, prompting the user to take immediate action. The user then takes safety measures based on the presented information. Specifically, smart glasses display alerts and guidance within the user's field of vision.

[0507] Step 5:

[0508] Users review hazardous scenarios presented through visual devices and take safety measures as needed. Users can input their safety measures into a terminal, which records their actions. This input is sent to a server and used to predict future risks.

[0509] Step 6:

[0510] The server stores the safety measures information received from users and their results in a database. This information is used to update algorithms and optimize risk assessments in order to improve the accuracy of the generated AI model. This information will be important data for improving the accuracy of future analyses.

[0511] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0512] This invention is a system that incorporates an emotion engine to recognize user emotions in order to facilitate hazard prediction activities at construction sites. Specific embodiments of this system are described below.

[0513] First, the server collects past project data and safety records from the database. This information contributes to improving the user's risk perception through the emotion engine and enables the generation of more sophisticated risk scenarios.

[0514] Next, the terminal provides an interface for field workers to input information. This interface incorporates input fields for the field situation, as well as a module for recognizing the user's emotions in real time. The emotion engine analyzes the user's facial expressions, tone of voice, and other factors to evaluate the user's psychological state.

[0515] Users input detailed on-site information via their devices and also review their own emotional information, which is recognized by the emotion engine. This information becomes a factor influencing on-site decision-making.

[0516] The server uses an AI model to generate sophisticated hazard scenarios based on on-site information and emotional data entered by users. Emotional data is a crucial element in scenario generation, considering the impact of specific emotional states on risk perception. The server also adjusts the severity of the risks according to the emotional data, providing optimal feedback to individual workers.

[0517] The device presents this generated scenario to the user and suggests specific safety measures based on their emotions. For example, if the emotion engine assesses the user's stress level, adjustments are made, such as suggesting additional warnings or taking a break from work.

[0518] Subsequently, the user formulates individual safety measures based on the presented scenario and proposed countermeasures, and records them on their device. These countermeasures and their results are stored in a database by the server and used to improve the AI ​​model and enhance the accuracy of the emotion engine for future use.

[0519] For example, if the emotion engine detects a worker is under stress while performing work at height, the server will suggest a hazard scenario such as "increasing the frequency of warnings when the wind speed exceeds safety limits." Psychological support information based on the emotion data will also be provided simultaneously.

[0520] This system allows workers to make safety decisions that take into account not only the specific risks at the site but also their emotional state. This helps prevent accidents at construction sites and improves safety awareness.

[0521] The following describes the processing flow.

[0522] Step 1:

[0523] The server collects past project data and safety records from the database. This allows it to extract information on past accident history and risk factors, preparing it for future risk assessments.

[0524] Step 2:

[0525] The terminal displays an interface for field workers to operate. This interface includes an emotion recognition module for recognizing the user's emotions, along with inputting field information.

[0526] Step 3:

[0527] The user inputs specific information collected on-site (location, weather conditions, work details, etc.) into the terminal. Additionally, an emotion recognition module automatically analyzes the user's emotional state through facial expressions and voice.

[0528] Step 4:

[0529] The server receives input field data and sentiment data, and uses AI to generate risk scenarios based on this data. Sentiment data is used to weight the scenarios and influence the perception of risk.

[0530] Step 5:

[0531] The device presents the user with risk scenarios generated from the server and specific safety measures based on emotional data. During this process, feedback and warnings are provided according to the user's psychological state.

[0532] Step 6:

[0533] Based on the presented scenario, the user considers safety measures and decides on the necessary actions. They record the decided measures on their device to prepare for the next task.

[0534] Step 7:

[0535] The server stores user feedback and implemented countermeasures in a database. This data is used for the continuous learning and improvement of the AI ​​model and emotion engine.

[0536] Step 8:

[0537] The server uses the accumulated data to refine the risk assessment algorithm, improving the accuracy of future analyses, while simultaneously optimizing project planning to take sentiment data into consideration.

[0538] (Example 2)

[0539] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] Improving safety management at construction sites is often difficult due to a lack of safety awareness among workers and inadequate risk perception. In such situations, conventional safety measures struggle to accurately predict potential hazards and provide optimal countermeasures for individual workers. Furthermore, the influence of workers' psychological states on safety decisions is often insufficient, resulting in many cases where accidents are not adequately prevented. This invention aims to solve these problems and achieve effective safety management.

[0541] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0542] In this invention, the server includes means for collecting past work data and safety history and storing it in an information management device, means for providing information input for inputting user information from the work site, and means for creating risk prediction information using an information processing device based on the information and input data. This makes it possible to recognize risks that take into account the individual emotional data of users and to provide appropriate feedback based on that.

[0543] "Past work data" refers to information about work performed in the past, and specifically includes records of the work content, working conditions, and results.

[0544] "Safety history" refers to records of past safety-related events and countermeasures, including accidents and near misses in the workplace, as well as the associated countermeasures and evaluations.

[0545] An "information management device" refers to a system or hardware that stores data and allows access to that data as needed, and is primarily a device with database functionality.

[0546] "Information input volume" refers to the interfaces necessary for users to input data, including text input fields, voice input devices, cameras, etc.

[0547] An "information processing device" refers to a computer system or software that analyzes and processes collected data to derive necessary information, and is the environment in which artificial intelligence algorithms are executed.

[0548] "Risk prediction information" refers to predictive information created based on collected data, which includes potential risk factors and appropriate countermeasures for those risks.

[0549] "Feedback" refers to advice, warnings, or information provided by the system to the user, including guidance to help the user take appropriate action.

[0550] This invention is a system for improving safety at construction sites, utilizing an AI model generated based on past work data and safety history. This allows users to consider their own psychological state and work environment and take appropriate risk mitigation measures.

[0551] First, the server collects past work data and safety history from the information management device and stores this information in a database. This may involve using a cloud-based database system or a local server. This accumulated information forms the basis for creating hazard prediction information using a generative AI model.

[0552] Next, the terminal provides information input to users at the work site. This interface allows workers to input the situation at the site and operates on devices such as tablets and smartphones. Input is done via text or voice, and the terminal is equipped with a camera and microphone, and implements emotion recognition functionality that analyzes the user's psychological state through their facial expressions and tone of voice.

[0553] This recognized emotion data is sent to an information processing unit on the server. The information processing unit uses a generative AI model to create risk prediction information. The AI ​​model predicts typical risk factors under specific circumstances and analyzes how those risks affect the user's psychological state.

[0554] For example, if the emotion recognition function detects a user experiencing stress while working at height, the server will generate a danger scenario such as "strengthen countermeasures in case the wind speed exceeds safety standards." Psychological support information based on emotion data will also be provided.

[0555] In this way, users can formulate individual safety measures based on the presented risk prediction information and feedback, and record them on their devices. This improves the overall accuracy of the system and the safety awareness of the users.

[0556] As an example of a prompt, the AI ​​model can be input with a message like, "Please provide the psychological state of a worker when strong winds blow while they are working at height, and the associated risk scenarios," to obtain appropriate risk prediction information.

[0557] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0558] Step 1:

[0559] The server collects historical work data and safety history from the database. It takes work details and safety-related information from the existing database as input and processes it into a dataset. The output is a data file containing aggregated information necessary for risk analysis. This data is then used for subsequent analysis by an AI model.

[0560] Step 2:

[0561] The terminal provides field workers with a means of inputting information. Users input information about the surrounding environment and work details. Inputs include real-time data from the site and a brief health status report of the worker. Output is a data stream in which the input information is sent to the server in an appropriate format. The terminal also utilizes information from the camera and microphone as needed.

[0562] Step 3:

[0563] The emotion recognition function built into the device evaluates the user's psychological state. This includes acquiring facial expression data from the camera and analyzing voice tone from the microphone. The input is real-time visual and auditory data from the user. The output is evaluation data indicating the user's emotional state, which is sent to the server.

[0564] Step 4:

[0565] The server generates hazard prediction information using an AI model based on acquired field data and sentiment data. Input information includes user information and sentiment recognition data transmitted from the terminal, as well as past work data. Statistical methods and machine learning algorithms are used for data processing to predict specific risk factors. The output consists of the generated hazard prediction scenario and its supporting data, which is then returned to the terminal.

[0566] Step 5:

[0567] The terminal presents the user with generated risk prediction information and safety measures. The user is required to review the proposed countermeasures and consider additional safety measures based on their own judgment. The input is the prediction information provided by the server. The output is the countermeasure information that the user reviews and, if necessary, enters into the terminal. The terminal then sends this information back to the server.

[0568] Step 6:

[0569] The server stores newly entered countermeasures in a database and uses them to improve the AI ​​model for future analyses. This enables the provision of even more accurate risk prediction scenarios in subsequent analyses. The input consists of user feedback and countermeasure data, and the output is the result of analyzing and appropriately storing this data.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] In modern work environments, it is essential to properly manage the interaction between workers' emotions and safety measures. However, conventional safety management systems have struggled to recognize workers' psychological states in real time and provide immediate feedback accordingly. As a result, there is a challenge in implementing effective safety measures that take into account fluctuations in risk due to workers' emotional states.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes means for collecting past work data and safety records and storing them in an information storage means, means for providing an information input device for inputting relevant information from the work site, and means for recognizing the emotional state of technicians in real time and generating psychological feedback. This enables the immediate presentation of hazard scenarios based on the worker's psychological state and personalized safety feedback based on emotions.

[0575] "Past operational data" refers to the history of work performed at the work site, details of the work, and related information, and is an information resource that forms the basis for system improvement and risk assessment.

[0576] "Safety records" refer to documented information about past safety-related incidents and the measures taken to address them, and serve as important reference information for future safety management.

[0577] "Information storage means" refers to digital or physical storage devices used to collect and store various data and information, such as databases.

[0578] An "information input device" is a device equipped with a user interface for workers to input on-site conditions and specific data into a system, and this includes computer terminals and tablets.

[0579] Machine learning is a technology that improves the accuracy of specific tasks by using large amounts of data to enable computers to self-improve.

[0580] A "risk scenario" refers to a plan or set of countermeasures that include predictive information about dangerous events that may occur under specific conditions, and plays an important role in ensuring safety during work.

[0581] "The emotional state of engineers" refers to the psychological and emotional state that workers experience while performing their tasks, and appropriately understanding this state contributes to improving workplace safety.

[0582] "Psychological feedback" refers to providing appropriate advice and information based on the worker's psychological state, with the aim of avoiding risks and improving safety.

[0583] This invention is a system that improves workplace safety by providing immediate and appropriate feedback according to the worker's emotional state and work environment. The server collects past work data and safety records and stores them in an information storage means. This prepares the basic information necessary for data analysis. The server receives relevant information from workers on site through an information input device. The information input device includes smart glasses and mobile terminals, which workers can use to input their current work situation and psychological state.

[0584] The server uses Google Cloud's Vision API and Speech-to-Text API to recognize the emotional state of workers in real time. This allows for the evaluation of their psychological state through facial expression and voice analysis. Furthermore, machine learning technology is used to generate risk scenarios based on this data. These generated risk scenarios are presented to workers as psychological feedback, further enhancing workplace safety.

[0585] For example, if a worker experiences a problem while performing maintenance on complex machinery and feels anxious, the system recognizes this state through its emotion engine and displays feedback on the worker's display such as, "Take a deep breath and take a break." Similarly, when a worker experiences tension when performing a new work procedure for the first time, advice such as, "Work at your own pace," is displayed to reduce stress.

[0586] An example of a prompt message for a generative AI model is: "Use the worker's facial image and voice data to evaluate their psychological state (stress, anxiety, tension, etc.) and create and provide appropriate feedback."

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The server collects historical operational data and safety records from information storage systems. This provides the basic information necessary for data analysis. This data is used for risk assessment and training machine learning models.

[0590] Step 2:

[0591] The terminal receives information about the worker's work status through an input interface. This includes data related to the work content, location, and current psychological state. The terminal sends this information to a server, which is then prepared for further analysis.

[0592] Step 3:

[0593] The server uses received work information and machine learning techniques to recognize the worker's emotional state in real time. It performs facial expression analysis using Google Cloud's Vision API and speech analysis using the Speech-to-Text API. This identifies the worker's psychological state, such as stress levels and anxiety.

[0594] Step 4:

[0595] The server combines emotional data with historical operational data and uses a generative AI model to generate risk scenarios. Based on the results of the emotional analysis, the optimal risk scenario is created, and suggestions such as warnings during work or the need for rest are made.

[0596] Step 5:

[0597] The terminal displays risk scenarios and feedback sent from the server to the worker. The worker reviews this information directly on the display and applies it as needed. The feedback includes specific safety actions and psychological support information.

[0598] Step 6:

[0599] Users implement safety measures based on the feedback provided and record the results from their device to the server. This verifies the effectiveness of the countermeasures and contributes to improving future risk assessments and the accuracy of feedback.

[0600] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0601] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0602] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0603] [Fourth Embodiment]

[0604] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0605] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0606] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0607] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0608] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0609] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0610] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0611] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0612] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0613] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0614] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0615] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0616] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] This invention is a system that supports hazard prediction activities to improve safety awareness at construction sites, and analyzes risks using artificial intelligence based on past project data and safety records. A specific embodiment of this system is described below.

[0618] First, the server collects past project data and safety records from the database. This allows for a comprehensive understanding of past cases and key safety points at construction sites. The server then formats the collected information for analysis, making it usable by AI models.

[0619] Next, the terminal provides an interface that allows field workers to input information. The user (worker) uses this interface to input specific information such as the site's location, weather conditions, and work status. This field information then becomes an important data source for the AI.

[0620] The server uses artificial intelligence to generate hazard scenarios based on this information. The generated hazard scenarios take into account different risk factors and serve to attract the user's attention and raise safety awareness by visualizing potential dangers.

[0621] Next, the device presents the user with this risk scenario. Based on the presented scenario, the user considers appropriate countermeasures and records a specific action plan on the device. This enables the user to independently consider and implement risk mitigation measures.

[0622] Subsequently, the server stores the entered countermeasures and their implementation results in a database. This data will be used to improve future AI models and develop new risk assessment algorithms, enabling more accurate risk prediction.

[0623] For example, if work is being done at a high altitude at a certain site, the server uses data from past fall accidents during high-altitude work to generate a dangerous scenario where "the scaffolding becomes unstable in strong winds." The user reviews this scenario, devises countermeasures such as installing safety devices according to the wind speed or temporarily suspending work, and inputs them into the terminal. The server records this information and uses it for future risk assessments.

[0624] In this way, this system provides effective measures to prevent accidents at construction sites while raising safety awareness.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] The server collects historical project data and safety records from the database. It then formats this data and converts it into a format that the AI ​​model can use for analysis.

[0628] Step 2:

[0629] The terminal provides the user with an interface for inputting specific information at the work site. This interface allows input of location information, weather conditions for the day, and the health status of the workers.

[0630] Step 3:

[0631] Users input information about the field situation through the terminal interface. They meticulously record any unusual circumstances or important information at the site.

[0632] Step 4:

[0633] The server receives information entered by the user, integrates it with historical data, and analyzes it. Using an AI model, it generates hazard scenarios based on the actual situation on site.

[0634] Step 5:

[0635] The device presents the generated risk scenario to the user. The user reviews the risk information and its background described in the displayed scenario.

[0636] Step 6:

[0637] Based on the presented hazard scenarios, the user considers safety measures and decides on specific countermeasures. The decided countermeasures are then entered into the terminal and recorded.

[0638] Step 7:

[0639] The server stores the countermeasures entered by users and the results of their implementation in a database. This accumulated data will be used to improve future AI models and enhance the accuracy of risk assessments.

[0640] Step 8:

[0641] The server regularly updates its risk assessment algorithm to achieve more accurate hazard prediction. It also encourages user participation through a points system and awards program, aiming to improve safety awareness.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] Ensuring safety at construction sites is a crucial issue, but traditional hazard prediction activities often rely on experience and intuition, making it easy to overlook risk factors. Furthermore, the lack of concrete, systematic support to effectively improve workers' safety awareness has resulted in limited improvements in site safety. Therefore, it is necessary to effectively utilize past data and automatically predict potential risks to achieve effective risk management and improve workers' safety awareness.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for collecting past similar activity data and safety records and storing them in an information storage device; means for providing an interface that enables the input of situational information from a specific site; and means for generating hazard prediction scenarios using a generative AI model based on the information and the inputted situation. This enables immediate and effective risk management and improved safety awareness by automatically predicting potential risks at the work site and presenting them visually to workers.

[0647] "Similar activity data" refers to records of similar projects or activities carried out in the past, and includes information such as the work performed, the duration, the equipment used, and any problems that occurred.

[0648] "Safety records" refer to records of safety-related incidents in past projects, including details of accidents, causes, countermeasures, and lessons learned.

[0649] An "information storage device" refers to a device that can store data for a long period of time and retrieve it as needed, and this includes hard disks, flash memory, and other similar devices.

[0650] An "interface" refers to a mechanism that provides a means for a user to operate or input information into a system, and this includes graphical user interfaces.

[0651] A "generative AI model" refers to a model that uses machine learning techniques to learn patterns from specific data and then makes predictions and decisions based on new data.

[0652] A "risk prediction scenario" refers to a presentation format that summarizes potential future risks, generated based on past data and current circumstances.

[0653] "Worker" refers to an individual responsible for a specific task or project at a construction site, and is responsible for implementing safety measures and providing input information.

[0654] This invention is a risk prediction support system aimed at improving safety at construction sites. Based on past similar activity data and safety records, this system utilizes a generated AI model to predict potential risks and provides workers with specific hazard prediction scenarios, thereby raising safety awareness and preventing accidents.

[0655] The server first collects historical data on similar activities and safety records from the database. This allows for a comprehensive understanding of problems in past operations and the effectiveness of safety measures. Database management and data processing tools such as SQL queries and Pandas are used for data collection and formatting.

[0656] Next, the terminal provides a user interface for field workers to input situation information. This interface provides workers with a means to input specific information such as location information, weather conditions, and work status at the work site using a tablet or smartphone.

[0657] Users can input data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" using their terminal. This information is collected by the server and used as input for the generated AI model.

[0658] The generative AI model uses machine learning libraries such as TensorFlow and PyTorch to generate hazard prediction scenarios based on data provided by the server. These scenarios include important warnings for field workers, such as predicting the risk of scaffolding becoming unstable as wind speed increases.

[0659] As a concrete example, based on the prompt message, "Use past data on falls during work at heights to generate potential hazard scenarios and suggest risk mitigation measures," the AI ​​model can generate and present specific safety suggestions. This allows workers to immediately take countermeasures against risks.

[0660] This system allows on-site workers to independently devise and implement safety measures, resulting in a reduction in the risk of accidents at construction sites and an improvement in safety awareness.

[0661] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0662] Step 1:

[0663] The server retrieves historical similar activity data and safety records from the database. This process uses SQL queries to extract the necessary data and store it in the information storage device. The input is raw data from the database, and the output is formatted structured data. Specifically, the server executes a query like "SELECT FROM safety_records WHERE date >= '2020-01-01'".

[0664] Step 2:

[0665] The server formats the acquired data into a format that the AI ​​model can process. This process uses the Pandas library to clean the data and transform it into a format suitable for machine learning models. The input is the raw dataset, and the output is an input format optimized for the AI ​​model. Specifically, it performs tasks such as imputing missing values ​​and converting categorical data to numerical values.

[0666] Step 3:

[0667] The terminal provides an interface that allows field workers to input situation information. The user interface allows for the input of specific information such as location, weather, and work status. Input is field information from the worker, and output is a new information packet sent to the server. Specifically, data such as "Current location: Site A, Weather: Sunny, Work status: Installing pipes" is input.

[0668] Step 4:

[0669] The server uses a generative AI model to generate hazard prediction scenarios, taking on-site information and formatted data received from the terminal as input. TensorFlow or PyTorch are used to run the model. The input is on-site information and formatted historical data, and the output is potential hazard scenarios. Specifically, the server makes predictions such as "the scaffolding may become unstable during strong winds."

[0670] Step 5:

[0671] The terminal presents the user with generated hazard scenarios. The user interface visually displays the scenarios, and the user devises countermeasures based on them. The input is scenario data from the server, and the output is the user's confirmation and subsequent input of countermeasures. Specifically, countermeasures such as "installing safety devices" are devised for the presented scenarios.

[0672] Step 6:

[0673] The server saves the user's input response and its result to a database. This process records the data as new information, which is then used for later analysis and improvement of the AI ​​model. The input is the response and result entered by the user on the terminal, and the output is the updated database record. Specifically, it executes an SQL query such as "INSERT INTO response_log (date, location, action, result) VALUES ('2023-10-01', 'Site A', 'Safety device installed', 'Success')".

[0674] (Application Example 1)

[0675] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] Improving safety awareness within factories and ensuring a safe working environment are extremely important. However, in many factories today, workers have limited means of receiving safety-related information in real time, making it difficult to immediately identify and respond to potential risks. Furthermore, the lack of well-established methods for risk prediction using historical safety data hinders the improvement of workers' safety awareness.

[0677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0678] In this invention, the server includes means for collecting past work data and safety records and storing them in an information source; means for providing a user interface for inputting location information and environmental conditions from the work environment; means for generating hazard scenarios using a machine learning model based on the data and input information; and means for presenting hazard information to workers in real time via a visual device and prompting them to pay attention. This enables workers to immediately grasp potential risks in the work environment and take appropriate safety measures. Furthermore, by updating the risk assessment algorithm using past safety data, the accuracy of subsequent analyses can be improved, leading to a further enhancement of safety awareness.

[0679] "Work data" refers to information about the work performed within the factory and the activities of the workers, and includes data from past cases.

[0680] A "safety record" is a record of data related to past safety incidents and countermeasures.

[0681] "Information sources" refer to databases or storage systems used to store and manage collected work data and safety records.

[0682] A "user interface" is the operating environment that allows workers to input information or receive presented information.

[0683] A "machine learning model" is a program that includes algorithms for analyzing accumulated data and predicting risky scenarios.

[0684] A "risk scenario" is an indicator that shows potential risks and related situations, generated based on past and on-site information.

[0685] "Visual devices" are devices used to visually display information, and include head-mounted displays and smart glasses worn and used by workers on site.

[0686] The system that realizes this invention functions primarily through the interaction between a server, a terminal, and a user. The server first collects past work data and safety records as information sources and stores them in a database. The database can be a relational database such as PostgreSQL.

[0687] Users input location information and environmental conditions from their work environment via terminals within the factory. The terminals are equipped with an interface that allows for real-time input and verification of work status and environmental data. The terminals also transmit information to a server via an internet connection.

[0688] The server generates risk scenarios using machine learning models based on collected data and real-time input information. Machine learning frameworks such as TensorFlow and PyTorch can be used for this process. The generative AI model assesses and predicts potential risks in the work environment based on historical data and current conditions.

[0689] Users can wear visual devices to view these hazardous scenarios in real time. These devices often include smart glasses or head-mounted displays. These devices provide hazard information and countermeasures, enhancing the user's safety awareness. This allows users to take appropriate action immediately.

[0690] A concrete example would be a scenario where the system detects the risk of a worker accidentally entering a forklift aisle, displays an alarm via smart glasses, and prompts appropriate avoidance action. An example of a prompt message would be, "Predict entry into the forklift aisle and display a warning alert to the worker."

[0691] Thus, the present invention provides concrete means for improving safety within a factory, thereby enhancing workers' safety awareness and ensuring a safe working environment.

[0692] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0693] Step 1:

[0694] The server collects historical work data and safety records. This information is entered from external databases and internal record systems. After collecting the data, data cleaning is performed and the format is standardized before storing it in the database. This creates a consistent information source and facilitates subsequent data analysis.

[0695] Step 2:

[0696] The user inputs location information and environmental conditions from the work environment via a terminal. Location information is obtained from a GPS module, and environmental conditions are obtained from sensors. The terminal aggregates this real-time data and converts it into a digital format. The collected information is transmitted to a server via the internet.

[0697] Step 3:

[0698] The server generates risk scenarios using a generative AI model based on historical data stored in the database and real-time data received from the terminal. The machine learning model performs risk analysis on the input data and predicts potential hazards. The generated risk scenarios are then prepared for presentation to the user.

[0699] Step 4:

[0700] The server transmits generated hazard scenarios to terminals connected to the visual device. In response, the visual device displays hazard information in real time, prompting the user to take immediate action. The user then takes safety measures based on the presented information. Specifically, smart glasses display alerts and guidance within the user's field of vision.

[0701] Step 5:

[0702] Users review hazardous scenarios presented through visual devices and take safety measures as needed. Users can input their safety measures into a terminal, which records their actions. This input is sent to a server and used to predict future risks.

[0703] Step 6:

[0704] The server stores the safety measures information received from users and their results in a database. This information is used to update algorithms and optimize risk assessments in order to improve the accuracy of the generated AI model. This information will be important data for improving the accuracy of future analyses.

[0705] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0706] This invention is a system that incorporates an emotion engine to recognize user emotions in order to facilitate hazard prediction activities at construction sites. Specific embodiments of this system are described below.

[0707] First, the server collects past project data and safety records from the database. This information contributes to improving the user's risk perception through the emotion engine and enables the generation of more sophisticated risk scenarios.

[0708] Next, the terminal provides an interface for field workers to input information. This interface incorporates input fields for the field situation, as well as a module for recognizing the user's emotions in real time. The emotion engine analyzes the user's facial expressions, tone of voice, and other factors to evaluate the user's psychological state.

[0709] Users input detailed on-site information via their devices and also review their own emotional information, which is recognized by the emotion engine. This information becomes a factor influencing on-site decision-making.

[0710] The server uses an AI model to generate sophisticated hazard scenarios based on on-site information and emotional data entered by users. Emotional data is a crucial element in scenario generation, considering the impact of specific emotional states on risk perception. The server also adjusts the severity of the risks according to the emotional data, providing optimal feedback to individual workers.

[0711] The device presents this generated scenario to the user and suggests specific safety measures based on their emotions. For example, if the emotion engine assesses the user's stress level, adjustments are made, such as suggesting additional warnings or taking a break from work.

[0712] Subsequently, the user formulates individual safety measures based on the presented scenario and proposed countermeasures, and records them on their device. These countermeasures and their results are stored in a database by the server and used to improve the AI ​​model and enhance the accuracy of the emotion engine for future use.

[0713] For example, if the emotion engine detects a worker is under stress while performing work at height, the server will suggest a hazard scenario such as "increasing the frequency of warnings when the wind speed exceeds safety limits." Psychological support information based on the emotion data will also be provided simultaneously.

[0714] This system allows workers to make safety decisions that take into account not only the specific risks at the site but also their emotional state. This helps prevent accidents at construction sites and improves safety awareness.

[0715] The following describes the processing flow.

[0716] Step 1:

[0717] The server collects past project data and safety records from the database. This allows it to extract information on past accident history and risk factors, preparing it for future risk assessments.

[0718] Step 2:

[0719] The terminal displays an interface for field workers to operate. This interface includes an emotion recognition module for recognizing the user's emotions, along with inputting field information.

[0720] Step 3:

[0721] The user inputs specific information collected on-site (location, weather conditions, work details, etc.) into the terminal. Additionally, an emotion recognition module automatically analyzes the user's emotional state through facial expressions and voice.

[0722] Step 4:

[0723] The server receives input field data and sentiment data, and uses AI to generate risk scenarios based on this data. Sentiment data is used to weight the scenarios and influence the perception of risk.

[0724] Step 5:

[0725] The device presents the user with risk scenarios generated from the server and specific safety measures based on emotional data. During this process, feedback and warnings are provided according to the user's psychological state.

[0726] Step 6:

[0727] Based on the presented scenario, the user considers safety measures and decides on the necessary actions. They record the decided measures on their device to prepare for the next task.

[0728] Step 7:

[0729] The server stores user feedback and implemented countermeasures in a database. This data is used for the continuous learning and improvement of the AI ​​model and emotion engine.

[0730] Step 8:

[0731] The server uses the accumulated data to refine the risk assessment algorithm, improving the accuracy of future analyses, while simultaneously optimizing project planning to take sentiment data into consideration.

[0732] (Example 2)

[0733] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0734] Improving safety management at construction sites is often difficult due to a lack of safety awareness among workers and inadequate risk perception. In such situations, conventional safety measures struggle to accurately predict potential hazards and provide optimal countermeasures for individual workers. Furthermore, the influence of workers' psychological states on safety decisions is often insufficient, resulting in many cases where accidents are not adequately prevented. This invention aims to solve these problems and achieve effective safety management.

[0735] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0736] In this invention, the server includes means for collecting past work data and safety history and storing it in an information management device, means for providing information input for inputting user information from the work site, and means for creating risk prediction information using an information processing device based on the information and input data. This makes it possible to recognize risks that take into account the individual emotional data of users and to provide appropriate feedback based on that.

[0737] "Past work data" refers to information about work performed in the past, and specifically includes records of the work content, working conditions, and results.

[0738] "Safety history" refers to records of past safety-related events and countermeasures, including accidents and near misses in the workplace, as well as the associated countermeasures and evaluations.

[0739] An "information management device" refers to a system or hardware that stores data and allows access to that data as needed, and is primarily a device with database functionality.

[0740] "Information input volume" refers to the interfaces necessary for users to input data, including text input fields, voice input devices, cameras, etc.

[0741] An "information processing device" refers to a computer system or software that analyzes and processes collected data to derive necessary information, and is the environment in which artificial intelligence algorithms are executed.

[0742] "Risk prediction information" refers to predictive information created based on collected data, which includes potential risk factors and appropriate countermeasures for those risks.

[0743] "Feedback" refers to advice, warnings, or information provided by the system to the user, including guidance to help the user take appropriate action.

[0744] This invention is a system for improving safety at construction sites, utilizing an AI model generated based on past work data and safety history. This allows users to consider their own psychological state and work environment and take appropriate risk mitigation measures.

[0745] First, the server collects past work data and safety history from the information management device and stores this information in a database. This may involve using a cloud-based database system or a local server. This accumulated information forms the basis for creating hazard prediction information using a generative AI model.

[0746] Next, the terminal provides information input to users at the work site. This interface allows workers to input the situation at the site and operates on devices such as tablets and smartphones. Input is done via text or voice, and the terminal is equipped with a camera and microphone, and implements emotion recognition functionality that analyzes the user's psychological state through their facial expressions and tone of voice.

[0747] This recognized emotion data is sent to an information processing unit on the server. The information processing unit uses a generative AI model to create risk prediction information. The AI ​​model predicts typical risk factors under specific circumstances and analyzes how those risks affect the user's psychological state.

[0748] For example, if the emotion recognition function detects a user experiencing stress while working at height, the server will generate a danger scenario such as "strengthen countermeasures in case the wind speed exceeds safety standards." Psychological support information based on emotion data will also be provided.

[0749] In this way, users can formulate individual safety measures based on the presented risk prediction information and feedback, and record them on their devices. This improves the overall accuracy of the system and the safety awareness of the users.

[0750] As an example of a prompt, the AI ​​model can be input with a message like, "Please provide the psychological state of a worker when strong winds blow while they are working at height, and the associated risk scenarios," to obtain appropriate risk prediction information.

[0751] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0752] Step 1:

[0753] The server collects historical work data and safety history from the database. It takes work details and safety-related information from the existing database as input and processes it into a dataset. The output is a data file containing aggregated information necessary for risk analysis. This data is then used for subsequent analysis by an AI model.

[0754] Step 2:

[0755] The terminal provides field workers with a means of inputting information. Users input information about the surrounding environment and work details. Inputs include real-time data from the site and a brief health status report of the worker. Output is a data stream in which the input information is sent to the server in an appropriate format. The terminal also utilizes information from the camera and microphone as needed.

[0756] Step 3:

[0757] The emotion recognition function built into the device evaluates the user's psychological state. This includes acquiring facial expression data from the camera and analyzing voice tone from the microphone. The input is real-time visual and auditory data from the user. The output is evaluation data indicating the user's emotional state, which is sent to the server.

[0758] Step 4:

[0759] The server generates hazard prediction information using an AI model based on acquired field data and sentiment data. Input information includes user information and sentiment recognition data transmitted from the terminal, as well as past work data. Statistical methods and machine learning algorithms are used for data processing to predict specific risk factors. The output consists of the generated hazard prediction scenario and its supporting data, which is then returned to the terminal.

[0760] Step 5:

[0761] The terminal presents the user with generated risk prediction information and safety measures. The user is required to review the proposed countermeasures and consider additional safety measures based on their own judgment. The input is the prediction information provided by the server. The output is the countermeasure information that the user reviews and, if necessary, enters into the terminal. The terminal then sends this information back to the server.

[0762] Step 6:

[0763] The server stores newly entered countermeasures in a database and uses them to improve the AI ​​model for future analyses. This enables the provision of even more accurate risk prediction scenarios in subsequent analyses. The input consists of user feedback and countermeasure data, and the output is the result of analyzing and appropriately storing this data.

[0764] (Application Example 2)

[0765] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0766] In modern work environments, it is essential to properly manage the interaction between workers' emotions and safety measures. However, conventional safety management systems have struggled to recognize workers' psychological states in real time and provide immediate feedback accordingly. As a result, there is a challenge in implementing effective safety measures that take into account fluctuations in risk due to workers' emotional states.

[0767] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0768] In this invention, the server includes means for collecting past work data and safety records and storing them in an information storage means, means for providing an information input device for inputting relevant information from the work site, and means for recognizing the emotional state of technicians in real time and generating psychological feedback. This enables the immediate presentation of hazard scenarios based on the worker's psychological state and personalized safety feedback based on emotions.

[0769] "Past operational data" refers to the history of work performed at the work site, details of the work, and related information, and is an information resource that forms the basis for system improvement and risk assessment.

[0770] "Safety records" refer to documented information about past safety-related incidents and the measures taken to address them, and serve as important reference information for future safety management.

[0771] "Information storage means" refers to digital or physical storage devices used to collect and store various data and information, such as databases.

[0772] An "information input device" is a device equipped with a user interface for workers to input on-site conditions and specific data into a system, and this includes computer terminals and tablets.

[0773] Machine learning is a technology that improves the accuracy of specific tasks by using large amounts of data to enable computers to self-improve.

[0774] A "risk scenario" refers to a plan or set of countermeasures that include predictive information about dangerous events that may occur under specific conditions, and plays an important role in ensuring safety during work.

[0775] "The emotional state of engineers" refers to the psychological and emotional state that workers experience while performing their tasks, and appropriately understanding this state contributes to improving workplace safety.

[0776] "Psychological feedback" refers to providing appropriate advice and information based on the worker's psychological state, with the aim of avoiding risks and improving safety.

[0777] This invention is a system that improves workplace safety by providing immediate and appropriate feedback according to the worker's emotional state and work environment. The server collects past work data and safety records and stores them in an information storage means. This prepares the basic information necessary for data analysis. The server receives relevant information from workers on site through an information input device. The information input device includes smart glasses and mobile terminals, which workers can use to input their current work situation and psychological state.

[0778] The server uses Google Cloud's Vision API and Speech-to-Text API to recognize the emotional state of workers in real time. This allows for the evaluation of their psychological state through facial expression and voice analysis. Furthermore, machine learning technology is used to generate risk scenarios based on this data. These generated risk scenarios are presented to workers as psychological feedback, further enhancing workplace safety.

[0779] For example, if a worker experiences a problem while performing maintenance on complex machinery and feels anxious, the system recognizes this state through its emotion engine and displays feedback on the worker's display such as, "Take a deep breath and take a break." Similarly, when a worker experiences tension when performing a new work procedure for the first time, advice such as, "Work at your own pace," is displayed to reduce stress.

[0780] An example of a prompt message for a generative AI model is: "Use the worker's facial image and voice data to evaluate their psychological state (stress, anxiety, tension, etc.) and create and provide appropriate feedback."

[0781] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0782] Step 1:

[0783] The server collects historical operational data and safety records from information storage systems. This provides the basic information necessary for data analysis. This data is used for risk assessment and training machine learning models.

[0784] Step 2:

[0785] The terminal receives information about the worker's work status through an input interface. This includes data related to the work content, location, and current psychological state. The terminal sends this information to a server, which is then prepared for further analysis.

[0786] Step 3:

[0787] The server uses received work information and machine learning techniques to recognize the worker's emotional state in real time. It performs facial expression analysis using Google Cloud's Vision API and speech analysis using the Speech-to-Text API. This identifies the worker's psychological state, such as stress levels and anxiety.

[0788] Step 4:

[0789] The server combines emotional data with historical operational data and uses a generative AI model to generate risk scenarios. Based on the results of the emotional analysis, the optimal risk scenario is created, and suggestions such as warnings during work or the need for rest are made.

[0790] Step 5:

[0791] The terminal displays risk scenarios and feedback sent from the server to the worker. The worker reviews this information directly on the display and applies it as needed. The feedback includes specific safety actions and psychological support information.

[0792] Step 6:

[0793] Users implement safety measures based on the feedback provided and record the results from their device to the server. This verifies the effectiveness of the countermeasures and contributes to improving future risk assessments and the accuracy of feedback.

[0794] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0795] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0796] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0797] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0798] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0799] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0800] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0801] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0802] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0803] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0804] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0805] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0806] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0807] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0808] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0809] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0810] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0811] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0812] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0813] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0814] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0815] The following is further disclosed regarding the embodiments described above.

[0816] (Claim 1)

[0817] A means of collecting past project data and safety records and storing them in a database,

[0818] A means of providing an interface for inputting specific information from the field,

[0819] A means for generating a risk scenario using artificial intelligence based on the aforementioned data and input information,

[0820] A means for presenting the generated risk scenario to the user and receiving input for countermeasures,

[0821] The aforementioned countermeasures and their results are stored in a database and analyzed by means of a database.

[0822] A system that includes this.

[0823] (Claim 2)

[0824] The system according to claim 1, which updates the risk assessment algorithm based on the information stored in the database and improves the accuracy of subsequent analyses.

[0825] (Claim 3)

[0826] The system according to claim 1, which incorporates a points system and an awards system to encourage user participation and improve safety awareness.

[0827] "Example 1"

[0828] (Claim 1)

[0829] A means for collecting past similar activity data and safety records and storing them in an information storage device,

[0830] A means of providing an interface that enables the input of situation information from a specific site,

[0831] A means for generating a risk prediction scenario using a generative AI model based on the aforementioned information and input conditions,

[0832] A means for presenting the generated hazard prediction scenario to the worker and receiving input for countermeasures,

[0833] The aforementioned countermeasures and their results are stored in an information storage device and analyzed by means of a device.

[0834] A system that includes this.

[0835] (Claim 2)

[0836] The system according to claim 1, which updates the calculation method for risk assessment based on the records stored in the information storage device, thereby improving the accuracy of subsequent diagnoses.

[0837] (Claim 3)

[0838] The system according to claim 1, which incorporates a reward system and an awareness system to promote active participation by workers and improve safety awareness.

[0839] "Application Example 1"

[0840] (Claim 1)

[0841] Means for collecting past work data and safety records and storing them in the information source,

[0842] A means of providing a user interface for inputting location information and environmental conditions from the work environment,

[0843] A means for generating risk scenarios using a machine learning model based on the aforementioned data and input information,

[0844] A means for presenting the generated hazard scenario to the worker and supporting the input and recording of countermeasures,

[0845] The aforementioned countermeasures and their results are stored in a data source, and a means for analyzing them based on a new algorithm,

[0846] A means of presenting hazard information to workers in real time via visual devices and prompting them to pay attention,

[0847] A system that includes this.

[0848] (Claim 2)

[0849] The system according to claim 1, which updates the risk assessment algorithm based on the information stored in the aforementioned information source and improves the accuracy of subsequent analyses.

[0850] (Claim 3)

[0851] The system according to claim 1, which incorporates a points system and an awards system to encourage worker participation and improve workplace safety awareness.

[0852] "Example 2 of combining an emotion engine"

[0853] (Claim 1)

[0854] A means for collecting past work data and safety history and storing it in an information management device,

[0855] A means of providing the amount of information input required to input user information from the work site,

[0856] A means for creating risk prediction information using an information processing device based on the aforementioned information and input data,

[0857] A means for presenting the created risk prediction information to the user and accepting input of countermeasures,

[0858] The aforementioned measures and their results are stored in an information management device and analyzed by means of the device.

[0859] A means of recognizing user psychological data and providing optimal feedback based on risk perception,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, which updates the risk assessment analysis based on the information stored in the information management device and improves the accuracy of subsequent analyses.

[0863] (Claim 3)

[0864] The system according to claim 1, which incorporates a points system and an incentive system to encourage user participation and improve safety awareness.

[0865] "Application example 2 when combining with an emotional engine"

[0866] (Claim 1)

[0867] A means for collecting past business data and safety records and storing them in an information storage means,

[0868] A means for providing an information input device for inputting relevant information from the work site,

[0869] A means for generating risk scenarios using machine learning based on the aforementioned data and input information,

[0870] A means for presenting the generated risk scenario to the engineer and receiving input for countermeasures,

[0871] A means of recognizing the emotional state of engineers in real time and generating psychological feedback,

[0872] The aforementioned countermeasures and their results are stored in an information storage means and analyzed by means of a

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, which updates the risk assessment algorithm based on the information stored in the information storage means and improves the accuracy of future analyses.

[0876] (Claim 3)

[0877] The system according to claim 1, which incorporates incentive programs and reward methods to promote user involvement and improve safety awareness. [Explanation of Symbols]

[0878] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting past project data and safety records and storing them in a database, A means of providing an interface for inputting specific information from the field, A means for generating a risk scenario using artificial intelligence based on the aforementioned data and input information, A means for presenting the generated risk scenario to the user and receiving input for countermeasures, The aforementioned countermeasures and their results are stored in a database and analyzed by means of a database. A system that includes this.

2. The system according to claim 1, which updates the risk assessment algorithm based on the information stored in the database and improves the accuracy of subsequent analyses.

3. The system according to claim 1, which incorporates a points system and an awards system to encourage user participation and improve safety awareness.

Images