System

Abstract

A system is provided.SOLUTION: Means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating a more detailed "why" question based on the collected answers, means for presenting the more detailed "why" question, and means for analyzing all the questions and answers to identify the root cause of the accident; Means for presenting the identified root cause in a report format.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When an accident occurs during facility construction, identifying the cause is extremely important, but there is a high possibility of human arbitrariness and bias. Furthermore, traditional "5-why analysis" methods require specialized knowledge and experience, and effective cause identification can be difficult. For this reason, there is a need for a method to objectively and efficiently identify the true cause of an accident. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including: means for providing an interface for inputting accident information; means for receiving the accident information and storing it in a database; means for generating first "why" questions based on the accident information; means for presenting the generated first "why" questions; means for collecting answers to the first "why" questions; means for generating more detailed "why" questions based on the collected answers; means for presenting the more detailed "why" questions; means for identifying the true cause of the accident by analyzing all the questions and answers; and means for presenting the identified true cause in the form of a report. This system makes it possible to eliminate human bias and identify the true cause of the accident objectively and efficiently.

[0006] "Accident information" is data that records details of an accident, such as the date, time, location, and summary of the accident.

[0007] An "interface" is an element that provides a screen or form for a user to input into a system.

[0008] "Generation means" refers to a piece of equipment or software that has the function of generating and analyzing questions using generative AI.

[0009] "Database" refers to a data storage system for storing received accident information and questions and answers.

[0010] A "question list" is a series of questions compiled to present the generated "why" questions to the user.

[0011] A "collection tool" is a piece of equipment or software that captures user responses into the system.

[0012] "Analysis" is the analytical process of identifying the true cause of the accident based on the collected questions and answers.

[0013] The "report format" is a document format that is organized to convey the final analysis results to the user in an easy-to-understand manner.

[0014] "Generative AI" is an artificial intelligence technology that automatically generates "why" questions based on accident information, and then generates further questions based on the answers.

[0015] "Facility construction" refers to all construction activities aimed at the construction, renovation, and maintenance of factories and facilities. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. Specific embodiments and the processing flow of this system are described below.

[0038] User input of accident information

[0039] First, the user enters accident information through a dedicated interface. The entered accident information includes the date and time of the accident, location, a summary of the accident, and information on the people involved. This information is used as the base data for subsequent analysis.

[0040] Accident information registration and initial analysis

[0041] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[0042] Generating initial "why" questions

[0043] The server uses generative AI to generate the first "why" question based on the stored accident information. For example, it automatically generates a specific question such as "Why did the equipment fall?"

[0044] Asking questions and collecting answers

[0045] The server sends the generated questions to the terminal, which displays them to the user. The user then inputs an answer to the question. An example of an answer would be something like, "There was a problem with the way the equipment was handled."

[0046] Generate further questions based on answers

[0047] The device sends the user-entered answers to the server, which again uses generative AI to generate further "why" questions based on the previous answers. For example, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[0048] Identifying the root cause and generating a final report

[0049] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, which includes safety measures and improvement methods to be applied. Finally, the server sends this report to the terminal, which displays it to the user.

[0050] Specific examples

[0051] Case 1:

[0052] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0053] The server uses a generative AI to generate a question such as "Why did the equipment fall?"

[0054] The user responded that "there was a problem with the way it was handled."

[0055] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0056] The user responded that "the cause is a lack of training on work procedures."

[0057] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." It then presents the final report to the user.

[0058] In this way, the system can automate the process of efficiently and objectively identifying the root causes of accidents, eliminating human bias.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user opens a dedicated interface and enters accident information, including the date, time, location, summary, and information about the people involved.

[0062] Step 2:

[0063] The terminal receives the user's input and sends it to the server.

[0064] Step 3:

[0065] The server stores the received accident information in a database.

[0066] Step 4:

[0067] The server launches a generation AI to generate the first "why" question based on the stored accident information, such as "Why did the equipment fall?"

[0068] Step 5:

[0069] The server generates a list of first "why" questions and sends them to the terminal.

[0070] Step 6:

[0071] The terminal displays a list of questions to the user.

[0072] Step 7:

[0073] The user inputs an answer to the displayed question, for example, "There was a problem with the way it was handled."

[0074] Step 8:

[0075] The terminal sends the user's answer to the server.

[0076] Step 9:

[0077] The server stores the received response in a database.

[0078] Step 10:

[0079] The server then launches the generation AI again, generating further "why" questions based on the previous answers, such as "Why was there a problem with the handling method?"

[0080] Step 11:

[0081] The server generates a list of further questions and sends them to the terminal.

[0082] Step 12:

[0083] The terminal displays the new list of questions to the user.

[0084] Step 13:

[0085] The user again inputs an answer to the displayed question. For example, the user answers, "It is due to insufficient training on work procedures."

[0086] Step 14:

[0087] The terminal sends the user's answer to the server.

[0088] Step 15:

[0089] The server stores the received answer in a database. This process is repeated until the root cause is identified.

[0090] Step 16:

[0091] The server analyzes all questions and answers to identify the true cause of the accident.

[0092] Step 17:

[0093] The server will create a report based on the identified root cause, which will include details of the incident, the identified root cause, and recommended countermeasures.

[0094] Step 18:

[0095] The server sends the final report to the terminal.

[0096] Step 19:

[0097] The terminal displays the final report to the user.

[0098] Through this cycle, the system automates the process of efficiently and objectively identifying the root cause of an accident.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] There is a growing need for reliable accident analysis systems, but conventional methods have issues with human bias and analytical efficiency. In particular, it is currently difficult to quickly and objectively identify the true cause of an accident. In addition, there are many manual processes involved in generating questions and collecting answers, so efficient automation is required.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes a means for receiving accident information and storing it in a database, a means for generating questions using a generative AI model, and a means for managing input prompts to the generative AI model, which enables the storage and analysis of accident information, automatic generation of questions, and rapid identification of the root cause.

[0104] "Accident information" refers to detailed data about an accident, such as the date and time of the accident, the location, a summary of the accident, and information about the people involved.

[0105] An "interface" refers to the screens and functions that allow users to input information into a system, and is usually accessed through a web browser or dedicated application.

[0106] A "server" is a computer system used to receive information from users and store it in a database.

[0107] A "database" is a system for structuring and storing received accident information, and a relational database is usually used.

[0108] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on input data and generates new questions and answers.

[0109] A "prompt sentence" is an input sentence that gives a specific question or instruction to a generative AI model.

[0110] "Question generation" is the process of automatically generating specific "why" questions using a generative AI model based on accident information.

[0111] "Answer collection" is the process of collecting answers entered by users to questions posed and storing them in a database.

[0112] "Root cause identification" is the process of analyzing all collected questions and answers to find the underlying cause of the accident.

[0113] The "report format" is a document format that summarizes the true cause of the accident, the analysis results based on that, and improvement measures.

[0114] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. A specific embodiment of this system is described below.

[0115] User input of accident information

[0116] Users can enter accident information using a dedicated interface, which can be provided via a web browser or a dedicated application, and can enter information about the date and time of the accident, the location, a summary of the accident, and the people involved.

[0117] Accident information registration and initial analysis

[0118] The terminal sends the accident information entered by the user to the server, which then stores the received accident information in a relational database (e.g., MySQL (registered trademark), PostgreSQL). This structured the accident information and makes it available for later analysis and query generation.

[0119] Generating initial "why" questions

[0120] The server uses a generative AI model (e.g., GPT-4 (registered trademark)) to generate an initial "why" question based on the stored accident information. An input prompt is sent to the generative AI model, which then outputs an appropriate question. This generation process automatically generates a specific question.

[0121] Asking questions and collecting answers

[0122] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the displayed question, and this answer is sent to the server via the terminal. The server stores the received answer in the database again.

[0123] Generate further questions based on answers

[0124] Based on the collected answers, the server re-enters the prompt sentence into the generative AI model to generate a new "why" question, which then automatically generates the next question based on the user's answer. This process is repeated until the root cause is identified.

[0125] Identifying the root cause and generating a final report

[0126] The server analyzes all questions and answers to identify the root cause of the accident. The generated report includes the root cause of the accident, the analysis results based on the root cause, and the safety measures and improvement methods to be applied. The server then sends the final report to the terminal, which displays it to the user.

[0127] Specific examples

[0128] Case 1:

[0129] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0130] The server uses generative AI to generate a question such as "Why did the equipment fall?"

[0131] The user answers, "There was a problem with the way it was handled."

[0132] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0133] The user answers, "It is due to insufficient training on work procedures."

[0134] The server analyzes the responses, identifies the true cause of the accident as "insufficient training on work procedures," and generates a final report, which is displayed to the user via their terminal.

[0135] This system uses a generative AI model to automate the question generation process, efficiently identifying the true cause of an accident while eliminating human bias. By sending the prompt text "Accident summary: Equipment fell, injuring a worker. Why did the equipment fall?" to the generative AI model, the system can generate appropriate questions.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1:

[0138] The user inputs accident information using a dedicated interface. This interface is provided via a web browser or a dedicated application. The input accident information includes the date and time of the accident, the location, a summary of the accident, and information on the people involved. Once input is complete, the device sends this information to the server.

[0139] Input: Accident information (date, time, location, summary, information of people involved)

[0140] Output: Accident information sent to the server

[0141] Step 2:

[0142] The server stores the received accident information in a relational database, such as MySQL or PostgreSQL. The information stored in the database is used for subsequent analysis and query generation.

[0143] Input: Accident information sent from the device

[0144] Output: Accident information stored in the database

[0145] Specific operation: The server issues an INSERT query to the database to save the incident information.

[0146] Step 3:

[0147] The server uses a generative AI model to generate an initial "why" question based on the stored accident information. This process uses a generative AI (e.g., GPT-4). The server generates a prompt sentence containing a summary of the accident and sends it to the AI ​​model to output an appropriate question.

[0148] Input: Accident information stored in the database

[0149] Output: Generated "why" questions

[0150] Specific operation: Send a prompt (e.g., "Accident summary: Equipment fell and injured a worker. Why did the equipment fall?") ​​to the generation AI and get a question.

[0151] Step 4:

[0152] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the question, and the terminal sends the answer to the server.

[0153] Input: Generated Question

[0154] Output: User's answer

[0155] Specific operation: The device displays a question on the user interface, and the user enters the answer in the text box and submits it.

[0156] Step 5:

[0157] The server stores the answers received from the user in a database again. Then, it generates new "why" questions based on the generative AI model. The server inputs the collected answers as prompts into the generative AI model to generate further questions.

[0158] Input: User's answer

[0159] Output: New "why" questions

[0160] Specific operation: The server saves the answer in the database by issuing an INSERT query, and sends a new prompt to the generation AI to get the next question.

[0161] Step 6:

[0162] The server repeats this process until the root cause is identified. It analyzes all questions and answers to identify the root cause of the accident. The final analysis results are generated in the form of a report and displayed to the user via the terminal.

[0163] Input: All questions and answers

[0164] Output: The true cause of the accident and a report based on it

[0165] Specific operation: The server analyzes the accumulated data using an analysis algorithm and generates a report. The generated report is sent to the terminal and displayed to the user.

[0166] (Application example 1)

[0167] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0168] When an accident occurs in equipment construction or industrial facilities, there is a need to quickly and objectively identify and analyze the cause of the accident. However, current manual analysis methods are inefficient and prone to human bias. Accurately recording the accident situation and quickly analyzing it are particularly difficult when it comes to accidents involving robots. To address this issue, there is an urgent need to provide an automated system.

[0169] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0170] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" questions, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for automatically transmitting accident information when the robot detects an accident, means for the server to perform an initial analysis of the accident information in real time and generate questions using a generation AI, and means for the robot administrator to answer the questions and then generate new questions based on the answers. This enables accurate recording and rapid analysis of accident information.

[0171] The "interface for inputting accident information" is a component of the system that allows the user to input information such as the date and time of the accident, the location, and a summary of the accident.

[0172] The "means for receiving the accident information and storing it in a database" is a function for taking the accident information entered by the user into the system and storing it in a database.

[0173] The "generation means for generating a first 'why' question based on the accident information" is a function that automatically generates the first 'why' question using a generation AI based on the input accident information.

[0174] The "means for presenting the generated first 'why' question" is a function for displaying the generated question to the user.

[0175] The "means for collecting answers to the first 'why' question" is a function that allows a user to input an answer to a 'why' question and collect the answer within the system.

[0176] The "generation means for generating more detailed 'why' questions based on the collected answers" is a function that uses a generation AI to create more detailed 'why' questions based on the user's answers.

[0177] The "means for presenting the more detailed 'why' question" is a function for displaying a more detailed question to the user.

[0178] The "means for analyzing all of the questions and answers to identify the true cause of the accident" is a function for comprehensively analyzing all of the questions and answers to identify the root cause of the accident.

[0179] The "means for presenting the identified true cause in the form of a report" is a function for creating the identified true cause of the accident in the form of a report and presenting it to the user.

[0180] The "means for automatically transmitting accident information when the robot detects an accident" is a function for automatically transmitting information about an accident to a server when the robot detects an accident.

[0181] "Means for the server to perform initial analysis of accident information in real time and generate questions using generation AI" refers to a function that allows the server to immediately perform initial analysis of the accident information received and generate "why" questions using generation AI based on that information.

[0182] "Means for the robot administrator to answer questions and generate new questions based on those answers" is a system function that allows the robot administrator to answer questions presented to them and generate new "why" questions based on those answers.

[0183] As an application example of the present invention, we will show how to specifically implement an accident cause analysis system for a factory robot. The system is configured as follows.

[0184] System configuration

[0185] 1. Interface for entering accident information

[0186] The factory robot's control terminal is provided with an interface for inputting accident information. The user enters the date and time of the accident, the location, a summary of the accident, information about the people involved, etc. This interface is operated using a touch panel display and keyboard.

[0187] 2. A means of storing accident information in a database

[0188] Accident information entered on the terminal is sent to the server, which stores the information in a database, allowing for efficient management of accident information.

[0189] 3. Generating "why" questions

[0190] The server analyzes the accumulated accident information and uses a generative AI model to generate the first "why" question. For example, a specific question such as "Why did the equipment fall?" is automatically generated based on the input accident information.

[0191] 4. How to pose questions and collect answers

[0192] The generated questions are displayed on the management terminal, and the user inputs answers to the questions. The terminal has a function to collect the answers, and the information is sent to the server.

[0193] 5. Generators for generating detailed "why" questions

[0194] The server uses the user's answers to generate more detailed "why" questions. This process is carried out using generative AI, and a question such as "Why was there a problem with the handling method?" is generated.

[0195] 6. Rephrasing the question

[0196] The generated new questions are then displayed on the terminal again, and the user enters their answers. This process continues until the true cause of the accident is identified.

[0197] 7. Identifying the root cause and presenting it in a report format

[0198] The server analyzes all questions and answers to identify the root cause of the accident. Finally, the analysis results are generated in the form of a report and presented to the terminal. This report contains the cause of the accident, all related information, and remedial measures.

[0199] Hardware and software used

[0200] Hardware: Robots in factories, devices for inputting accident information (tablets, PCs, etc.), servers

[0201] Software: Python, Requests library, OpenAI® API

[0202] Data processing and calculation

[0203] After receiving the accident information, the server stores it in a database. Based on the stored data, a generative AI model is used to generate "why" questions. The answers collected from users are analyzed again, and more detailed questions are generated using the generative AI model. By repeating this process, the true cause of the accident can be identified.

[0204] Specific examples

[0205] For example, a general outline of an accident in a factory might be entered, such as "A robot arm broke down, injuring a worker." Based on this information, the server uses generative AI to generate the first question: "Why did the robot arm break down?" If the user answers, "It was caused by lack of maintenance," the server generates a more detailed question, such as, "Why was there lack of maintenance?" In this way, the true cause of the failure is identified and compiled into a report.

[0206] Prompt Sentence Examples

[0207] Accident summary: The robot arm malfunctioned, injuring a worker.

[0208] Generate initial "why" questions to analyze the cause of this accident.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] The user uses a terminal to input accident information. The input includes the date, time, location, summary, and information on the people involved. This information is imported into the system as the initial accident data. An example of input data would be "October 5, 2023, factory line 1, a robot arm malfunctioned, injuring a worker." This provides the system with basic data for analysis.

[0212] Step 2:

[0213] The device sends the entered accident information to the server, which receives the information and stores it in a database. This creates an environment in which accident information can be continuously managed within the system.

[0214] Step 3:

[0215] The server uses a generative AI model to generate a first "why" question based on the stored accident information. For example, a question such as "Why did the robot arm break down?" is generated. In the generation process, the accident information is used as input data, and the generative AI model analyzes it and outputs a prompt sentence.

[0216] Step 4:

[0217] The server sends the generated "why" question to the device, which then presents it to the user. The user then responds to the displayed question. An example response might include specific details such as "It's due to lack of maintenance." This interaction provides the basis for subsequent analysis.

[0218] Step 5:

[0219] The device sends the user's answer to the server, which then uses the generative AI model to generate a more detailed "why" question, such as "Why was maintenance lacking?" This allows the server to investigate the cause at a deeper level.

[0220] Step 6:

[0221] The server then sends the generated detailed "why" question to the device, which then presents it to the user. The user then re-enters the answer, and the process is repeated. For example, the answer could be "there was no maintenance plan in place." This repetition helps pinpoint the true cause of the accident.

[0222] Step 7:

[0223] After collecting all the questions and answers, the server analyzes them to identify the root cause of the final accident. This analysis uses all the questions and answers stored in the database. The server generates a final report based on the root cause identified.

[0224] Step 8:

[0225] The server then sends the final report to the terminal, where the user can review it. The report includes details of the accident, its causes, and remedial measures, enabling efficient and objective identification of the cause of the accident and the formulation of countermeasures.

[0226] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0227] This invention is a system for objectively identifying and analyzing the cause of accidents that occur during facility construction. In particular, this system incorporates an emotion engine that recognizes the user's emotions and realizes interactions that correspond to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user. Specific embodiments and the processing flow are described below.

[0228] User input of accident information

[0229] The user first inputs accident information through a dedicated interface. The input accident information includes the date, time, location, summary, and information on the people involved. The emotion engine analyzes the voice and input content as the user inputs and recognizes the user's emotional state.

[0230] Accident information registration and initial analysis

[0231] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[0232] Generating initial "why" questions

[0233] The server uses a generative AI to generate the first "why" question based on the stored accident information. Based on the analysis results of the emotion engine, the server adjusts the number and difficulty of questions if the user is confused, for example. For example, it automatically generates specific questions such as "Why did the equipment fall?"

[0234] Asking questions and collecting answers

[0235] The server sends the generated questions to the device, which displays them to the user. The emotion engine continuously monitors the user's emotional state and flexibly adjusts the way questions are presented if the user is feeling stressed. The user then inputs answers to the questions. An example answer might be, "There was a problem with the way the equipment was handled."

[0236] Generate further questions based on answers

[0237] The device sends the user's answer to the server, which again uses generative AI to generate further "why" questions based on the previous answer. The emotion engine takes the user's emotional state into account and re-adjusts the question and how it is presented. For example, a question might be generated such as, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[0238] Identifying the root cause and generating a final report

[0239] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and the emotion engine adjusts the explanation taking into account the user's level of understanding. Finally, the server sends this report to the terminal, which displays it to the user.

[0240] Specific examples

[0241] Case 1:

[0242] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0243] The server uses a generative AI to generate the question, "Why did the equipment fall?" The question "Why?" is softened by the emotion engine: "Did something go wrong?"

[0244] If the user answers "there was a problem with the way it was handled," the emotion engine will detect the user's stress level and adjust the next question to be shorter.

[0245] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0246] The user responded that "the cause is a lack of training on work procedures."

[0247] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then presents the user with a final report with an easy-to-understand explanation.

[0248] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[0249] The processing flow will be explained below.

[0250] Processing steps of an accident cause analysis system that combines an emotion engine

[0251] Step 1:

[0252] The user opens the accident information entry screen. The interface is visually easy to understand and provides fields for entering details such as the date, time, location, and circumstances of the accident.

[0253] Step 2:

[0254] The user inputs the incident information, and the emotion engine recognizes emotions from the user's input and voice to assess the level of stress and confusion.

[0255] Step 3:

[0256] The terminal transmits the input accident information and the emotion data evaluated by the emotion engine to the server.

[0257] Step 4:

[0258] The server stores the received accident information and emotion data in a database.

[0259] Step 5:

[0260] The server activates the AI ​​generator, which generates the first "why" question based on the stored accident information. It takes into account emotional data and adjusts the question accordingly, such as shortening the question if the user is nervous.

[0261] Step 6:

[0262] The server generates a first "why" question and sends it to the terminal as a question list.

[0263] Step 7:

[0264] The device displays a list of questions to the user, adjusting them based on emotional data and presenting them in a format that is easy for the user to understand.

[0265] Step 8:

[0266] The user inputs an answer to the question presented, for example, "There was a problem with the way the equipment was handled."

[0267] Step 9:

[0268] The terminal sends the user's answer to the server.

[0269] Step 10:

[0270] The server stores the received response in a database.

[0271] Step 11:

[0272] The server then reactivates the generative AI to generate further "why" questions based on the previous answers, while the emotion engine continues to monitor the user's emotional state and adjusts the complexity and format of the questions accordingly.

[0273] Step 12:

[0274] The server generates a new list of questions and sends it to the terminal.

[0275] Step 13:

[0276] The device presents the user with a new list of questions, the order and format of which are adjusted based on the user's emotional state.

[0277] Step 14:

[0278] The user again inputs an answer to the question. For example, the user answers, "It is due to a lack of training on work procedures."

[0279] Step 15:

[0280] The device sends the user's answer to the server, and this procedure is repeated until the root cause is identified.

[0281] Step 16:

[0282] The server analyzes all questions and answers to identify the root cause of the accident. The emotion engine adjusts the results during the final analysis to make them easier for users to understand.

[0283] Step 17:

[0284] The server creates a report based on the identified root cause, using a document format that takes emotion data into account.

[0285] Step 18:

[0286] The server sends the final report to the terminal.

[0287] Step 19:

[0288] The terminal displays the final report to the user, which includes details of the accident, the identified root causes, and recommended actions.

[0289] Specific examples

[0290] Case 1:

[0291] A user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The emotion engine evaluates the user's stress level as "high."

[0292] The server generates the question "Why did the equipment fall?", but the emotion engine changes this to a softer expression such as "Please tell us what caused the equipment to fall."

[0293] The user responded that "there was a problem with the way it was handled." The emotion engine detected that the user was still feeling stressed.

[0294] The server generates the next "why" question: "Why was there a problem with the handling method?" and presents it in a concise and easy-to-understand manner.

[0295] The user responded that "it was due to a lack of training on work procedures." The emotion engine detected that the user's stress had been alleviated.

[0296] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then creates a final report in an easy-to-understand document format and presents it to the user.

[0297] In this way, the system automates the process of efficiently and objectively identifying the root cause of an accident, while also enabling interaction that responds to the user's emotional state.

[0298] Example 2

[0299] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0300] When an accident occurs during facility construction, a process is needed to objectively and efficiently identify and analyze the cause. However, conventional systems are prone to human bias and lack interaction that takes into account the user's emotional state, which can cause users to feel stressed and reduce the quality of answers. Another issue is that the process of entering accident information, generating questions, and analyzing them is complex and time-consuming.

[0301] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0302] In this invention, the server includes means for providing a user interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating a more detailed "why" question based on the collected answers, means for presenting the more detailed "why" question, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for recognizing the user's emotional state and adjusting the wording of the question, and means for adjusting the method of presenting the question in consideration of the user's emotional state. This makes it possible to efficiently and objectively identify the true cause of the accident and realize interaction that takes the user's emotional state into consideration.

[0303] "Accident Information"

[0304] This refers to the date, time, location, summary, and information on the people involved in accidents during facility construction work.

[0305] User Interface

[0306] This refers to the screen or input means through which the user can input accident information.

[0307] Database

[0308] refers to an information storage device that stores received accident information and is used for later analysis and question generation.

[0309] "Generation means"

[0310] refers to methods and techniques for generating "why" questions based on the accident information.

[0311] "Method of presenting questions"

[0312] refers to the means by which the server displays generated questions to the user.

[0313] "Method of collecting responses"

[0314] refers to methods and techniques for collecting answers to questions entered by users.

[0315] A way to identify the root cause

[0316] refers to the methods and techniques used to analyze all questions and answers and identify the root cause of an accident.

[0317] "Report format"

[0318] This refers to the format and layout used to present identified root causes and analysis results in an easy-to-understand manner for users.

[0319] "A means of recognizing emotional states"

[0320] refers to technologies and methods for analyzing a user's voice or text to detect their emotional state.

[0321] "A way to adjust the wording of a question"

[0322] refers to techniques and methods for modifying the content and wording of questions based on the detected emotional state of the user.

[0323] "A way to tailor how questions are presented"

[0324] refers to techniques and methods for taking into account the user's emotional state and modifying the way questions are presented.

[0325] This invention is a system for objectively identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates an emotion engine that recognizes the user's emotions, enabling interaction that corresponds to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[0326] User input of accident information

[0327] Users first enter accident information through a dedicated interface. This interface is provided by an application installed on devices such as PCs and smartphones. The entered accident information includes the date, time, location, summary, and information on the people involved in the accident. For example, the information entered might be, "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[0328] The device collects this information and uses an emotion engine to analyze the user's voice and text to recognize their emotional state. The emotion engine uses voice recognition and natural language processing (NLP) to detect when the user is stressed or confused.

[0329] Accident information registration and initial analysis

[0330] The terminal transmits the accident information received from the user to the server. This is done in real time via the network. The server stores the received accident information in a database. This database is used for later analysis and query generation.

[0331] Generating initial "why" questions

[0332] The server uses a generative AI model to generate the first "why" question based on the collected accident information. The generative AI model uses an advanced natural language generation model such as GPT-4. For example, a specific question such as "Why did the equipment fall?" is generated.

[0333] The emotion engine takes into account the user's emotional state and adjusts the number and difficulty of questions. If the user is confused, it may soften the wording of the questions. For example, it may change the question "Why?" to "Did something go wrong?"

[0334] Asking questions and collecting answers

[0335] The server sends the generated questions to the device, which then displays them to the user. The display method is adjusted according to the user's emotional state. For example, it may be a visual display or a voice prompt. The user then inputs an answer to the question. For example, the user may reply, "There was a problem with the way it was handled."

[0336] Generate further questions based on answers

[0337] The device sends the user's answer to the server, which again uses the generative AI model to generate the next "why" question. The emotion engine again analyzes the user's emotional state and readjusts the question and how it is presented. For example, it generates a question like, "Why was there a problem with the handling?" This process is repeated until the true cause of the accident is identified.

[0338] Identifying the root cause and generating a final report

[0339] The server analyzes all questions and answers to identify the true cause of the accident. The analysis results are generated in the form of a report, and an emotion engine adjusts the explanation based on the user's level of understanding. The final report is sent from the server to the terminal and displayed to the user.

[0340] Prompt Sentence Examples

[0341] "Analyze why a piece of equipment fell on a construction site on October 5, 2023, injuring a worker. Use an emotion engine to adjust the difficulty and presentation of questions according to the user's stress level, and generate a final report."

[0342] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[0343] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0344] Step 1:

[0345] The user inputs the accident information.

[0346] The input items are the date and time of the accident, location, summary, and information on the people involved.

[0347] For example, enter "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[0348] The user interface receives the user's voice and text input and converts it into digital information using speech recognition and natural language processing.

[0349] Step 2:

[0350] The terminal transmits the input accident information to the server.

[0351] At this time, the transmitted data is structured in JSON format or similar.

[0352] The server parses the received data and stores it in a database.

[0353] For example, an SQL query is used to insert data into an "accident information" table.

[0354] Step 3:

[0355] The server generates the first "why" question using a generative AI model.

[0356] The input is stored accident information, and a question such as "Why did the equipment fall?" is input as a prompt to the generative AI model (e.g., GPT-4).

[0357] The output is the generated "why" question.

[0358] Step 4:

[0359] An emotional engine takes into account the user's emotional state to adjust the number and difficulty of questions.

[0360] The input is data on the user's emotional state, including the results of speech and text analysis.

[0361] For example, if you determine that the person is confused, change the question "Why?" to a softer phrase such as "Did something go wrong?"

[0362] Step 5:

[0363] The server sends the tailored question to the terminal.

[0364] The data sent is restructured and contains the question data in JSON format.

[0365] The terminal receives this and displays it to the user on the user interface.

[0366] Step 6:

[0367] The user enters answers to the questions presented.

[0368] For example, you might answer, "There was a problem with the way it was handled."

[0369] The answers entered are in voice or text format, which the terminal collects and converts into digital data.

[0370] Step 7:

[0371] The terminal sends the user's answer to the server.

[0372] The data sent is again structured in JSON format or similar.

[0373] The server parses the received data and uses it to generate the next "why" question.

[0374] Step 8:

[0375] The server again uses the generative AI model to generate the next "why" question.

[0376] The input is the user's previous answer, and a question is generated, for example, "Why was there a problem with the handling method?"

[0377] The output is the following "why" question:

[0378] Step 9:

[0379] The emotion engine again takes into account the user's emotional state and readjusts the content and presentation of the next question.

[0380] For example, if the user is feeling stressed, shorten the questions.

[0381] Step 10:

[0382] The server sends the retuned question to the terminal, which displays it to the user.

[0383] This process is repeated until the true cause of the accident is identified.

[0384] Step 11:

[0385] The server analyzes all questions and answers to identify the true cause of the accident.

[0386] The input is all question and answer data, which is analyzed using natural language processing techniques.

[0387] For example, identify it as being caused by "lack of training on how to handle it."

[0388] Step 12:

[0389] The server generates the final report.

[0390] The input is the analysis results, and the emotion engine adjusts the report description to match the user's level of understanding.

[0391] The generated report is saved in PDF format.

[0392] Step 13:

[0393] The server sends the final report to the terminal, which displays it to the user.

[0394] Users can view the reports through the application.

[0395] (Application example 2)

[0396] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0397] In conventional accident response systems, questions to identify the true cause of an accident are asked mechanically, and the user's emotional state cannot be taken into consideration. As a result, stressed users are unable to answer the questions accurately, making it difficult to identify the true cause. In addition, it is difficult to reflect emotions and the specific situation at the scene when inputting and analyzing accident information, so improvements in efficiency and accuracy are required.

[0398] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0399] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" question, means for analyzing all the questions and answers to identify the true cause of the accident, means for presenting the identified true cause in the form of a report, emotion analysis means for recognizing the emotions of workers, adjustment means for adjusting the question method and question content based on the emotion analysis means, and data collection means for collecting video and audio of the accident site. This makes it possible to efficiently and accurately identify the true cause of the accident while taking emotions and circumstances into consideration.

[0400] "Accident information" is data including the date, time, location, and specific circumstances of the accident.

[0401] An "interface" is a means by which a user inputs accident information.

[0402] "Database" refers to a system for storing received accident information.

[0403] The "generation means" is a means for generating a question based on the accident information.

[0404] The "presentation means" is a means for presenting the generated question to the user.

[0405] The "collection means" is a means by which users input answers and collect the answers.

[0406] The "emotion analysis means" is a means for recognizing and analyzing the emotions of workers.

[0407] The "adjustment means" is a means for adjusting the method and content of questions based on the results of the emotion analysis means.

[0408] "Data collection means" refers to means for collecting video and audio data from the accident scene.

[0409] "Generative AI" is a technology that uses artificial intelligence to generate questions based on accident information.

[0410] A "report" is a document that presents the identified root causes of an accident.

[0411] This invention is a system for identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates emotion analysis means that recognizes the user's emotions, enabling interaction according to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[0412] Hardware and software used

[0413] Hardware:

[0414] Robots operating in a factory

[0415] Microphones, cameras, and high-precision sensors

[0416] software:

[0417] Server (example databases are MySQL or PostgreSQL)

[0418] Generative AI (e.g., GPT-4)

[0419] Sentiment analysis tools (e.g., a BERT model using Transformers on Hugging Face)

[0420] System configuration and operation

[0421] Entering accident information

[0422] First, the user inputs accident information through a dedicated interface. This input includes the date, time, location, and summary of the accident. Audio and video data collected by the robot are also added to the accident information. The emotion analysis means recognizes the emotional state indicated by the user through their input and voice.

[0423] Accident information registration and initial analysis

[0424] The device sends the accident information entered by the user to the server. The server stores the received accident information in a database. This database is used for subsequent analysis and question generation. The generative AI generates an initial "why" question and adjusts the wording of the question based on the results of sentiment analysis.

[0425] Asking questions and collecting answers

[0426] The server sends the generated questions to the terminal, which displays them to the user. The emotion analysis means monitors the user's emotional state and adjusts the wording and length of the questions if the user is feeling stressed. The user enters answers to the questions into the interface. The answers are sent to the server and stored in a database.

[0427] Generate further questions based on answers

[0428] The server uses generative AI to generate additional "why" questions based on the user's answers, again adjusting the wording and content of the questions based on the user's emotional state. This process is repeated until the true cause of the accident is identified.

[0429] Identifying the root cause and generating a final report

[0430] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and a sentiment analysis tool adjusts the explanation to take into account the user's level of understanding. The final report is sent to the terminal and presented to the user.

[0431] Specific examples

[0432] Prompt statement

[0433] Previous answer: There was a problem with the way it was handled. Generate the following "why" questions:

[0434] Specific scenarios

[0435] The user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The server uses a generation AI to generate the question, "Why did the equipment fall?" Based on the analysis results of the sentiment analysis means, this question is changed to a softer expression, "Did something go wrong?" If the user answers, "There was a problem with the handling," the sentiment analysis means detects the user's stress level and adjusts the next question to be shorter. The server generates the next "why" question, "Why was there a problem with the handling?" If the user answers, "It was caused by insufficient training on work procedures," the server analyzes the answer and identifies the true cause of the accident as "insufficient training on work procedures." The sentiment analysis means then presents the user with a final report that has been adjusted to an easy-to-understand explanation.

[0436] This makes it possible to efficiently and accurately identify the true cause of an accident while taking into account the user's emotional state.

[0437] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0438] Step 1:

[0439] Users input accident information through a dedicated interface. The input accident information includes the date, time, location, and summary of the accident. Audio and video data are also collected during input. The input from this interface becomes the initial input data for the system.

[0440] Step 2:

[0441] The collected accident information is sent to the server by the terminal. The server stores the received information in a database. At this time, the voice input data is converted to text, and the video data is also recorded as accident information.

[0442] Step 3:

[0443] The server uses a generative AI to generate a first "why" question based on the stored accident information. For example, it generates a specific question such as "Why did the equipment fall?" The generative AI analyzes the input data and generates a question using a prompt sentence.

[0444] Step 4:

[0445] The server sends the generated question to the terminal, which then displays the question to the user in a voice or text format that is easy for the user to answer.

[0446] Step 5:

[0447] The user inputs answers to the questions. This input answer is also captured in the system in the form of voice or text. The emotion analysis means analyzes this input and recognizes the user's emotional state.

[0448] Step 6:

[0449] The device sends the user's response data to the server, which then stores the received response data in a database, and also stores the emotional commentary data along with the analysis results.

[0450] Step 7:

[0451] The server uses generative AI to generate further "why" questions based on the previous answer, and adjusts the content and wording of the generated questions based on the results of sentiment analysis. The questions are then sent back to the device.

[0452] Step 8:

[0453] The question is again presented to the user from the terminal. The question is adjusted to allow the user to enter a short or concise answer. The user again enters the answer to the question.

[0454] Step 9:

[0455] The device repeatedly sends the user's input to the server, which again uses generative AI to generate the next "why" question, and this process is repeated until the true cause of the accident is identified.

[0456] Step 10:

[0457] The server analyzes all questions and answers and ultimately identifies the true cause of the accident. The analysis results are generated in the form of a report, and an emotion analysis means adjusts the explanation according to the user's level of understanding.

[0458] Step 11:

[0459] The final report is sent to the terminal, which then presents the report to the user, who reviews the final report and receives information about the true cause of the accident.

[0460] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0461] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0462] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0463] [Second embodiment]

[0464] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0465] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0466] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0467] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0468] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0469] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0470] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0471] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0472] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0473] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0474] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0475] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0476] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. Specific embodiments and the processing flow of this system are described below.

[0477] User input of accident information

[0478] First, the user enters accident information through a dedicated interface. The entered accident information includes the date and time of the accident, location, a summary of the accident, and information on the people involved. This information is used as the base data for subsequent analysis.

[0479] Accident information registration and initial analysis

[0480] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[0481] Generating initial "why" questions

[0482] The server uses generative AI to generate the first "why" question based on the stored accident information. For example, it automatically generates a specific question such as "Why did the equipment fall?"

[0483] Asking questions and collecting answers

[0484] The server sends the generated questions to the terminal, which displays them to the user. The user then inputs an answer to the question. An example of an answer would be something like, "There was a problem with the way the equipment was handled."

[0485] Generate further questions based on answers

[0486] The device sends the user-entered answers to the server, which again uses generative AI to generate further "why" questions based on the previous answers. For example, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[0487] Identifying the root cause and generating a final report

[0488] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, which includes safety measures and improvement methods to be applied. Finally, the server sends this report to the terminal, which displays it to the user.

[0489] Specific examples

[0490] Case 1:

[0491] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0492] The server uses a generative AI to generate a question such as "Why did the equipment fall?"

[0493] The user responded that "there was a problem with the way it was handled."

[0494] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0495] The user responded that "the cause is a lack of training on work procedures."

[0496] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." It then presents the final report to the user.

[0497] In this way, the system can automate the process of efficiently and objectively identifying the root causes of accidents, eliminating human bias.

[0498] The processing flow will be explained below.

[0499] Step 1:

[0500] The user opens a dedicated interface and enters accident information, including the date, time, location, summary, and information about the people involved.

[0501] Step 2:

[0502] The terminal receives the user's input and sends it to the server.

[0503] Step 3:

[0504] The server stores the received accident information in a database.

[0505] Step 4:

[0506] The server launches a generation AI to generate the first "why" question based on the stored accident information, such as "Why did the equipment fall?"

[0507] Step 5:

[0508] The server generates a list of first "why" questions and sends them to the terminal.

[0509] Step 6:

[0510] The terminal displays a list of questions to the user.

[0511] Step 7:

[0512] The user inputs an answer to the displayed question, for example, "There was a problem with the way it was handled."

[0513] Step 8:

[0514] The terminal sends the user's answer to the server.

[0515] Step 9:

[0516] The server stores the received response in a database.

[0517] Step 10:

[0518] The server then launches the generation AI again, generating further "why" questions based on the previous answers, such as "Why was there a problem with the handling method?"

[0519] Step 11:

[0520] The server generates a list of further questions and sends them to the terminal.

[0521] Step 12:

[0522] The terminal displays the new list of questions to the user.

[0523] Step 13:

[0524] The user again inputs an answer to the displayed question. For example, the user answers, "It is due to insufficient training on work procedures."

[0525] Step 14:

[0526] The terminal sends the user's answer to the server.

[0527] Step 15:

[0528] The server stores the received answer in a database. This process is repeated until the root cause is identified.

[0529] Step 16:

[0530] The server analyzes all questions and answers to identify the true cause of the accident.

[0531] Step 17:

[0532] The server will create a report based on the identified root cause, which will include details of the incident, the identified root cause, and recommended countermeasures.

[0533] Step 18:

[0534] The server sends the final report to the terminal.

[0535] Step 19:

[0536] The terminal displays the final report to the user.

[0537] Through this cycle, the system automates the process of efficiently and objectively identifying the root cause of an accident.

[0538] Example 1

[0539] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0540] There is a growing need for reliable accident analysis systems, but conventional methods have issues with human bias and analytical efficiency. In particular, it is currently difficult to quickly and objectively identify the true cause of an accident. In addition, there are many manual processes involved in generating questions and collecting answers, so efficient automation is required.

[0541] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0542] In this invention, the server includes a means for receiving accident information and storing it in a database, a means for generating questions using a generative AI model, and a means for managing input prompts to the generative AI model, which enables the storage and analysis of accident information, automatic generation of questions, and rapid identification of the root cause.

[0543] "Accident information" refers to detailed data about an accident, such as the date and time of the accident, the location, a summary of the accident, and information about the people involved.

[0544] An "interface" refers to the screens and functions that allow users to input information into a system, and is usually accessed through a web browser or dedicated application.

[0545] A "server" is a computer system used to receive information from users and store it in a database.

[0546] A "database" is a system for structuring and storing received accident information, and a relational database is usually used.

[0547] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on input data and generates new questions and answers.

[0548] A "prompt sentence" is an input sentence that gives a specific question or instruction to a generative AI model.

[0549] "Question generation" is the process of automatically generating specific "why" questions using a generative AI model based on accident information.

[0550] "Answer collection" is the process of collecting answers entered by users to questions posed and storing them in a database.

[0551] "Root cause identification" is the process of analyzing all collected questions and answers to find the underlying cause of the accident.

[0552] The "report format" is a document format that summarizes the true cause of the accident, the analysis results based on that, and improvement measures.

[0553] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. A specific embodiment of this system is described below.

[0554] User input of accident information

[0555] Users can enter accident information using a dedicated interface, which can be provided via a web browser or a dedicated application, and can enter information about the date and time of the accident, the location, a summary of the accident, and the people involved.

[0556] Accident information registration and initial analysis

[0557] The device sends the accident information entered by the user to the server, which then stores the received accident information in a relational database (e.g., MySQL or PostgreSQL). This structured the accident information and makes it available for later analysis and query generation.

[0558] Generating initial "why" questions

[0559] The server uses a generative AI model (e.g., GPT-4) to generate an initial "why" question based on the stored accident information. The generative AI model receives an input prompt and outputs an appropriate question. This generation process automatically generates a specific question.

[0560] Asking questions and collecting answers

[0561] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the displayed question, and this answer is sent to the server via the terminal. The server stores the received answer in the database again.

[0562] Generate further questions based on answers

[0563] Based on the collected answers, the server re-enters the prompt sentence into the generative AI model to generate a new "why" question, which then automatically generates the next question based on the user's answer. This process is repeated until the root cause is identified.

[0564] Identifying the root cause and generating a final report

[0565] The server analyzes all questions and answers to identify the root cause of the accident. The generated report includes the root cause of the accident, the analysis results based on the root cause, and the safety measures and improvement methods to be applied. The server then sends the final report to the terminal, which displays it to the user.

[0566] Specific examples

[0567] Case 1:

[0568] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0569] The server uses generative AI to generate a question such as "Why did the equipment fall?"

[0570] The user answers, "There was a problem with the way it was handled."

[0571] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0572] The user answers, "It is due to insufficient training on work procedures."

[0573] The server analyzes the responses, identifies the true cause of the accident as "insufficient training on work procedures," and generates a final report, which is displayed to the user via their terminal.

[0574] This system uses a generative AI model to automate the question generation process, efficiently identifying the true cause of an accident while eliminating human bias. By sending the prompt text "Accident summary: Equipment fell, injuring a worker. Why did the equipment fall?" to the generative AI model, the system can generate appropriate questions.

[0575] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0576] Step 1:

[0577] The user inputs accident information using a dedicated interface. This interface is provided via a web browser or a dedicated application. The input accident information includes the date and time of the accident, the location, a summary of the accident, and information on the people involved. Once input is complete, the device sends this information to the server.

[0578] Input: Accident information (date, time, location, summary, information of people involved)

[0579] Output: Accident information sent to the server

[0580] Step 2:

[0581] The server stores the received accident information in a relational database, such as MySQL or PostgreSQL. The information stored in the database is used for subsequent analysis and query generation.

[0582] Input: Accident information sent from the device

[0583] Output: Accident information stored in the database

[0584] Specific operation: The server issues an INSERT query to the database to save the incident information.

[0585] Step 3:

[0586] The server uses a generative AI model to generate an initial "why" question based on the stored accident information. This process uses a generative AI (e.g., GPT-4). The server generates a prompt sentence containing a summary of the accident and sends it to the AI ​​model to output an appropriate question.

[0587] Input: Accident information stored in the database

[0588] Output: Generated "why" questions

[0589] Specific operation: Send a prompt (e.g., "Accident summary: Equipment fell and injured a worker. Why did the equipment fall?") ​​to the generation AI and get a question.

[0590] Step 4:

[0591] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the question, and the terminal sends the answer to the server.

[0592] Input: Generated Question

[0593] Output: User's answer

[0594] Specific operation: The device displays a question on the user interface, and the user enters the answer in the text box and submits it.

[0595] Step 5:

[0596] The server stores the answers received from the user in a database again. Then, it generates new "why" questions based on the generative AI model. The server inputs the collected answers as prompts into the generative AI model to generate further questions.

[0597] Input: User's answer

[0598] Output: New "why" questions

[0599] Specific operation: The server saves the answer in the database by issuing an INSERT query, and sends a new prompt to the generation AI to get the next question.

[0600] Step 6:

[0601] The server repeats this process until the root cause is identified. It analyzes all questions and answers to identify the root cause of the accident. The final analysis results are generated in the form of a report and displayed to the user via the terminal.

[0602] Input: All questions and answers

[0603] Output: The true cause of the accident and a report based on it

[0604] Specific operation: The server analyzes the accumulated data using an analysis algorithm and generates a report. The generated report is sent to the terminal and displayed to the user.

[0605] (Application example 1)

[0606] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0607] When an accident occurs in equipment construction or industrial facilities, there is a need to quickly and objectively identify and analyze the cause of the accident. However, current manual analysis methods are inefficient and prone to human bias. Accurately recording the accident situation and quickly analyzing it are particularly difficult when it comes to accidents involving robots. To address this issue, there is an urgent need to provide an automated system.

[0608] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0609] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" questions, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for automatically transmitting accident information when the robot detects an accident, means for the server to perform an initial analysis of the accident information in real time and generate questions using a generation AI, and means for the robot administrator to answer the questions and then generate new questions based on the answers. This enables accurate recording and rapid analysis of accident information.

[0610] The "interface for inputting accident information" is a component of the system that allows the user to input information such as the date and time of the accident, the location, and a summary of the accident.

[0611] The "means for receiving the accident information and storing it in a database" is a function for taking the accident information entered by the user into the system and storing it in a database.

[0612] The "generation means for generating a first 'why' question based on the accident information" is a function that automatically generates the first 'why' question using a generation AI based on the input accident information.

[0613] The "means for presenting the generated first 'why' question" is a function for displaying the generated question to the user.

[0614] The "means for collecting answers to the first 'why' question" is a function that allows a user to input an answer to a 'why' question and collect the answer within the system.

[0615] The "generation means for generating more detailed 'why' questions based on the collected answers" is a function that uses a generation AI to create more detailed 'why' questions based on the user's answers.

[0616] The "means for presenting the more detailed 'why' question" is a function for displaying a more detailed question to the user.

[0617] The "means for analyzing all of the questions and answers to identify the true cause of the accident" is a function for comprehensively analyzing all of the questions and answers to identify the root cause of the accident.

[0618] The "means for presenting the identified true cause in the form of a report" is a function for creating the identified true cause of the accident in the form of a report and presenting it to the user.

[0619] The "means for automatically transmitting accident information when the robot detects an accident" is a function for automatically transmitting information about an accident to a server when the robot detects an accident.

[0620] "Means for the server to perform initial analysis of accident information in real time and generate questions using generation AI" refers to a function that allows the server to immediately perform initial analysis of the accident information received and generate "why" questions using generation AI based on that information.

[0621] "Means for the robot administrator to answer questions and generate new questions based on those answers" is a system function that allows the robot administrator to answer questions presented to them and generate new "why" questions based on those answers.

[0622] As an application example of the present invention, we will show how to specifically implement an accident cause analysis system for a factory robot. The system is configured as follows.

[0623] System configuration

[0624] 1. Interface for entering accident information

[0625] The factory robot's control terminal is provided with an interface for inputting accident information. The user enters the date and time of the accident, the location, a summary of the accident, information about the people involved, etc. This interface is operated using a touch panel display and keyboard.

[0626] 2. A means of storing accident information in a database

[0627] Accident information entered on the terminal is sent to the server, which stores the information in a database, allowing for efficient management of accident information.

[0628] 3. Generating "why" questions

[0629] The server analyzes the accumulated accident information and uses a generative AI model to generate the first "why" question. For example, a specific question such as "Why did the equipment fall?" is automatically generated based on the input accident information.

[0630] 4. How to pose questions and collect answers

[0631] The generated questions are displayed on the management terminal, and the user inputs answers to the questions. The terminal has a function to collect the answers, and the information is sent to the server.

[0632] 5. Generators for generating detailed "why" questions

[0633] The server uses the user's answers to generate more detailed "why" questions. This process is carried out using generative AI, and a question such as "Why was there a problem with the handling method?" is generated.

[0634] 6. Rephrasing the question

[0635] The generated new questions are then displayed on the terminal again, and the user enters their answers. This process continues until the true cause of the accident is identified.

[0636] 7. Identifying the root cause and presenting it in a report format

[0637] The server analyzes all questions and answers to identify the root cause of the accident. Finally, the analysis results are generated in the form of a report and presented to the terminal. This report contains the cause of the accident, all related information, and remedial measures.

[0638] Hardware and software used

[0639] Hardware: Robots in factories, devices for inputting accident information (tablets, PCs, etc.), servers

[0640] Software: Python, Requests library, OpenAI API

[0641] Data processing and calculation

[0642] After receiving the accident information, the server stores it in a database. Based on the stored data, a generative AI model is used to generate "why" questions. The answers collected from users are analyzed again, and more detailed questions are generated using the generative AI model. By repeating this process, the true cause of the accident can be identified.

[0643] Specific examples

[0644] For example, a general outline of an accident in a factory might be entered, such as "A robot arm broke down, injuring a worker." Based on this information, the server uses generative AI to generate the first question: "Why did the robot arm break down?" If the user answers, "It was caused by lack of maintenance," the server generates a more detailed question, such as, "Why was there lack of maintenance?" In this way, the true cause of the failure is identified and compiled into a report.

[0645] Prompt Sentence Examples

[0646] Accident summary: The robot arm malfunctioned, injuring a worker.

[0647] Generate initial "why" questions to analyze the cause of this accident.

[0648] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0649] Step 1:

[0650] The user uses a terminal to input accident information. The input includes the date, time, location, summary, and information on the people involved. This information is imported into the system as the initial accident data. An example of input data would be "October 5, 2023, factory line 1, a robot arm malfunctioned, injuring a worker." This provides the system with basic data for analysis.

[0651] Step 2:

[0652] The device sends the entered accident information to the server, which receives the information and stores it in a database. This creates an environment in which accident information can be continuously managed within the system.

[0653] Step 3:

[0654] The server uses a generative AI model to generate a first "why" question based on the stored accident information. For example, a question such as "Why did the robot arm break down?" is generated. In the generation process, the accident information is used as input data, and the generative AI model analyzes it and outputs a prompt sentence.

[0655] Step 4:

[0656] The server sends the generated "why" question to the device, which then presents it to the user. The user then responds to the displayed question. An example response might include specific details such as "It's due to lack of maintenance." This interaction provides the basis for subsequent analysis.

[0657] Step 5:

[0658] The device sends the user's answer to the server, which then uses the generative AI model to generate a more detailed "why" question, such as "Why was maintenance lacking?" This allows the server to investigate the cause at a deeper level.

[0659] Step 6:

[0660] The server then sends the generated detailed "why" question to the device, which then presents it to the user. The user then re-enters the answer, and the process is repeated. For example, the answer could be "there was no maintenance plan in place." This repetition helps pinpoint the true cause of the accident.

[0661] Step 7:

[0662] After collecting all the questions and answers, the server analyzes them to identify the root cause of the final accident. This analysis uses all the questions and answers stored in the database. The server generates a final report based on the root cause identified.

[0663] Step 8:

[0664] The server then sends the final report to the terminal, where the user can review it. The report includes details of the accident, its causes, and remedial measures, enabling efficient and objective identification of the cause of the accident and the formulation of countermeasures.

[0665] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0666] This invention is a system for objectively identifying and analyzing the cause of accidents that occur during facility construction. In particular, this system incorporates an emotion engine that recognizes the user's emotions and realizes interactions that correspond to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user. Specific embodiments and the processing flow are described below.

[0667] User input of accident information

[0668] The user first inputs accident information through a dedicated interface. The input accident information includes the date, time, location, summary, and information on the people involved. The emotion engine analyzes the voice and input content as the user inputs and recognizes the user's emotional state.

[0669] Accident information registration and initial analysis

[0670] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[0671] Generating initial "why" questions

[0672] The server uses a generative AI to generate the first "why" question based on the stored accident information. Based on the analysis results of the emotion engine, the server adjusts the number and difficulty of questions if the user is confused, for example. For example, it automatically generates specific questions such as "Why did the equipment fall?"

[0673] Asking questions and collecting answers

[0674] The server sends the generated questions to the device, which displays them to the user. The emotion engine continuously monitors the user's emotional state and flexibly adjusts the way questions are presented if the user is feeling stressed. The user then inputs answers to the questions. An example answer might be, "There was a problem with the way the equipment was handled."

[0675] Generate further questions based on answers

[0676] The device sends the user's answer to the server, which again uses generative AI to generate further "why" questions based on the previous answer. The emotion engine takes the user's emotional state into account and re-adjusts the question and how it is presented. For example, a question might be generated such as, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[0677] Identifying the root cause and generating a final report

[0678] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and the emotion engine adjusts the explanation taking into account the user's level of understanding. Finally, the server sends this report to the terminal, which displays it to the user.

[0679] Specific examples

[0680] Case 1:

[0681] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0682] The server uses a generative AI to generate the question, "Why did the equipment fall?" The question "Why?" is softened by the emotion engine: "Did something go wrong?"

[0683] If the user answers "there was a problem with the way it was handled," the emotion engine will detect the user's stress level and adjust the next question to be shorter.

[0684] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0685] The user responded that "the cause is a lack of training on work procedures."

[0686] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then presents the user with a final report with an easy-to-understand explanation.

[0687] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[0688] The processing flow will be explained below.

[0689] Processing steps of an accident cause analysis system that combines an emotion engine

[0690] Step 1:

[0691] The user opens the accident information entry screen. The interface is visually easy to understand and provides fields for entering details such as the date, time, location, and circumstances of the accident.

[0692] Step 2:

[0693] The user inputs the incident information, and the emotion engine recognizes emotions from the user's input and voice to assess the level of stress and confusion.

[0694] Step 3:

[0695] The terminal transmits the input accident information and the emotion data evaluated by the emotion engine to the server.

[0696] Step 4:

[0697] The server stores the received accident information and emotion data in a database.

[0698] Step 5:

[0699] The server activates the AI ​​generator, which generates the first "why" question based on the stored accident information. It takes into account emotional data and adjusts the question accordingly, such as shortening the question if the user is nervous.

[0700] Step 6:

[0701] The server generates a first "why" question and sends it to the terminal as a question list.

[0702] Step 7:

[0703] The device displays a list of questions to the user, adjusting them based on emotional data and presenting them in a format that is easy for the user to understand.

[0704] Step 8:

[0705] The user inputs an answer to the question presented, for example, "There was a problem with the way the equipment was handled."

[0706] Step 9:

[0707] The terminal sends the user's answer to the server.

[0708] Step 10:

[0709] The server stores the received response in a database.

[0710] Step 11:

[0711] The server then reactivates the generative AI to generate further "why" questions based on the previous answers, while the emotion engine continues to monitor the user's emotional state and adjusts the complexity and format of the questions accordingly.

[0712] Step 12:

[0713] The server generates a new list of questions and sends it to the terminal.

[0714] Step 13:

[0715] The device presents the user with a new list of questions, the order and format of which are adjusted based on the user's emotional state.

[0716] Step 14:

[0717] The user again inputs an answer to the question. For example, the user answers, "It is due to a lack of training on work procedures."

[0718] Step 15:

[0719] The device sends the user's answer to the server, and this procedure is repeated until the root cause is identified.

[0720] Step 16:

[0721] The server analyzes all questions and answers to identify the root cause of the accident. The emotion engine adjusts the results during the final analysis to make them easier for users to understand.

[0722] Step 17:

[0723] The server creates a report based on the identified root cause, using a document format that takes emotion data into account.

[0724] Step 18:

[0725] The server sends the final report to the terminal.

[0726] Step 19:

[0727] The terminal displays the final report to the user, which includes details of the accident, the identified root causes, and recommended actions.

[0728] Specific examples

[0729] Case 1:

[0730] A user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The emotion engine evaluates the user's stress level as "high."

[0731] The server generates the question "Why did the equipment fall?", but the emotion engine changes this to a softer expression such as "Please tell us what caused the equipment to fall."

[0732] The user responded that "there was a problem with the way it was handled." The emotion engine detected that the user was still feeling stressed.

[0733] The server generates the next "why" question: "Why was there a problem with the handling method?" and presents it in a concise and easy-to-understand manner.

[0734] The user responded that "it was due to a lack of training on work procedures." The emotion engine detected that the user's stress had been alleviated.

[0735] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then creates a final report in an easy-to-understand document format and presents it to the user.

[0736] In this way, the system automates the process of efficiently and objectively identifying the root cause of an accident, while also enabling interaction that responds to the user's emotional state.

[0737] Example 2

[0738] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0739] When an accident occurs during facility construction, a process is needed to objectively and efficiently identify and analyze the cause. However, conventional systems are prone to human bias and lack interaction that takes into account the user's emotional state, which can cause users to feel stressed and reduce the quality of answers. Another issue is that the process of entering accident information, generating questions, and analyzing them is complex and time-consuming.

[0740] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0741] In this invention, the server includes means for providing a user interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating a more detailed "why" question based on the collected answers, means for presenting the more detailed "why" question, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for recognizing the user's emotional state and adjusting the wording of the question, and means for adjusting the method of presenting the question in consideration of the user's emotional state. This makes it possible to efficiently and objectively identify the true cause of the accident and realize interaction that takes the user's emotional state into consideration.

[0742] "Accident Information"

[0743] This refers to the date, time, location, summary, and information on the people involved in accidents during facility construction work.

[0744] User Interface

[0745] This refers to the screen or input means through which the user can input accident information.

[0746] Database

[0747] refers to an information storage device that stores received accident information and is used for later analysis and question generation.

[0748] "Generation means"

[0749] refers to methods and techniques for generating "why" questions based on the accident information.

[0750] "Method of presenting questions"

[0751] refers to the means by which the server displays generated questions to the user.

[0752] "Method of collecting responses"

[0753] refers to methods and techniques for collecting answers to questions entered by users.

[0754] A way to identify the root cause

[0755] refers to the methods and techniques used to analyze all questions and answers and identify the root cause of an accident.

[0756] "Report format"

[0757] This refers to the format and layout used to present identified root causes and analysis results in an easy-to-understand manner for users.

[0758] "A means of recognizing emotional states"

[0759] refers to technologies and methods for analyzing a user's voice or text to detect their emotional state.

[0760] "A way to adjust the wording of a question"

[0761] refers to techniques and methods for modifying the content and wording of questions based on the detected emotional state of the user.

[0762] "A way to tailor how questions are presented"

[0763] refers to techniques and methods for taking into account the user's emotional state and modifying the way questions are presented.

[0764] This invention is a system for objectively identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates an emotion engine that recognizes the user's emotions, enabling interaction that corresponds to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[0765] User input of accident information

[0766] Users first enter accident information through a dedicated interface. This interface is provided by an application installed on devices such as PCs and smartphones. The entered accident information includes the date, time, location, summary, and information on the people involved in the accident. For example, the information entered might be, "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[0767] The device collects this information and uses an emotion engine to analyze the user's voice and text to recognize their emotional state. The emotion engine uses voice recognition and natural language processing (NLP) to detect when the user is stressed or confused.

[0768] Accident information registration and initial analysis

[0769] The terminal transmits the accident information received from the user to the server. This is done in real time via the network. The server stores the received accident information in a database. This database is used for later analysis and query generation.

[0770] Generating initial "why" questions

[0771] The server uses a generative AI model to generate the first "why" question based on the collected accident information. The generative AI model uses an advanced natural language generation model such as GPT-4. For example, a specific question such as "Why did the equipment fall?" is generated.

[0772] The emotion engine takes into account the user's emotional state and adjusts the number and difficulty of questions. If the user is confused, it may soften the wording of the questions. For example, it may change the question "Why?" to "Did something go wrong?"

[0773] Asking questions and collecting answers

[0774] The server sends the generated questions to the device, which then displays them to the user. The display method is adjusted according to the user's emotional state. For example, it may be a visual display or a voice prompt. The user then inputs an answer to the question. For example, the user may reply, "There was a problem with the way it was handled."

[0775] Generate further questions based on answers

[0776] The device sends the user's answer to the server, which again uses the generative AI model to generate the next "why" question. The emotion engine again analyzes the user's emotional state and readjusts the question and how it is presented. For example, it generates a question like, "Why was there a problem with the handling?" This process is repeated until the true cause of the accident is identified.

[0777] Identifying the root cause and generating a final report

[0778] The server analyzes all questions and answers to identify the true cause of the accident. The analysis results are generated in the form of a report, and an emotion engine adjusts the explanation based on the user's level of understanding. The final report is sent from the server to the terminal and displayed to the user.

[0779] Prompt Sentence Examples

[0780] "Analyze why a piece of equipment fell on a construction site on October 5, 2023, injuring a worker. Use an emotion engine to adjust the difficulty and presentation of questions according to the user's stress level, and generate a final report."

[0781] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[0782] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0783] Step 1:

[0784] The user inputs the accident information.

[0785] The input items are the date and time of the accident, location, summary, and information on the people involved.

[0786] For example, enter "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[0787] The user interface receives the user's voice and text input and converts it into digital information using speech recognition and natural language processing.

[0788] Step 2:

[0789] The terminal transmits the input accident information to the server.

[0790] At this time, the transmitted data is structured in JSON format or similar.

[0791] The server parses the received data and stores it in a database.

[0792] For example, an SQL query is used to insert data into an "accident information" table.

[0793] Step 3:

[0794] The server generates the first "why" question using a generative AI model.

[0795] The input is stored accident information, and a question such as "Why did the equipment fall?" is input as a prompt to the generative AI model (e.g., GPT-4).

[0796] The output is the generated "why" question.

[0797] Step 4:

[0798] An emotional engine takes into account the user's emotional state to adjust the number and difficulty of questions.

[0799] The input is data on the user's emotional state, including the results of speech and text analysis.

[0800] For example, if you determine that the person is confused, change the question "Why?" to a softer phrase such as "Did something go wrong?"

[0801] Step 5:

[0802] The server sends the tailored question to the terminal.

[0803] The data sent is restructured and contains the question data in JSON format.

[0804] The terminal receives this and displays it to the user on the user interface.

[0805] Step 6:

[0806] The user enters answers to the questions presented.

[0807] For example, you might answer, "There was a problem with the way it was handled."

[0808] The answers entered are in voice or text format, which the terminal collects and converts into digital data.

[0809] Step 7:

[0810] The terminal sends the user's answer to the server.

[0811] The data sent is again structured in JSON format or similar.

[0812] The server parses the received data and uses it to generate the next "why" question.

[0813] Step 8:

[0814] The server again uses the generative AI model to generate the next "why" question.

[0815] The input is the user's previous answer, and a question is generated, for example, "Why was there a problem with the handling method?"

[0816] The output is the following "why" question:

[0817] Step 9:

[0818] The emotion engine again takes into account the user's emotional state and readjusts the content and presentation of the next question.

[0819] For example, if the user is feeling stressed, shorten the questions.

[0820] Step 10:

[0821] The server sends the retuned question to the terminal, which displays it to the user.

[0822] This process is repeated until the true cause of the accident is identified.

[0823] Step 11:

[0824] The server analyzes all questions and answers to identify the true cause of the accident.

[0825] The input is all question and answer data, which is analyzed using natural language processing techniques.

[0826] For example, identify it as being caused by "lack of training on how to handle it."

[0827] Step 12:

[0828] The server generates the final report.

[0829] The input is the analysis results, and the emotion engine adjusts the report description to match the user's level of understanding.

[0830] The generated report is saved in PDF format.

[0831] Step 13:

[0832] The server sends the final report to the terminal, which displays it to the user.

[0833] Users can view the reports through the application.

[0834] (Application example 2)

[0835] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0836] In conventional accident response systems, questions to identify the true cause of an accident are asked mechanically, and the user's emotional state cannot be taken into consideration. As a result, stressed users are unable to answer the questions accurately, making it difficult to identify the true cause. In addition, it is difficult to reflect emotions and the specific situation at the scene when inputting and analyzing accident information, so improvements in efficiency and accuracy are required.

[0837] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0838] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" question, means for analyzing all the questions and answers to identify the true cause of the accident, means for presenting the identified true cause in the form of a report, emotion analysis means for recognizing the emotions of workers, adjustment means for adjusting the question method and question content based on the emotion analysis means, and data collection means for collecting video and audio of the accident site. This makes it possible to efficiently and accurately identify the true cause of the accident while taking emotions and circumstances into consideration.

[0839] "Accident information" is data including the date, time, location, and specific circumstances of the accident.

[0840] An "interface" is a means by which a user inputs accident information.

[0841] "Database" refers to a system for storing received accident information.

[0842] The "generation means" is a means for generating a question based on the accident information.

[0843] The "presentation means" is a means for presenting the generated question to the user.

[0844] The "collection means" is a means by which users input answers and collect the answers.

[0845] The "emotion analysis means" is a means for recognizing and analyzing the emotions of workers.

[0846] The "adjustment means" is a means for adjusting the method and content of questions based on the results of the emotion analysis means.

[0847] "Data collection means" refers to means for collecting video and audio data from the accident scene.

[0848] "Generative AI" is a technology that uses artificial intelligence to generate questions based on accident information.

[0849] A "report" is a document that presents the identified root causes of an accident.

[0850] This invention is a system for identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates emotion analysis means that recognizes the user's emotions, enabling interaction according to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[0851] Hardware and software used

[0852] Hardware:

[0853] Robots operating in a factory

[0854] Microphones, cameras, and high-precision sensors

[0855] software:

[0856] Server (example databases are MySQL or PostgreSQL)

[0857] Generative AI (e.g., GPT-4)

[0858] Sentiment analysis tools (e.g., a BERT model using Transformers on Hugging Face)

[0859] System configuration and operation

[0860] Entering accident information

[0861] First, the user inputs accident information through a dedicated interface. This input includes the date, time, location, and summary of the accident. Audio and video data collected by the robot are also added to the accident information. The emotion analysis means recognizes the emotional state indicated by the user through their input and voice.

[0862] Accident information registration and initial analysis

[0863] The device sends the accident information entered by the user to the server. The server stores the received accident information in a database. This database is used for subsequent analysis and question generation. The generative AI generates an initial "why" question and adjusts the wording of the question based on the results of sentiment analysis.

[0864] Asking questions and collecting answers

[0865] The server sends the generated questions to the terminal, which displays them to the user. The emotion analysis means monitors the user's emotional state and adjusts the wording and length of the questions if the user is feeling stressed. The user enters answers to the questions into the interface. The answers are sent to the server and stored in a database.

[0866] Generate further questions based on answers

[0867] The server uses generative AI to generate additional "why" questions based on the user's answers, again adjusting the wording and content of the questions based on the user's emotional state. This process is repeated until the true cause of the accident is identified.

[0868] Identifying the root cause and generating a final report

[0869] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and a sentiment analysis tool adjusts the explanation to take into account the user's level of understanding. The final report is sent to the terminal and presented to the user.

[0870] Specific examples

[0871] Prompt statement

[0872] Previous answer: There was a problem with the way it was handled. Generate the following "why" questions:

[0873] Specific scenarios

[0874] The user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The server uses a generation AI to generate the question, "Why did the equipment fall?" Based on the analysis results of the sentiment analysis means, this question is changed to a softer expression, "Did something go wrong?" If the user answers, "There was a problem with the handling," the sentiment analysis means detects the user's stress level and adjusts the next question to be shorter. The server generates the next "why" question, "Why was there a problem with the handling?" If the user answers, "It was caused by insufficient training on work procedures," the server analyzes the answer and identifies the true cause of the accident as "insufficient training on work procedures." The sentiment analysis means then presents the user with a final report that has been adjusted to an easy-to-understand explanation.

[0875] This makes it possible to efficiently and accurately identify the true cause of an accident while taking into account the user's emotional state.

[0876] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0877] Step 1:

[0878] Users input accident information through a dedicated interface. The input accident information includes the date, time, location, and summary of the accident. Audio and video data are also collected during input. The input from this interface becomes the initial input data for the system.

[0879] Step 2:

[0880] The collected accident information is sent to the server by the terminal. The server stores the received information in a database. At this time, the voice input data is converted to text, and the video data is also recorded as accident information.

[0881] Step 3:

[0882] The server uses a generative AI to generate a first "why" question based on the stored accident information. For example, it generates a specific question such as "Why did the equipment fall?" The generative AI analyzes the input data and generates a question using a prompt sentence.

[0883] Step 4:

[0884] The server sends the generated question to the terminal, which then displays the question to the user in a voice or text format that is easy for the user to answer.

[0885] Step 5:

[0886] The user inputs answers to the questions. This input answer is also captured in the system in the form of voice or text. The emotion analysis means analyzes this input and recognizes the user's emotional state.

[0887] Step 6:

[0888] The device sends the user's response data to the server, which then stores the received response data in a database, and also stores the emotional commentary data along with the analysis results.

[0889] Step 7:

[0890] The server uses generative AI to generate further "why" questions based on the previous answer, and adjusts the content and wording of the generated questions based on the results of sentiment analysis. The questions are then sent back to the device.

[0891] Step 8:

[0892] The question is again presented to the user from the terminal. The question is adjusted to allow the user to enter a short or concise answer. The user again enters the answer to the question.

[0893] Step 9:

[0894] The device repeatedly sends the user's input to the server, which again uses generative AI to generate the next "why" question, and this process is repeated until the true cause of the accident is identified.

[0895] Step 10:

[0896] The server analyzes all questions and answers and ultimately identifies the true cause of the accident. The analysis results are generated in the form of a report, and an emotion analysis means adjusts the explanation according to the user's level of understanding.

[0897] Step 11:

[0898] The final report is sent to the terminal, which then presents the report to the user, who reviews the final report and receives information about the true cause of the accident.

[0899] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0900] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0901] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0902] [Third embodiment]

[0903] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0904] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0905] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0906] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0907] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0908] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0909] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0910] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0911] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0912] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0913] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0914] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0915] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. Specific embodiments and the processing flow of this system are described below.

[0916] User input of accident information

[0917] First, the user enters accident information through a dedicated interface. The entered accident information includes the date and time of the accident, location, a summary of the accident, and information on the people involved. This information is used as the base data for subsequent analysis.

[0918] Accident information registration and initial analysis

[0919] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[0920] Generating initial "why" questions

[0921] The server uses generative AI to generate the first "why" question based on the stored accident information. For example, it automatically generates a specific question such as "Why did the equipment fall?"

[0922] Asking questions and collecting answers

[0923] The server sends the generated questions to the terminal, which displays them to the user. The user then inputs an answer to the question. An example of an answer would be something like, "There was a problem with the way the equipment was handled."

[0924] Generate further questions based on answers

[0925] The device sends the user-entered answers to the server, which again uses generative AI to generate further "why" questions based on the previous answers. For example, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[0926] Identifying the root cause and generating a final report

[0927] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, which includes safety measures and improvement methods to be applied. Finally, the server sends this report to the terminal, which displays it to the user.

[0928] Specific examples

[0929] Case 1:

[0930] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[0931] The server uses a generative AI to generate a question such as "Why did the equipment fall?"

[0932] The user responded that "there was a problem with the way it was handled."

[0933] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[0934] The user responded that "the cause is a lack of training on work procedures."

[0935] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." It then presents the final report to the user.

[0936] In this way, the system can automate the process of efficiently and objectively identifying the root causes of accidents, eliminating human bias.

[0937] The processing flow will be explained below.

[0938] Step 1:

[0939] The user opens a dedicated interface and enters accident information, including the date, time, location, summary, and information about the people involved.

[0940] Step 2:

[0941] The terminal receives the user's input and sends it to the server.

[0942] Step 3:

[0943] The server stores the received accident information in a database.

[0944] Step 4:

[0945] The server launches a generation AI to generate the first "why" question based on the stored accident information, such as "Why did the equipment fall?"

[0946] Step 5:

[0947] The server generates a list of first "why" questions and sends them to the terminal.

[0948] Step 6:

[0949] The terminal displays a list of questions to the user.

[0950] Step 7:

[0951] The user inputs an answer to the displayed question, for example, "There was a problem with the way it was handled."

[0952] Step 8:

[0953] The terminal sends the user's answer to the server.

[0954] Step 9:

[0955] The server stores the received response in a database.

[0956] Step 10:

[0957] The server then launches the generation AI again, generating further "why" questions based on the previous answers, such as "Why was there a problem with the handling method?"

[0958] Step 11:

[0959] The server generates a list of further questions and sends them to the terminal.

[0960] Step 12:

[0961] The terminal displays the new list of questions to the user.

[0962] Step 13:

[0963] The user again inputs an answer to the displayed question. For example, the user answers, "It is due to insufficient training on work procedures."

[0964] Step 14:

[0965] The terminal sends the user's answer to the server.

[0966] Step 15:

[0967] The server stores the received answer in a database. This process is repeated until the root cause is identified.

[0968] Step 16:

[0969] The server analyzes all questions and answers to identify the true cause of the accident.

[0970] Step 17:

[0971] The server will create a report based on the identified root cause, which will include details of the incident, the identified root cause, and recommended countermeasures.

[0972] Step 18:

[0973] The server sends the final report to the terminal.

[0974] Step 19:

[0975] The terminal displays the final report to the user.

[0976] Through this cycle, the system automates the process of efficiently and objectively identifying the root cause of an accident.

[0977] Example 1

[0978] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0979] There is a growing need for reliable accident analysis systems, but conventional methods have issues with human bias and analytical efficiency. In particular, it is currently difficult to quickly and objectively identify the true cause of an accident. In addition, there are many manual processes involved in generating questions and collecting answers, so efficient automation is required.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0981] In this invention, the server includes a means for receiving accident information and storing it in a database, a means for generating questions using a generative AI model, and a means for managing input prompts to the generative AI model, which enables the storage and analysis of accident information, automatic generation of questions, and rapid identification of the root cause.

[0982] "Accident information" refers to detailed data about an accident, such as the date and time of the accident, the location, a summary of the accident, and information about the people involved.

[0983] An "interface" refers to the screens and functions that allow users to input information into a system, and is usually accessed through a web browser or dedicated application.

[0984] A "server" is a computer system used to receive information from users and store it in a database.

[0985] A "database" is a system for structuring and storing received accident information, and a relational database is usually used.

[0986] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on input data and generates new questions and answers.

[0987] A "prompt sentence" is an input sentence that gives a specific question or instruction to a generative AI model.

[0988] "Question generation" is the process of automatically generating specific "why" questions using a generative AI model based on accident information.

[0989] "Answer collection" is the process of collecting answers entered by users to questions posed and storing them in a database.

[0990] "Root cause identification" is the process of analyzing all collected questions and answers to find the underlying cause of the accident.

[0991] The "report format" is a document format that summarizes the true cause of the accident, the analysis results based on that, and improvement measures.

[0992] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. A specific embodiment of this system is described below.

[0993] User input of accident information

[0994] Users can enter accident information using a dedicated interface, which can be provided via a web browser or a dedicated application, and can enter information about the date and time of the accident, the location, a summary of the accident, and the people involved.

[0995] Accident information registration and initial analysis

[0996] The device sends the accident information entered by the user to the server, which then stores the received accident information in a relational database (e.g., MySQL or PostgreSQL). This structured the accident information and makes it available for later analysis and query generation.

[0997] Generating initial "why" questions

[0998] The server uses a generative AI model (e.g., GPT-4) to generate an initial "why" question based on the stored accident information. The generative AI model receives an input prompt and outputs an appropriate question. This generation process automatically generates a specific question.

[0999] Asking questions and collecting answers

[1000] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the displayed question, and this answer is sent to the server via the terminal. The server stores the received answer in the database again.

[1001] Generate further questions based on answers

[1002] Based on the collected answers, the server re-enters the prompt sentence into the generative AI model to generate a new "why" question, which then automatically generates the next question based on the user's answer. This process is repeated until the root cause is identified.

[1003] Identifying the root cause and generating a final report

[1004] The server analyzes all questions and answers to identify the root cause of the accident. The generated report includes the root cause of the accident, the analysis results based on the root cause, and the safety measures and improvement methods to be applied. The server then sends the final report to the terminal, which displays it to the user.

[1005] Specific examples

[1006] Case 1:

[1007] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[1008] The server uses generative AI to generate a question such as "Why did the equipment fall?"

[1009] The user answers, "There was a problem with the way it was handled."

[1010] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[1011] The user answers, "It is due to insufficient training on work procedures."

[1012] The server analyzes the responses, identifies the true cause of the accident as "insufficient training on work procedures," and generates a final report, which is displayed to the user via their terminal.

[1013] This system uses a generative AI model to automate the question generation process, efficiently identifying the true cause of an accident while eliminating human bias. By sending the prompt text "Accident summary: Equipment fell, injuring a worker. Why did the equipment fall?" to the generative AI model, the system can generate appropriate questions.

[1014] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1015] Step 1:

[1016] The user inputs accident information using a dedicated interface. This interface is provided via a web browser or a dedicated application. The input accident information includes the date and time of the accident, the location, a summary of the accident, and information on the people involved. Once input is complete, the device sends this information to the server.

[1017] Input: Accident information (date, time, location, summary, information of people involved)

[1018] Output: Accident information sent to the server

[1019] Step 2:

[1020] The server stores the received accident information in a relational database, such as MySQL or PostgreSQL. The information stored in the database is used for subsequent analysis and query generation.

[1021] Input: Accident information sent from the device

[1022] Output: Accident information stored in the database

[1023] Specific operation: The server issues an INSERT query to the database to save the incident information.

[1024] Step 3:

[1025] The server uses a generative AI model to generate an initial "why" question based on the stored accident information. This process uses a generative AI (e.g., GPT-4). The server generates a prompt sentence containing a summary of the accident and sends it to the AI ​​model to output an appropriate question.

[1026] Input: Accident information stored in the database

[1027] Output: Generated "why" questions

[1028] Specific operation: Send a prompt (e.g., "Accident summary: Equipment fell and injured a worker. Why did the equipment fall?") ​​to the generation AI and get a question.

[1029] Step 4:

[1030] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the question, and the terminal sends the answer to the server.

[1031] Input: Generated Question

[1032] Output: User's answer

[1033] Specific operation: The device displays a question on the user interface, and the user enters the answer in the text box and submits it.

[1034] Step 5:

[1035] The server stores the answers received from the user in a database again. Then, it generates new "why" questions based on the generative AI model. The server inputs the collected answers as prompts into the generative AI model to generate further questions.

[1036] Input: User's answer

[1037] Output: New "why" questions

[1038] Specific operation: The server saves the answer in the database by issuing an INSERT query, and sends a new prompt to the generation AI to get the next question.

[1039] Step 6:

[1040] The server repeats this process until the root cause is identified. It analyzes all questions and answers to identify the root cause of the accident. The final analysis results are generated in the form of a report and displayed to the user via the terminal.

[1041] Input: All questions and answers

[1042] Output: The true cause of the accident and a report based on it

[1043] Specific operation: The server analyzes the accumulated data using an analysis algorithm and generates a report. The generated report is sent to the terminal and displayed to the user.

[1044] (Application example 1)

[1045] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1046] When an accident occurs in equipment construction or industrial facilities, there is a need to quickly and objectively identify and analyze the cause of the accident. However, current manual analysis methods are inefficient and prone to human bias. Accurately recording the accident situation and quickly analyzing it are particularly difficult when it comes to accidents involving robots. To address this issue, there is an urgent need to provide an automated system.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1048] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" questions, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for automatically transmitting accident information when the robot detects an accident, means for the server to perform an initial analysis of the accident information in real time and generate questions using a generation AI, and means for the robot administrator to answer the questions and then generate new questions based on the answers. This enables accurate recording and rapid analysis of accident information.

[1049] The "interface for inputting accident information" is a component of the system that allows the user to input information such as the date and time of the accident, the location, and a summary of the accident.

[1050] The "means for receiving the accident information and storing it in a database" is a function for taking the accident information entered by the user into the system and storing it in a database.

[1051] The "generation means for generating a first 'why' question based on the accident information" is a function that automatically generates the first 'why' question using a generation AI based on the input accident information.

[1052] The "means for presenting the generated first 'why' question" is a function for displaying the generated question to the user.

[1053] The "means for collecting answers to the first 'why' question" is a function that allows a user to input an answer to a 'why' question and collect the answer within the system.

[1054] The "generation means for generating more detailed 'why' questions based on the collected answers" is a function that uses a generation AI to create more detailed 'why' questions based on the user's answers.

[1055] The "means for presenting the more detailed 'why' question" is a function for displaying a more detailed question to the user.

[1056] The "means for analyzing all of the questions and answers to identify the true cause of the accident" is a function for comprehensively analyzing all of the questions and answers to identify the root cause of the accident.

[1057] The "means for presenting the identified true cause in the form of a report" is a function for creating the identified true cause of the accident in the form of a report and presenting it to the user.

[1058] The "means for automatically transmitting accident information when the robot detects an accident" is a function for automatically transmitting information about an accident to a server when the robot detects an accident.

[1059] "Means for the server to perform initial analysis of accident information in real time and generate questions using generation AI" refers to a function that allows the server to immediately perform initial analysis of the accident information received and generate "why" questions using generation AI based on that information.

[1060] "Means for the robot administrator to answer questions and generate new questions based on those answers" is a system function that allows the robot administrator to answer questions presented to them and generate new "why" questions based on those answers.

[1061] As an application example of the present invention, we will show how to specifically implement an accident cause analysis system for a factory robot. The system is configured as follows.

[1062] System configuration

[1063] 1. Interface for entering accident information

[1064] The factory robot's control terminal is provided with an interface for inputting accident information. The user enters the date and time of the accident, the location, a summary of the accident, information about the people involved, etc. This interface is operated using a touch panel display and keyboard.

[1065] 2. A means of storing accident information in a database

[1066] Accident information entered on the terminal is sent to the server, which stores the information in a database, allowing for efficient management of accident information.

[1067] 3. Generating "why" questions

[1068] The server analyzes the accumulated accident information and uses a generative AI model to generate the first "why" question. For example, a specific question such as "Why did the equipment fall?" is automatically generated based on the input accident information.

[1069] 4. How to pose questions and collect answers

[1070] The generated questions are displayed on the management terminal, and the user inputs answers to the questions. The terminal has a function to collect the answers, and the information is sent to the server.

[1071] 5. Generators for generating detailed "why" questions

[1072] The server uses the user's answers to generate more detailed "why" questions. This process is carried out using generative AI, and a question such as "Why was there a problem with the handling method?" is generated.

[1073] 6. Rephrasing the question

[1074] The generated new questions are then displayed on the terminal again, and the user enters their answers. This process continues until the true cause of the accident is identified.

[1075] 7. Identifying the root cause and presenting it in a report format

[1076] The server analyzes all questions and answers to identify the root cause of the accident. Finally, the analysis results are generated in the form of a report and presented to the terminal. This report contains the cause of the accident, all related information, and remedial measures.

[1077] Hardware and software used

[1078] Hardware: Robots in factories, devices for inputting accident information (tablets, PCs, etc.), servers

[1079] Software: Python, Requests library, OpenAI API

[1080] Data processing and calculation

[1081] After receiving the accident information, the server stores it in a database. Based on the stored data, a generative AI model is used to generate "why" questions. The answers collected from users are analyzed again, and more detailed questions are generated using the generative AI model. By repeating this process, the true cause of the accident can be identified.

[1082] Specific examples

[1083] For example, a general outline of an accident in a factory might be entered, such as "A robot arm broke down, injuring a worker." Based on this information, the server uses generative AI to generate the first question: "Why did the robot arm break down?" If the user answers, "It was caused by lack of maintenance," the server generates a more detailed question, such as, "Why was there lack of maintenance?" In this way, the true cause of the failure is identified and compiled into a report.

[1084] Prompt Sentence Examples

[1085] Accident summary: The robot arm malfunctioned, injuring a worker.

[1086] Generate initial "why" questions to analyze the cause of this accident.

[1087] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1088] Step 1:

[1089] The user uses a terminal to input accident information. The input includes the date, time, location, summary, and information on the people involved. This information is imported into the system as the initial accident data. An example of input data would be "October 5, 2023, factory line 1, a robot arm malfunctioned, injuring a worker." This provides the system with basic data for analysis.

[1090] Step 2:

[1091] The device sends the entered accident information to the server, which receives the information and stores it in a database. This creates an environment in which accident information can be continuously managed within the system.

[1092] Step 3:

[1093] The server uses a generative AI model to generate a first "why" question based on the stored accident information. For example, a question such as "Why did the robot arm break down?" is generated. In the generation process, the accident information is used as input data, and the generative AI model analyzes it and outputs a prompt sentence.

[1094] Step 4:

[1095] The server sends the generated "why" question to the device, which then presents it to the user. The user then responds to the displayed question. An example response might include specific details such as "It's due to lack of maintenance." This interaction provides the basis for subsequent analysis.

[1096] Step 5:

[1097] The device sends the user's answer to the server, which then uses the generative AI model to generate a more detailed "why" question, such as "Why was maintenance lacking?" This allows the server to investigate the cause at a deeper level.

[1098] Step 6:

[1099] The server then sends the generated detailed "why" question to the device, which then presents it to the user. The user then re-enters the answer, and the process is repeated. For example, the answer could be "there was no maintenance plan in place." This repetition helps pinpoint the true cause of the accident.

[1100] Step 7:

[1101] After collecting all the questions and answers, the server analyzes them to identify the root cause of the final accident. This analysis uses all the questions and answers stored in the database. The server generates a final report based on the root cause identified.

[1102] Step 8:

[1103] The server then sends the final report to the terminal, where the user can review it. The report includes details of the accident, its causes, and remedial measures, enabling efficient and objective identification of the cause of the accident and the formulation of countermeasures.

[1104] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1105] This invention is a system for objectively identifying and analyzing the cause of accidents that occur during facility construction. In particular, this system incorporates an emotion engine that recognizes the user's emotions and realizes interactions that correspond to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user. Specific embodiments and the processing flow are described below.

[1106] User input of accident information

[1107] The user first inputs accident information through a dedicated interface. The input accident information includes the date, time, location, summary, and information on the people involved. The emotion engine analyzes the voice and input content as the user inputs and recognizes the user's emotional state.

[1108] Accident information registration and initial analysis

[1109] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[1110] Generating initial "why" questions

[1111] The server uses a generative AI to generate the first "why" question based on the stored accident information. Based on the analysis results of the emotion engine, the server adjusts the number and difficulty of questions if the user is confused, for example. For example, it automatically generates specific questions such as "Why did the equipment fall?"

[1112] Asking questions and collecting answers

[1113] The server sends the generated questions to the device, which displays them to the user. The emotion engine continuously monitors the user's emotional state and flexibly adjusts the way questions are presented if the user is feeling stressed. The user then inputs answers to the questions. An example answer might be, "There was a problem with the way the equipment was handled."

[1114] Generate further questions based on answers

[1115] The device sends the user's answer to the server, which again uses generative AI to generate further "why" questions based on the previous answer. The emotion engine takes the user's emotional state into account and re-adjusts the question and how it is presented. For example, a question might be generated such as, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[1116] Identifying the root cause and generating a final report

[1117] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and the emotion engine adjusts the explanation taking into account the user's level of understanding. Finally, the server sends this report to the terminal, which displays it to the user.

[1118] Specific examples

[1119] Case 1:

[1120] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[1121] The server uses a generative AI to generate the question, "Why did the equipment fall?" The question "Why?" is softened by the emotion engine: "Did something go wrong?"

[1122] If the user answers "there was a problem with the way it was handled," the emotion engine will detect the user's stress level and adjust the next question to be shorter.

[1123] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[1124] The user responded that "the cause is a lack of training on work procedures."

[1125] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then presents the user with a final report with an easy-to-understand explanation.

[1126] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[1127] The processing flow will be explained below.

[1128] Processing steps of an accident cause analysis system that combines an emotion engine

[1129] Step 1:

[1130] The user opens the accident information entry screen. The interface is visually easy to understand and provides fields for entering details such as the date, time, location, and circumstances of the accident.

[1131] Step 2:

[1132] The user inputs the incident information, and the emotion engine recognizes emotions from the user's input and voice to assess the level of stress and confusion.

[1133] Step 3:

[1134] The terminal transmits the input accident information and the emotion data evaluated by the emotion engine to the server.

[1135] Step 4:

[1136] The server stores the received accident information and emotion data in a database.

[1137] Step 5:

[1138] The server activates the AI ​​generator, which generates the first "why" question based on the stored accident information. It takes into account emotional data and adjusts the question accordingly, such as shortening the question if the user is nervous.

[1139] Step 6:

[1140] The server generates a first "why" question and sends it to the terminal as a question list.

[1141] Step 7:

[1142] The device displays a list of questions to the user, adjusting them based on emotional data and presenting them in a format that is easy for the user to understand.

[1143] Step 8:

[1144] The user inputs an answer to the question presented, for example, "There was a problem with the way the equipment was handled."

[1145] Step 9:

[1146] The terminal sends the user's answer to the server.

[1147] Step 10:

[1148] The server stores the received response in a database.

[1149] Step 11:

[1150] The server then reactivates the generative AI to generate further "why" questions based on the previous answers, while the emotion engine continues to monitor the user's emotional state and adjusts the complexity and format of the questions accordingly.

[1151] Step 12:

[1152] The server generates a new list of questions and sends it to the terminal.

[1153] Step 13:

[1154] The device presents the user with a new list of questions, the order and format of which are adjusted based on the user's emotional state.

[1155] Step 14:

[1156] The user again inputs an answer to the question. For example, the user answers, "It is due to a lack of training on work procedures."

[1157] Step 15:

[1158] The device sends the user's answer to the server, and this procedure is repeated until the root cause is identified.

[1159] Step 16:

[1160] The server analyzes all questions and answers to identify the root cause of the accident. The emotion engine adjusts the results during the final analysis to make them easier for users to understand.

[1161] Step 17:

[1162] The server creates a report based on the identified root cause, using a document format that takes emotion data into account.

[1163] Step 18:

[1164] The server sends the final report to the terminal.

[1165] Step 19:

[1166] The terminal displays the final report to the user, which includes details of the accident, the identified root causes, and recommended actions.

[1167] Specific examples

[1168] Case 1:

[1169] A user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The emotion engine evaluates the user's stress level as "high."

[1170] The server generates the question "Why did the equipment fall?", but the emotion engine changes this to a softer expression such as "Please tell us what caused the equipment to fall."

[1171] The user responded that "there was a problem with the way it was handled." The emotion engine detected that the user was still feeling stressed.

[1172] The server generates the next "why" question: "Why was there a problem with the handling method?" and presents it in a concise and easy-to-understand manner.

[1173] The user responded that "it was due to a lack of training on work procedures." The emotion engine detected that the user's stress had been alleviated.

[1174] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then creates a final report in an easy-to-understand document format and presents it to the user.

[1175] In this way, the system automates the process of efficiently and objectively identifying the root cause of an accident, while also enabling interaction that responds to the user's emotional state.

[1176] Example 2

[1177] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1178] When an accident occurs during facility construction, a process is needed to objectively and efficiently identify and analyze the cause. However, conventional systems are prone to human bias and lack interaction that takes into account the user's emotional state, which can cause users to feel stressed and reduce the quality of answers. Another issue is that the process of entering accident information, generating questions, and analyzing them is complex and time-consuming.

[1179] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1180] In this invention, the server includes means for providing a user interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating a more detailed "why" question based on the collected answers, means for presenting the more detailed "why" question, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for recognizing the user's emotional state and adjusting the wording of the question, and means for adjusting the method of presenting the question in consideration of the user's emotional state. This makes it possible to efficiently and objectively identify the true cause of the accident and realize interaction that takes the user's emotional state into consideration.

[1181] "Accident Information"

[1182] This refers to the date, time, location, summary, and information on the people involved in accidents during facility construction work.

[1183] User Interface

[1184] This refers to the screen or input means through which the user can input accident information.

[1185] Database

[1186] refers to an information storage device that stores received accident information and is used for later analysis and question generation.

[1187] "Generation means"

[1188] refers to methods and techniques for generating "why" questions based on the accident information.

[1189] "Method of presenting questions"

[1190] refers to the means by which the server displays generated questions to the user.

[1191] "Method of collecting responses"

[1192] refers to methods and techniques for collecting answers to questions entered by users.

[1193] A way to identify the root cause

[1194] refers to the methods and techniques used to analyze all questions and answers and identify the root cause of an accident.

[1195] "Report format"

[1196] This refers to the format and layout used to present identified root causes and analysis results in an easy-to-understand manner for users.

[1197] "A means of recognizing emotional states"

[1198] refers to technologies and methods for analyzing a user's voice or text to detect their emotional state.

[1199] "A way to adjust the wording of a question"

[1200] refers to techniques and methods for modifying the content and wording of questions based on the detected emotional state of the user.

[1201] "A way to tailor how questions are presented"

[1202] refers to techniques and methods for taking into account the user's emotional state and modifying the way questions are presented.

[1203] This invention is a system for objectively identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates an emotion engine that recognizes the user's emotions, enabling interaction that corresponds to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[1204] User input of accident information

[1205] Users first enter accident information through a dedicated interface. This interface is provided by an application installed on devices such as PCs and smartphones. The entered accident information includes the date, time, location, summary, and information on the people involved in the accident. For example, the information entered might be, "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[1206] The device collects this information and uses an emotion engine to analyze the user's voice and text to recognize their emotional state. The emotion engine uses voice recognition and natural language processing (NLP) to detect when the user is stressed or confused.

[1207] Accident information registration and initial analysis

[1208] The terminal transmits the accident information received from the user to the server. This is done in real time via the network. The server stores the received accident information in a database. This database is used for later analysis and query generation.

[1209] Generating initial "why" questions

[1210] The server uses a generative AI model to generate the first "why" question based on the collected accident information. The generative AI model uses an advanced natural language generation model such as GPT-4. For example, a specific question such as "Why did the equipment fall?" is generated.

[1211] The emotion engine takes into account the user's emotional state and adjusts the number and difficulty of questions. If the user is confused, it may soften the wording of the questions. For example, it may change the question "Why?" to "Did something go wrong?"

[1212] Asking questions and collecting answers

[1213] The server sends the generated questions to the device, which then displays them to the user. The display method is adjusted according to the user's emotional state. For example, it may be a visual display or a voice prompt. The user then inputs an answer to the question. For example, the user may reply, "There was a problem with the way it was handled."

[1214] Generate further questions based on answers

[1215] The device sends the user's answer to the server, which again uses the generative AI model to generate the next "why" question. The emotion engine again analyzes the user's emotional state and readjusts the question and how it is presented. For example, it generates a question like, "Why was there a problem with the handling?" This process is repeated until the true cause of the accident is identified.

[1216] Identifying the root cause and generating a final report

[1217] The server analyzes all questions and answers to identify the true cause of the accident. The analysis results are generated in the form of a report, and an emotion engine adjusts the explanation based on the user's level of understanding. The final report is sent from the server to the terminal and displayed to the user.

[1218] Prompt Sentence Examples

[1219] "Analyze why a piece of equipment fell on a construction site on October 5, 2023, injuring a worker. Use an emotion engine to adjust the difficulty and presentation of questions according to the user's stress level, and generate a final report."

[1220] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[1221] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1222] Step 1:

[1223] The user inputs the accident information.

[1224] The input items are the date and time of the accident, location, summary, and information on the people involved.

[1225] For example, enter "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[1226] The user interface receives the user's voice and text input and converts it into digital information using speech recognition and natural language processing.

[1227] Step 2:

[1228] The terminal transmits the input accident information to the server.

[1229] At this time, the transmitted data is structured in JSON format or similar.

[1230] The server parses the received data and stores it in a database.

[1231] For example, an SQL query is used to insert data into an "accident information" table.

[1232] Step 3:

[1233] The server generates the first "why" question using a generative AI model.

[1234] The input is stored accident information, and a question such as "Why did the equipment fall?" is input as a prompt to the generative AI model (e.g., GPT-4).

[1235] The output is the generated "why" question.

[1236] Step 4:

[1237] An emotional engine takes into account the user's emotional state to adjust the number and difficulty of questions.

[1238] The input is data on the user's emotional state, including the results of speech and text analysis.

[1239] For example, if you determine that the person is confused, change the question "Why?" to a softer phrase such as "Did something go wrong?"

[1240] Step 5:

[1241] The server sends the tailored question to the terminal.

[1242] The data sent is restructured and contains the question data in JSON format.

[1243] The terminal receives this and displays it to the user on the user interface.

[1244] Step 6:

[1245] The user enters answers to the questions presented.

[1246] For example, you might answer, "There was a problem with the way it was handled."

[1247] The answers entered are in voice or text format, which the terminal collects and converts into digital data.

[1248] Step 7:

[1249] The terminal sends the user's answer to the server.

[1250] The data sent is again structured in JSON format or similar.

[1251] The server parses the received data and uses it to generate the next "why" question.

[1252] Step 8:

[1253] The server again uses the generative AI model to generate the next "why" question.

[1254] The input is the user's previous answer, and a question is generated, for example, "Why was there a problem with the handling method?"

[1255] The output is the following "why" question:

[1256] Step 9:

[1257] The emotion engine again takes into account the user's emotional state and readjusts the content and presentation of the next question.

[1258] For example, if the user is feeling stressed, shorten the questions.

[1259] Step 10:

[1260] The server sends the retuned question to the terminal, which displays it to the user.

[1261] This process is repeated until the true cause of the accident is identified.

[1262] Step 11:

[1263] The server analyzes all questions and answers to identify the true cause of the accident.

[1264] The input is all question and answer data, which is analyzed using natural language processing techniques.

[1265] For example, identify it as being caused by "lack of training on how to handle it."

[1266] Step 12:

[1267] The server generates the final report.

[1268] The input is the analysis results, and the emotion engine adjusts the report description to match the user's level of understanding.

[1269] The generated report is saved in PDF format.

[1270] Step 13:

[1271] The server sends the final report to the terminal, which displays it to the user.

[1272] Users can view the reports through the application.

[1273] (Application example 2)

[1274] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1275] In conventional accident response systems, questions to identify the true cause of an accident are asked mechanically, and the user's emotional state cannot be taken into consideration. As a result, stressed users are unable to answer the questions accurately, making it difficult to identify the true cause. In addition, it is difficult to reflect emotions and the specific situation at the scene when inputting and analyzing accident information, so improvements in efficiency and accuracy are required.

[1276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1277] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" question, means for analyzing all the questions and answers to identify the true cause of the accident, means for presenting the identified true cause in the form of a report, emotion analysis means for recognizing the emotions of workers, adjustment means for adjusting the question method and question content based on the emotion analysis means, and data collection means for collecting video and audio of the accident site. This makes it possible to efficiently and accurately identify the true cause of the accident while taking emotions and circumstances into consideration.

[1278] "Accident information" is data including the date, time, location, and specific circumstances of the accident.

[1279] An "interface" is a means by which a user inputs accident information.

[1280] "Database" refers to a system for storing received accident information.

[1281] The "generation means" is a means for generating a question based on the accident information.

[1282] The "presentation means" is a means for presenting the generated question to the user.

[1283] The "collection means" is a means by which users input answers and collect the answers.

[1284] The "emotion analysis means" is a means for recognizing and analyzing the emotions of workers.

[1285] The "adjustment means" is a means for adjusting the method and content of questions based on the results of the emotion analysis means.

[1286] "Data collection means" refers to means for collecting video and audio data from the accident scene.

[1287] "Generative AI" is a technology that uses artificial intelligence to generate questions based on accident information.

[1288] A "report" is a document that presents the identified root causes of an accident.

[1289] This invention is a system for identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates emotion analysis means that recognizes the user's emotions, enabling interaction according to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[1290] Hardware and software used

[1291] Hardware:

[1292] Robots operating in a factory

[1293] Microphones, cameras, and high-precision sensors

[1294] software:

[1295] Server (example databases are MySQL or PostgreSQL)

[1296] Generative AI (e.g., GPT-4)

[1297] Sentiment analysis tools (e.g., a BERT model using Transformers on Hugging Face)

[1298] System configuration and operation

[1299] Entering accident information

[1300] First, the user inputs accident information through a dedicated interface. This input includes the date, time, location, and summary of the accident. Audio and video data collected by the robot are also added to the accident information. The emotion analysis means recognizes the emotional state indicated by the user through their input and voice.

[1301] Accident information registration and initial analysis

[1302] The device sends the accident information entered by the user to the server. The server stores the received accident information in a database. This database is used for subsequent analysis and question generation. The generative AI generates an initial "why" question and adjusts the wording of the question based on the results of sentiment analysis.

[1303] Asking questions and collecting answers

[1304] The server sends the generated questions to the terminal, which displays them to the user. The emotion analysis means monitors the user's emotional state and adjusts the wording and length of the questions if the user is feeling stressed. The user enters answers to the questions into the interface. The answers are sent to the server and stored in a database.

[1305] Generate further questions based on answers

[1306] The server uses generative AI to generate additional "why" questions based on the user's answers, again adjusting the wording and content of the questions based on the user's emotional state. This process is repeated until the true cause of the accident is identified.

[1307] Identifying the root cause and generating a final report

[1308] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and a sentiment analysis tool adjusts the explanation to take into account the user's level of understanding. The final report is sent to the terminal and presented to the user.

[1309] Specific examples

[1310] Prompt statement

[1311] Previous answer: There was a problem with the way it was handled. Generate the following "why" questions:

[1312] Specific scenarios

[1313] The user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The server uses a generation AI to generate the question, "Why did the equipment fall?" Based on the analysis results of the sentiment analysis means, this question is changed to a softer expression, "Did something go wrong?" If the user answers, "There was a problem with the handling," the sentiment analysis means detects the user's stress level and adjusts the next question to be shorter. The server generates the next "why" question, "Why was there a problem with the handling?" If the user answers, "It was caused by insufficient training on work procedures," the server analyzes the answer and identifies the true cause of the accident as "insufficient training on work procedures." The sentiment analysis means then presents the user with a final report that has been adjusted to an easy-to-understand explanation.

[1314] This makes it possible to efficiently and accurately identify the true cause of an accident while taking into account the user's emotional state.

[1315] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1316] Step 1:

[1317] Users input accident information through a dedicated interface. The input accident information includes the date, time, location, and summary of the accident. Audio and video data are also collected during input. The input from this interface becomes the initial input data for the system.

[1318] Step 2:

[1319] The collected accident information is sent to the server by the terminal. The server stores the received information in a database. At this time, the voice input data is converted to text, and the video data is also recorded as accident information.

[1320] Step 3:

[1321] The server uses a generative AI to generate a first "why" question based on the stored accident information. For example, it generates a specific question such as "Why did the equipment fall?" The generative AI analyzes the input data and generates a question using a prompt sentence.

[1322] Step 4:

[1323] The server sends the generated question to the terminal, which then displays the question to the user in a voice or text format that is easy for the user to answer.

[1324] Step 5:

[1325] The user inputs answers to the questions. This input answer is also captured in the system in the form of voice or text. The emotion analysis means analyzes this input and recognizes the user's emotional state.

[1326] Step 6:

[1327] The device sends the user's response data to the server, which then stores the received response data in a database, and also stores the emotional commentary data along with the analysis results.

[1328] Step 7:

[1329] The server uses generative AI to generate further "why" questions based on the previous answer, and adjusts the content and wording of the generated questions based on the results of sentiment analysis. The questions are then sent back to the device.

[1330] Step 8:

[1331] The question is again presented to the user from the terminal. The question is adjusted to allow the user to enter a short or concise answer. The user again enters the answer to the question.

[1332] Step 9:

[1333] The device repeatedly sends the user's input to the server, which again uses generative AI to generate the next "why" question, and this process is repeated until the true cause of the accident is identified.

[1334] Step 10:

[1335] The server analyzes all questions and answers and ultimately identifies the true cause of the accident. The analysis results are generated in the form of a report, and an emotion analysis means adjusts the explanation according to the user's level of understanding.

[1336] Step 11:

[1337] The final report is sent to the terminal, which then presents the report to the user, who reviews the final report and receives information about the true cause of the accident.

[1338] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1339] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1340] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1341] [Fourth embodiment]

[1342] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1343] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1344] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1345] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1346] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1347] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1348] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1349] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1350] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1351] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1352] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1353] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1354] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. Specific embodiments and the processing flow of this system are described below.

[1356] User input of accident information

[1357] First, the user enters accident information through a dedicated interface. The entered accident information includes the date and time of the accident, location, a summary of the accident, and information on the people involved. This information is used as the base data for subsequent analysis.

[1358] Accident information registration and initial analysis

[1359] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[1360] Generating initial "why" questions

[1361] The server uses generative AI to generate the first "why" question based on the stored accident information. For example, it automatically generates a specific question such as "Why did the equipment fall?"

[1362] Asking questions and collecting answers

[1363] The server sends the generated questions to the terminal, which displays them to the user. The user then inputs an answer to the question. An example of an answer would be something like, "There was a problem with the way the equipment was handled."

[1364] Generate further questions based on answers

[1365] The device sends the user-entered answers to the server, which again uses generative AI to generate further "why" questions based on the previous answers. For example, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[1366] Identifying the root cause and generating a final report

[1367] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, which includes safety measures and improvement methods to be applied. Finally, the server sends this report to the terminal, which displays it to the user.

[1368] Specific examples

[1369] Case 1:

[1370] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[1371] The server uses a generative AI to generate a question such as "Why did the equipment fall?"

[1372] The user responded that "there was a problem with the way it was handled."

[1373] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[1374] The user responded that "the cause is a lack of training on work procedures."

[1375] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." It then presents the final report to the user.

[1376] In this way, the system can automate the process of efficiently and objectively identifying the root causes of accidents, eliminating human bias.

[1377] The processing flow will be explained below.

[1378] Step 1:

[1379] The user opens a dedicated interface and enters accident information, including the date, time, location, summary, and information about the people involved.

[1380] Step 2:

[1381] The terminal receives the user's input and sends it to the server.

[1382] Step 3:

[1383] The server stores the received accident information in a database.

[1384] Step 4:

[1385] The server launches a generation AI to generate the first "why" question based on the stored accident information, such as "Why did the equipment fall?"

[1386] Step 5:

[1387] The server generates a list of first "why" questions and sends them to the terminal.

[1388] Step 6:

[1389] The terminal displays a list of questions to the user.

[1390] Step 7:

[1391] The user inputs an answer to the displayed question, for example, "There was a problem with the way it was handled."

[1392] Step 8:

[1393] The terminal sends the user's answer to the server.

[1394] Step 9:

[1395] The server stores the received response in a database.

[1396] Step 10:

[1397] The server then launches the generation AI again, generating further "why" questions based on the previous answers, such as "Why was there a problem with the handling method?"

[1398] Step 11:

[1399] The server generates a list of further questions and sends them to the terminal.

[1400] Step 12:

[1401] The terminal displays the new list of questions to the user.

[1402] Step 13:

[1403] The user again inputs an answer to the displayed question. For example, the user answers, "It is due to insufficient training on work procedures."

[1404] Step 14:

[1405] The terminal sends the user's answer to the server.

[1406] Step 15:

[1407] The server stores the received answer in a database. This process is repeated until the root cause is identified.

[1408] Step 16:

[1409] The server analyzes all questions and answers to identify the true cause of the accident.

[1410] Step 17:

[1411] The server will create a report based on the identified root cause, which will include details of the incident, the identified root cause, and recommended countermeasures.

[1412] Step 18:

[1413] The server sends the final report to the terminal.

[1414] Step 19:

[1415] The terminal displays the final report to the user.

[1416] Through this cycle, the system automates the process of efficiently and objectively identifying the root cause of an accident.

[1417] Example 1

[1418] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1419] There is a growing need for reliable accident analysis systems, but conventional methods have issues with human bias and analytical efficiency. In particular, it is currently difficult to quickly and objectively identify the true cause of an accident. In addition, there are many manual processes involved in generating questions and collecting answers, so efficient automation is required.

[1420] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1421] In this invention, the server includes a means for receiving accident information and storing it in a database, a means for generating questions using a generative AI model, and a means for managing input prompts to the generative AI model, which enables the storage and analysis of accident information, automatic generation of questions, and rapid identification of the root cause.

[1422] "Accident information" refers to detailed data about an accident, such as the date and time of the accident, the location, a summary of the accident, and information about the people involved.

[1423] An "interface" refers to the screens and functions that allow users to input information into a system, and is usually accessed through a web browser or dedicated application.

[1424] A "server" is a computer system used to receive information from users and store it in a database.

[1425] A "database" is a system for structuring and storing received accident information, and a relational database is usually used.

[1426] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on input data and generates new questions and answers.

[1427] A "prompt sentence" is an input sentence that gives a specific question or instruction to a generative AI model.

[1428] "Question generation" is the process of automatically generating specific "why" questions using a generative AI model based on accident information.

[1429] "Answer collection" is the process of collecting answers entered by users to questions posed and storing them in a database.

[1430] "Root cause identification" is the process of analyzing all collected questions and answers to find the underlying cause of the accident.

[1431] The "report format" is a document format that summarizes the true cause of the accident, the analysis results based on that, and improvement measures.

[1432] This invention is a system for objectively identifying and analyzing the cause of an accident that occurs during facility construction. This system is designed to operate in cooperation with three parties: a server, a terminal, and a user. A specific embodiment of this system is described below.

[1433] User input of accident information

[1434] Users can enter accident information using a dedicated interface, which can be provided via a web browser or a dedicated application, and can enter information about the date and time of the accident, the location, a summary of the accident, and the people involved.

[1435] Accident information registration and initial analysis

[1436] The device sends the accident information entered by the user to the server, which then stores the received accident information in a relational database (e.g., MySQL or PostgreSQL). This structured the accident information and makes it available for later analysis and query generation.

[1437] Generating initial "why" questions

[1438] The server uses a generative AI model (e.g., GPT-4) to generate an initial "why" question based on the stored accident information. The generative AI model receives an input prompt and outputs an appropriate question. This generation process automatically generates a specific question.

[1439] Asking questions and collecting answers

[1440] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the displayed question, and this answer is sent to the server via the terminal. The server stores the received answer in the database again.

[1441] Generate further questions based on answers

[1442] Based on the collected answers, the server re-enters the prompt sentence into the generative AI model to generate a new "why" question, which then automatically generates the next question based on the user's answer. This process is repeated until the root cause is identified.

[1443] Identifying the root cause and generating a final report

[1444] The server analyzes all questions and answers to identify the root cause of the accident. The generated report includes the root cause of the accident, the analysis results based on the root cause, and the safety measures and improvement methods to be applied. The server then sends the final report to the terminal, which displays it to the user.

[1445] Specific examples

[1446] Case 1:

[1447] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[1448] The server uses generative AI to generate a question such as "Why did the equipment fall?"

[1449] The user answers, "There was a problem with the way it was handled."

[1450] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[1451] The user answers, "It is due to insufficient training on work procedures."

[1452] The server analyzes the responses, identifies the true cause of the accident as "insufficient training on work procedures," and generates a final report, which is displayed to the user via their terminal.

[1453] This system uses a generative AI model to automate the question generation process, efficiently identifying the true cause of an accident while eliminating human bias. By sending the prompt text "Accident summary: Equipment fell, injuring a worker. Why did the equipment fall?" to the generative AI model, the system can generate appropriate questions.

[1454] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1455] Step 1:

[1456] The user inputs accident information using a dedicated interface. This interface is provided via a web browser or a dedicated application. The input accident information includes the date and time of the accident, the location, a summary of the accident, and information on the people involved. Once input is complete, the device sends this information to the server.

[1457] Input: Accident information (date, time, location, summary, information of people involved)

[1458] Output: Accident information sent to the server

[1459] Step 2:

[1460] The server stores the received accident information in a relational database, such as MySQL or PostgreSQL. The information stored in the database is used for subsequent analysis and query generation.

[1461] Input: Accident information sent from the device

[1462] Output: Accident information stored in the database

[1463] Specific operation: The server issues an INSERT query to the database to save the incident information.

[1464] Step 3:

[1465] The server uses a generative AI model to generate an initial "why" question based on the stored accident information. This process uses a generative AI (e.g., GPT-4). The server generates a prompt sentence containing a summary of the accident and sends it to the AI ​​model to output an appropriate question.

[1466] Input: Accident information stored in the database

[1467] Output: Generated "why" questions

[1468] Specific operation: Send a prompt (e.g., "Accident summary: Equipment fell and injured a worker. Why did the equipment fall?") ​​to the generation AI and get a question.

[1469] Step 4:

[1470] The server sends the generated question to the terminal, which displays it to the user. The user enters an answer to the question, and the terminal sends the answer to the server.

[1471] Input: Generated Question

[1472] Output: User's answer

[1473] Specific operation: The device displays a question on the user interface, and the user enters the answer in the text box and submits it.

[1474] Step 5:

[1475] The server stores the answers received from the user in a database again. Then, it generates new "why" questions based on the generative AI model. The server inputs the collected answers as prompts into the generative AI model to generate further questions.

[1476] Input: User's answer

[1477] Output: New "why" questions

[1478] Specific operation: The server saves the answer in the database by issuing an INSERT query, and sends a new prompt to the generation AI to get the next question.

[1479] Step 6:

[1480] The server repeats this process until the root cause is identified. It analyzes all questions and answers to identify the root cause of the accident. The final analysis results are generated in the form of a report and displayed to the user via the terminal.

[1481] Input: All questions and answers

[1482] Output: The true cause of the accident and a report based on it

[1483] Specific operation: The server analyzes the accumulated data using an analysis algorithm and generates a report. The generated report is sent to the terminal and displayed to the user.

[1484] (Application example 1)

[1485] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1486] When an accident occurs in equipment construction or industrial facilities, there is a need to quickly and objectively identify and analyze the cause of the accident. However, current manual analysis methods are inefficient and prone to human bias. Accurately recording the accident situation and quickly analyzing it are particularly difficult when it comes to accidents involving robots. To address this issue, there is an urgent need to provide an automated system.

[1487] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1488] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" questions, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for automatically transmitting accident information when the robot detects an accident, means for the server to perform an initial analysis of the accident information in real time and generate questions using a generation AI, and means for the robot administrator to answer the questions and then generate new questions based on the answers. This enables accurate recording and rapid analysis of accident information.

[1489] The "interface for inputting accident information" is a component of the system that allows the user to input information such as the date and time of the accident, the location, and a summary of the accident.

[1490] The "means for receiving the accident information and storing it in a database" is a function for taking the accident information entered by the user into the system and storing it in a database.

[1491] The "generation means for generating a first 'why' question based on the accident information" is a function that automatically generates the first 'why' question using a generation AI based on the input accident information.

[1492] The "means for presenting the generated first 'why' question" is a function for displaying the generated question to the user.

[1493] The "means for collecting answers to the first 'why' question" is a function that allows a user to input an answer to a 'why' question and collect the answer within the system.

[1494] The "generation means for generating more detailed 'why' questions based on the collected answers" is a function that uses a generation AI to create more detailed 'why' questions based on the user's answers.

[1495] The "means for presenting the more detailed 'why' question" is a function for displaying a more detailed question to the user.

[1496] The "means for analyzing all of the questions and answers to identify the true cause of the accident" is a function for comprehensively analyzing all of the questions and answers to identify the root cause of the accident.

[1497] The "means for presenting the identified true cause in the form of a report" is a function for creating the identified true cause of the accident in the form of a report and presenting it to the user.

[1498] The "means for automatically transmitting accident information when the robot detects an accident" is a function for automatically transmitting information about an accident to a server when the robot detects an accident.

[1499] "Means for the server to perform initial analysis of accident information in real time and generate questions using generation AI" refers to a function that allows the server to immediately perform initial analysis of the accident information received and generate "why" questions using generation AI based on that information.

[1500] "Means for the robot administrator to answer questions and generate new questions based on those answers" is a system function that allows the robot administrator to answer questions presented to them and generate new "why" questions based on those answers.

[1501] As an application example of the present invention, we will show how to specifically implement an accident cause analysis system for a factory robot. The system is configured as follows.

[1502] System configuration

[1503] 1. Interface for entering accident information

[1504] The factory robot's control terminal is provided with an interface for inputting accident information. The user enters the date and time of the accident, the location, a summary of the accident, information about the people involved, etc. This interface is operated using a touch panel display and keyboard.

[1505] 2. A means of storing accident information in a database

[1506] Accident information entered on the terminal is sent to the server, which stores the information in a database, allowing for efficient management of accident information.

[1507] 3. Generating "why" questions

[1508] The server analyzes the accumulated accident information and uses a generative AI model to generate the first "why" question. For example, a specific question such as "Why did the equipment fall?" is automatically generated based on the input accident information.

[1509] 4. How to pose questions and collect answers

[1510] The generated questions are displayed on the management terminal, and the user inputs answers to the questions. The terminal has a function to collect the answers, and the information is sent to the server.

[1511] 5. Generators for generating detailed "why" questions

[1512] The server uses the user's answers to generate more detailed "why" questions. This process is carried out using generative AI, and a question such as "Why was there a problem with the handling method?" is generated.

[1513] 6. Rephrasing the question

[1514] The generated new questions are then displayed on the terminal again, and the user enters their answers. This process continues until the true cause of the accident is identified.

[1515] 7. Identifying the root cause and presenting it in a report format

[1516] The server analyzes all questions and answers to identify the root cause of the accident. Finally, the analysis results are generated in the form of a report and presented to the terminal. This report contains the cause of the accident, all related information, and remedial measures.

[1517] Hardware and software used

[1518] Hardware: Robots in factories, devices for inputting accident information (tablets, PCs, etc.), servers

[1519] Software: Python, Requests library, OpenAI API

[1520] Data processing and calculation

[1521] After receiving the accident information, the server stores it in a database. Based on the stored data, a generative AI model is used to generate "why" questions. The answers collected from users are analyzed again, and more detailed questions are generated using the generative AI model. By repeating this process, the true cause of the accident can be identified.

[1522] Specific examples

[1523] For example, a general outline of an accident in a factory might be entered, such as "A robot arm broke down, injuring a worker." Based on this information, the server uses generative AI to generate the first question: "Why did the robot arm break down?" If the user answers, "It was caused by lack of maintenance," the server generates a more detailed question, such as, "Why was there lack of maintenance?" In this way, the true cause of the failure is identified and compiled into a report.

[1524] Prompt Sentence Examples

[1525] Accident summary: The robot arm malfunctioned, injuring a worker.

[1526] Generate initial "why" questions to analyze the cause of this accident.

[1527] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1528] Step 1:

[1529] The user uses a terminal to input accident information. The input includes the date, time, location, summary, and information on the people involved. This information is imported into the system as the initial accident data. An example of input data would be "October 5, 2023, factory line 1, a robot arm malfunctioned, injuring a worker." This provides the system with basic data for analysis.

[1530] Step 2:

[1531] The device sends the entered accident information to the server, which receives the information and stores it in a database. This creates an environment in which accident information can be continuously managed within the system.

[1532] Step 3:

[1533] The server uses a generative AI model to generate a first "why" question based on the stored accident information. For example, a question such as "Why did the robot arm break down?" is generated. In the generation process, the accident information is used as input data, and the generative AI model analyzes it and outputs a prompt sentence.

[1534] Step 4:

[1535] The server sends the generated "why" question to the device, which then presents it to the user. The user then responds to the displayed question. An example response might include specific details such as "It's due to lack of maintenance." This interaction provides the basis for subsequent analysis.

[1536] Step 5:

[1537] The device sends the user's answer to the server, which then uses the generative AI model to generate a more detailed "why" question, such as "Why was maintenance lacking?" This allows the server to investigate the cause at a deeper level.

[1538] Step 6:

[1539] The server then sends the generated detailed "why" question to the device, which then presents it to the user. The user then re-enters the answer, and the process is repeated. For example, the answer could be "there was no maintenance plan in place." This repetition helps pinpoint the true cause of the accident.

[1540] Step 7:

[1541] After collecting all the questions and answers, the server analyzes them to identify the root cause of the final accident. This analysis uses all the questions and answers stored in the database. The server generates a final report based on the root cause identified.

[1542] Step 8:

[1543] The server then sends the final report to the terminal, where the user can review it. The report includes details of the accident, its causes, and remedial measures, enabling efficient and objective identification of the cause of the accident and the formulation of countermeasures.

[1544] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1545] This invention is a system for objectively identifying and analyzing the cause of accidents that occur during facility construction. In particular, this system incorporates an emotion engine that recognizes the user's emotions and realizes interactions that correspond to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user. Specific embodiments and the processing flow are described below.

[1546] User input of accident information

[1547] The user first inputs accident information through a dedicated interface. The input accident information includes the date, time, location, summary, and information on the people involved. The emotion engine analyzes the voice and input content as the user inputs and recognizes the user's emotional state.

[1548] Accident information registration and initial analysis

[1549] The device sends the accident information entered by the user to the server, which stores the received accident information in a database that is used for subsequent analysis and query generation.

[1550] Generating initial "why" questions

[1551] The server uses a generative AI to generate the first "why" question based on the stored accident information. Based on the analysis results of the emotion engine, the server adjusts the number and difficulty of questions if the user is confused, for example. For example, it automatically generates specific questions such as "Why did the equipment fall?"

[1552] Asking questions and collecting answers

[1553] The server sends the generated questions to the device, which displays them to the user. The emotion engine continuously monitors the user's emotional state and flexibly adjusts the way questions are presented if the user is feeling stressed. The user then inputs answers to the questions. An example answer might be, "There was a problem with the way the equipment was handled."

[1554] Generate further questions based on answers

[1555] The device sends the user's answer to the server, which again uses generative AI to generate further "why" questions based on the previous answer. The emotion engine takes the user's emotional state into account and re-adjusts the question and how it is presented. For example, a question might be generated such as, "Why was there a problem with the handling?" This process is repeated until the root cause is identified.

[1556] Identifying the root cause and generating a final report

[1557] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and the emotion engine adjusts the explanation taking into account the user's level of understanding. Finally, the server sends this report to the terminal, which displays it to the user.

[1558] Specific examples

[1559] Case 1:

[1560] A user types, "On October 5, 2023, equipment fell on a construction site, injuring a worker."

[1561] The server uses a generative AI to generate the question, "Why did the equipment fall?" The question "Why?" is softened by the emotion engine: "Did something go wrong?"

[1562] If the user answers "there was a problem with the way it was handled," the emotion engine will detect the user's stress level and adjust the next question to be shorter.

[1563] The server generates the next "why" question: "Why was there a problem with the way it was handled?"

[1564] The user responded that "the cause is a lack of training on work procedures."

[1565] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then presents the user with a final report with an easy-to-understand explanation.

[1566] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[1567] The processing flow will be explained below.

[1568] Processing steps of an accident cause analysis system that combines an emotion engine

[1569] Step 1:

[1570] The user opens the accident information entry screen. The interface is visually easy to understand and provides fields for entering details such as the date, time, location, and circumstances of the accident.

[1571] Step 2:

[1572] The user inputs the incident information, and the emotion engine recognizes emotions from the user's input and voice to assess the level of stress and confusion.

[1573] Step 3:

[1574] The terminal transmits the input accident information and the emotion data evaluated by the emotion engine to the server.

[1575] Step 4:

[1576] The server stores the received accident information and emotion data in a database.

[1577] Step 5:

[1578] The server activates the AI ​​generator, which generates the first "why" question based on the stored accident information. It takes into account emotional data and adjusts the question accordingly, such as shortening the question if the user is nervous.

[1579] Step 6:

[1580] The server generates a first "why" question and sends it to the terminal as a question list.

[1581] Step 7:

[1582] The device displays a list of questions to the user, adjusting them based on emotional data and presenting them in a format that is easy for the user to understand.

[1583] Step 8:

[1584] The user inputs an answer to the question presented, for example, "There was a problem with the way the equipment was handled."

[1585] Step 9:

[1586] The terminal sends the user's answer to the server.

[1587] Step 10:

[1588] The server stores the received response in a database.

[1589] Step 11:

[1590] The server then reactivates the generative AI to generate further "why" questions based on the previous answers, while the emotion engine continues to monitor the user's emotional state and adjusts the complexity and format of the questions accordingly.

[1591] Step 12:

[1592] The server generates a new list of questions and sends it to the terminal.

[1593] Step 13:

[1594] The device presents the user with a new list of questions, the order and format of which are adjusted based on the user's emotional state.

[1595] Step 14:

[1596] The user again inputs an answer to the question. For example, the user answers, "It is due to a lack of training on work procedures."

[1597] Step 15:

[1598] The device sends the user's answer to the server, and this procedure is repeated until the root cause is identified.

[1599] Step 16:

[1600] The server analyzes all questions and answers to identify the root cause of the accident. The emotion engine adjusts the results during the final analysis to make them easier for users to understand.

[1601] Step 17:

[1602] The server creates a report based on the identified root cause, using a document format that takes emotion data into account.

[1603] Step 18:

[1604] The server sends the final report to the terminal.

[1605] Step 19:

[1606] The terminal displays the final report to the user, which includes details of the accident, the identified root causes, and recommended actions.

[1607] Specific examples

[1608] Case 1:

[1609] A user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The emotion engine evaluates the user's stress level as "high."

[1610] The server generates the question "Why did the equipment fall?", but the emotion engine changes this to a softer expression such as "Please tell us what caused the equipment to fall."

[1611] The user responded that "there was a problem with the way it was handled." The emotion engine detected that the user was still feeling stressed.

[1612] The server generates the next "why" question: "Why was there a problem with the handling method?" and presents it in a concise and easy-to-understand manner.

[1613] The user responded that "it was due to a lack of training on work procedures." The emotion engine detected that the user's stress had been alleviated.

[1614] The server analyzes the responses and identifies the true cause of the accident as "insufficient training on work procedures." The emotion engine then creates a final report in an easy-to-understand document format and presents it to the user.

[1615] In this way, the system automates the process of efficiently and objectively identifying the root cause of an accident, while also enabling interaction that responds to the user's emotional state.

[1616] Example 2

[1617] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1618] When an accident occurs during facility construction, a process is needed to objectively and efficiently identify and analyze the cause. However, conventional systems are prone to human bias and lack interaction that takes into account the user's emotional state, which can cause users to feel stressed and reduce the quality of answers. Another issue is that the process of entering accident information, generating questions, and analyzing them is complex and time-consuming.

[1619] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1620] In this invention, the server includes means for providing a user interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating a more detailed "why" question based on the collected answers, means for presenting the more detailed "why" question, means for identifying the true cause of the accident by analyzing all the questions and answers, means for presenting the identified true cause in the form of a report, means for recognizing the user's emotional state and adjusting the wording of the question, and means for adjusting the method of presenting the question in consideration of the user's emotional state. This makes it possible to efficiently and objectively identify the true cause of the accident and realize interaction that takes the user's emotional state into consideration.

[1621] "Accident Information"

[1622] This refers to the date, time, location, summary, and information on the people involved in accidents during facility construction work.

[1623] User Interface

[1624] This refers to the screen or input means through which the user can input accident information.

[1625] Database

[1626] refers to an information storage device that stores received accident information and is used for later analysis and question generation.

[1627] "Generation means"

[1628] refers to methods and techniques for generating "why" questions based on the accident information.

[1629] "Method of presenting questions"

[1630] refers to the means by which the server displays generated questions to the user.

[1631] "Method of collecting responses"

[1632] refers to methods and techniques for collecting answers to questions entered by users.

[1633] A way to identify the root cause

[1634] refers to the methods and techniques used to analyze all questions and answers and identify the root cause of an accident.

[1635] "Report format"

[1636] This refers to the format and layout used to present identified root causes and analysis results in an easy-to-understand manner for users.

[1637] "A means of recognizing emotional states"

[1638] refers to technologies and methods for analyzing a user's voice or text to detect their emotional state.

[1639] "A way to adjust the wording of a question"

[1640] refers to techniques and methods for modifying the content and wording of questions based on the detected emotional state of the user.

[1641] "A way to tailor how questions are presented"

[1642] refers to techniques and methods for taking into account the user's emotional state and modifying the way questions are presented.

[1643] This invention is a system for objectively identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates an emotion engine that recognizes the user's emotions, enabling interaction that corresponds to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[1644] User input of accident information

[1645] Users first enter accident information through a dedicated interface. This interface is provided by an application installed on devices such as PCs and smartphones. The entered accident information includes the date, time, location, summary, and information on the people involved in the accident. For example, the information entered might be, "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[1646] The device collects this information and uses an emotion engine to analyze the user's voice and text to recognize their emotional state. The emotion engine uses voice recognition and natural language processing (NLP) to detect when the user is stressed or confused.

[1647] Accident information registration and initial analysis

[1648] The terminal transmits the accident information received from the user to the server. This is done in real time via the network. The server stores the received accident information in a database. This database is used for later analysis and query generation.

[1649] Generating initial "why" questions

[1650] The server uses a generative AI model to generate the first "why" question based on the collected accident information. The generative AI model uses an advanced natural language generation model such as GPT-4. For example, a specific question such as "Why did the equipment fall?" is generated.

[1651] The emotion engine takes into account the user's emotional state and adjusts the number and difficulty of questions. If the user is confused, it may soften the wording of the questions. For example, it may change the question "Why?" to "Did something go wrong?"

[1652] Asking questions and collecting answers

[1653] The server sends the generated questions to the device, which then displays them to the user. The display method is adjusted according to the user's emotional state. For example, it may be a visual display or a voice prompt. The user then inputs an answer to the question. For example, the user may reply, "There was a problem with the way it was handled."

[1654] Generate further questions based on answers

[1655] The device sends the user's answer to the server, which again uses the generative AI model to generate the next "why" question. The emotion engine again analyzes the user's emotional state and readjusts the question and how it is presented. For example, it generates a question like, "Why was there a problem with the handling?" This process is repeated until the true cause of the accident is identified.

[1656] Identifying the root cause and generating a final report

[1657] The server analyzes all questions and answers to identify the true cause of the accident. The analysis results are generated in the form of a report, and an emotion engine adjusts the explanation based on the user's level of understanding. The final report is sent from the server to the terminal and displayed to the user.

[1658] Prompt Sentence Examples

[1659] "Analyze why a piece of equipment fell on a construction site on October 5, 2023, injuring a worker. Use an emotion engine to adjust the difficulty and presentation of questions according to the user's stress level, and generate a final report."

[1660] In this way, the system not only automates the process of efficiently and objectively identifying the root cause of an accident, eliminating human bias, but also enables interactions that take into account the user's emotional state.

[1661] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1662] Step 1:

[1663] The user inputs the accident information.

[1664] The input items are the date and time of the accident, location, summary, and information on the people involved.

[1665] For example, enter "On October 5, 2023, equipment fell at a construction site, injuring a worker."

[1666] The user interface receives the user's voice and text input and converts it into digital information using speech recognition and natural language processing.

[1667] Step 2:

[1668] The terminal transmits the input accident information to the server.

[1669] At this time, the transmitted data is structured in JSON format or similar.

[1670] The server parses the received data and stores it in a database.

[1671] For example, an SQL query is used to insert data into an "accident information" table.

[1672] Step 3:

[1673] The server generates the first "why" question using a generative AI model.

[1674] The input is stored accident information, and a question such as "Why did the equipment fall?" is input as a prompt to the generative AI model (e.g., GPT-4).

[1675] The output is the generated "why" question.

[1676] Step 4:

[1677] An emotional engine takes into account the user's emotional state to adjust the number and difficulty of questions.

[1678] The input is data on the user's emotional state, including the results of speech and text analysis.

[1679] For example, if you determine that the person is confused, change the question "Why?" to a softer phrase such as "Did something go wrong?"

[1680] Step 5:

[1681] The server sends the tailored question to the terminal.

[1682] The data sent is restructured and contains the question data in JSON format.

[1683] The terminal receives this and displays it to the user on the user interface.

[1684] Step 6:

[1685] The user enters answers to the questions presented.

[1686] For example, you might answer, "There was a problem with the way it was handled."

[1687] The answers entered are in voice or text format, which the terminal collects and converts into digital data.

[1688] Step 7:

[1689] The terminal sends the user's answer to the server.

[1690] The data sent is again structured in JSON format or similar.

[1691] The server parses the received data and uses it to generate the next "why" question.

[1692] Step 8:

[1693] The server again uses the generative AI model to generate the next "why" question.

[1694] The input is the user's previous answer, and a question is generated, for example, "Why was there a problem with the handling method?"

[1695] The output is the following "why" question:

[1696] Step 9:

[1697] The emotion engine again takes into account the user's emotional state and readjusts the content and presentation of the next question.

[1698] For example, if the user is feeling stressed, shorten the questions.

[1699] Step 10:

[1700] The server sends the retuned question to the terminal, which displays it to the user.

[1701] This process is repeated until the true cause of the accident is identified.

[1702] Step 11:

[1703] The server analyzes all questions and answers to identify the true cause of the accident.

[1704] The input is all question and answer data, which is analyzed using natural language processing techniques.

[1705] For example, identify it as being caused by "lack of training on how to handle it."

[1706] Step 12:

[1707] The server generates the final report.

[1708] The input is the analysis results, and the emotion engine adjusts the report description to match the user's level of understanding.

[1709] The generated report is saved in PDF format.

[1710] Step 13:

[1711] The server sends the final report to the terminal, which displays it to the user.

[1712] Users can view the reports through the application.

[1713] (Application example 2)

[1714] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1715] In conventional accident response systems, questions to identify the true cause of an accident are asked mechanically, and the user's emotional state cannot be taken into consideration. As a result, stressed users are unable to answer the questions accurately, making it difficult to identify the true cause. In addition, it is difficult to reflect emotions and the specific situation at the scene when inputting and analyzing accident information, so improvements in efficiency and accuracy are required.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1717] In this invention, the server includes means for providing an interface for inputting accident information, means for receiving the accident information and storing it in a database, means for generating a first "why" question based on the accident information, means for presenting the generated first "why" question, means for collecting answers to the first "why" question, means for generating more detailed "why" questions based on the collected answers, means for presenting the more detailed "why" question, means for analyzing all the questions and answers to identify the true cause of the accident, means for presenting the identified true cause in the form of a report, emotion analysis means for recognizing the emotions of workers, adjustment means for adjusting the question method and question content based on the emotion analysis means, and data collection means for collecting video and audio of the accident site. This makes it possible to efficiently and accurately identify the true cause of the accident while taking emotions and circumstances into consideration.

[1718] "Accident information" is data including the date, time, location, and specific circumstances of the accident.

[1719] An "interface" is a means by which a user inputs accident information.

[1720] "Database" refers to a system for storing received accident information.

[1721] The "generation means" is a means for generating a question based on the accident information.

[1722] The "presentation means" is a means for presenting the generated question to the user.

[1723] The "collection means" is a means by which users input answers and collect the answers.

[1724] The "emotion analysis means" is a means for recognizing and analyzing the emotions of workers.

[1725] The "adjustment means" is a means for adjusting the method and content of questions based on the results of the emotion analysis means.

[1726] "Data collection means" refers to means for collecting video and audio data from the accident scene.

[1727] "Generative AI" is a technology that uses artificial intelligence to generate questions based on accident information.

[1728] A "report" is a document that presents the identified root causes of an accident.

[1729] This invention is a system for identifying and analyzing the causes of accidents that occur during facility construction. In particular, it incorporates emotion analysis means that recognizes the user's emotions, enabling interaction according to the user's emotional state. This system is designed to operate in cooperation with the server, terminal, and user.

[1730] Hardware and software used

[1731] Hardware:

[1732] Robots operating in a factory

[1733] Microphones, cameras, and high-precision sensors

[1734] software:

[1735] Server (example databases are MySQL or PostgreSQL)

[1736] Generative AI (e.g., GPT-4)

[1737] Sentiment analysis tools (e.g., a BERT model using Transformers on Hugging Face)

[1738] System configuration and operation

[1739] Entering accident information

[1740] First, the user inputs accident information through a dedicated interface. This input includes the date, time, location, and summary of the accident. Audio and video data collected by the robot are also added to the accident information. The emotion analysis means recognizes the emotional state indicated by the user through their input and voice.

[1741] Accident information registration and initial analysis

[1742] The device sends the accident information entered by the user to the server. The server stores the received accident information in a database. This database is used for subsequent analysis and question generation. The generative AI generates an initial "why" question and adjusts the wording of the question based on the results of sentiment analysis.

[1743] Asking questions and collecting answers

[1744] The server sends the generated questions to the terminal, which displays them to the user. The emotion analysis means monitors the user's emotional state and adjusts the wording and length of the questions if the user is feeling stressed. The user enters answers to the questions into the interface. The answers are sent to the server and stored in a database.

[1745] Generate further questions based on answers

[1746] The server uses generative AI to generate additional "why" questions based on the user's answers, again adjusting the wording and content of the questions based on the user's emotional state. This process is repeated until the true cause of the accident is identified.

[1747] Identifying the root cause and generating a final report

[1748] The server analyzes all questions and answers to identify the root cause of the accident. The analysis results are generated in the form of a report, and a sentiment analysis tool adjusts the explanation to take into account the user's level of understanding. The final report is sent to the terminal and presented to the user.

[1749] Specific examples

[1750] Prompt statement

[1751] Previous answer: There was a problem with the way it was handled. Generate the following "why" questions:

[1752] Specific scenarios

[1753] The user enters, "On October 5, 2023, equipment fell at a construction site, injuring a worker." The server uses a generation AI to generate the question, "Why did the equipment fall?" Based on the analysis results of the sentiment analysis means, this question is changed to a softer expression, "Did something go wrong?" If the user answers, "There was a problem with the handling," the sentiment analysis means detects the user's stress level and adjusts the next question to be shorter. The server generates the next "why" question, "Why was there a problem with the handling?" If the user answers, "It was caused by insufficient training on work procedures," the server analyzes the answer and identifies the true cause of the accident as "insufficient training on work procedures." The sentiment analysis means then presents the user with a final report that has been adjusted to an easy-to-understand explanation.

[1754] This makes it possible to efficiently and accurately identify the true cause of an accident while taking into account the user's emotional state.

[1755] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1756] Step 1:

[1757] Users input accident information through a dedicated interface. The input accident information includes the date, time, location, and summary of the accident. Audio and video data are also collected during input. The input from this interface becomes the initial input data for the system.

[1758] Step 2:

[1759] The collected accident information is sent to the server by the terminal. The server stores the received information in a database. At this time, the voice input data is converted to text, and the video data is also recorded as accident information.

[1760] Step 3:

[1761] The server uses a generative AI to generate a first "why" question based on the stored accident information. For example, it generates a specific question such as "Why did the equipment fall?" The generative AI analyzes the input data and generates a question using a prompt sentence.

[1762] Step 4:

[1763] The server sends the generated question to the terminal, which then displays the question to the user in a voice or text format that is easy for the user to answer.

[1764] Step 5:

[1765] The user inputs answers to the questions. This input answer is also captured in the system in the form of voice or text. The emotion analysis means analyzes this input and recognizes the user's emotional state.

[1766] Step 6:

[1767] The device sends the user's response data to the server, which then stores the received response data in a database, and also stores the emotional commentary data along with the analysis results.

[1768] Step 7:

[1769] The server uses generative AI to generate further "why" questions based on the previous answer, and adjusts the content and wording of the generated questions based on the results of sentiment analysis. The questions are then sent back to the device.

[1770] Step 8:

[1771] The question is again presented to the user from the terminal. The question is adjusted to allow the user to enter a short or concise answer. The user again enters the answer to the question.

[1772] Step 9:

[1773] The device repeatedly sends the user's input to the server, which again uses generative AI to generate the next "why" question, and this process is repeated until the true cause of the accident is identified.

[1774] Step 10:

[1775] The server analyzes all questions and answers and ultimately identifies the true cause of the accident. The analysis results are generated in the form of a report, and an emotion analysis means adjusts the explanation according to the user's level of understanding.

[1776] Step 11:

[1777] The final report is sent to the terminal, which then presents the report to the user, who reviews the final report and receives information about the true cause of the accident.

[1778] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1779] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1780] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1781] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1782] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1783] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1784] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1785] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1786] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1787] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1788] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1789] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1790] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1791] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1792] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1793] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1794] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1795] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1796] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1797] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1798] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1799] The following is further disclosed regarding the above embodiment.

[1800] (Claim 1)

[1801] means for providing an interface for inputting accident information;

[1802] means for receiving the accident information and storing it in a database;

[1803] generating means for generating a first "why" question based on the accident information;

[1804] means for presenting the generated first "why" question;

[1805] means for collecting answers to said first "why" question;

[1806] generating means for generating more detailed "why" questions based on the collected answers;

[1807] means for posing said further detailed "why" questions;

[1808] A means for analyzing all of the questions and answers to identify the true cause of the accident;

[1809] means for presenting the identified root cause in the form of a report;

[1810] A system including:

[1811] (Claim 2)

[1812] 10. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

[1813] (Claim 3)

[1814] 2. The system according to claim 1, wherein the generating means generates "why" questions using a generating AI.

[1815] "Example 1"

[1816] (Claim 1)

[1817] means for providing an interface for inputting accident information;

[1818] means for receiving the accident information and storing it in a database;

[1819] generating means for generating a first "why" question based on the accident information;

[1820] means for presenting the generated first "why" question;

[1821] means for collecting answers to said first "why" question;

[1822] generating means for generating more detailed "why" questions based on the collected answers;

[1823] means for posing said further detailed "why" questions;

[1824] A means for analyzing all of the questions and answers to identify the true cause of the accident;

[1825] means for presenting the identified root cause in the form of a report;

[1826] means for generating questions using a generative AI model;

[1827] means for managing input prompt sentences to the generative AI model;

[1828] A system including:

[1829] (Claim 2)

[1830] 10. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

[1831] (Claim 3)

[1832] 2. The system according to claim 1, wherein the generating means generates "why" questions using a generating AI.

[1833] "Application Example 1"

[1834] (Claim 1)

[1835] means for providing an interface for inputting accident information;

[1836] means for receiving the accident information and storing it in a database;

[1837] generating means for generating a first "why" question based on the accident information;

[1838] means for presenting the generated first "why" question;

[1839] means for collecting answers to said first "why" question;

[1840] generating means for generating more detailed "why" questions based on the collected answers;

[1841] means for posing said further detailed "why" questions;

[1842] A means for analyzing all of the questions and answers to identify the true cause of the accident;

[1843] means for presenting the identified root cause in the form of a report;

[1844] a means for automatically transmitting accident information when the robot detects an accident;

[1845] The server performs an initial analysis of the accident information in real time and generates questions using a generation AI.

[1846] A means for the robot administrator to answer questions and generate new questions based on those answers;

[1847] A system including:

[1848] (Claim 2)

[1849] 10. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

[1850] (Claim 3)

[1851] 2. The system according to claim 1, wherein the generating means generates "why" questions using a generating AI.

[1852] "Example 2: Combining Emotion Engines"

[1853] (Claim 1)

[1854] means for providing a user interface for inputting accident information;

[1855] means for receiving the accident information and storing it in a database;

[1856] generating means for generating a first "why" question based on the accident information;

[1857] means for presenting the generated first "why" question;

[1858] means for collecting answers to said first "why" question;

[1859] generating means for generating more detailed "why" questions based on the collected answers;

[1860] means for posing said further detailed "why" questions;

[1861] A means for analyzing all of the questions and answers to identify the true cause of the accident;

[1862] means for presenting the identified root cause in the form of a report;

[1863] means for recognizing the user's emotional state and adjusting the wording of the question;

[1864] A means for adjusting the way questions are presented to take into account the user's emotions;

[1865] A system including:

[1866] (Claim 2)

[1867] 10. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

[1868] (Claim 3)

[1869] 2. The system of claim 1, wherein the generating means generates "why" questions using a generative AI model.

[1870] "Application example 2 when combining emotion engines"

[1871] (Claim 1)

[1872] means for providing an interface for inputting accident information;

[1873] means for receiving the accident information and storing it in a database;

[1874] generating means for generating a first "why" question based on the accident information;

[1875] means for presenting the generated first "why" question;

[1876] means for collecting answers to said first "why" question;

[1877] generating means for generating more detailed "why" questions based on the collected answers;

[1878] means for posing said further detailed "why" questions;

[1879] A means for analyzing all of the questions and answers to identify the true cause of the accident;

[1880] means for presenting the identified root cause in the form of a report;

[1881] an emotion analysis means for recognizing the emotions of the workers;

[1882] an adjustment means for adjusting a question method and question content based on the emotion analysis means;

[1883] a data collection means for collecting video and audio from the accident scene;

[1884] A system including:

[1885] (Claim 2)

[1886] 10. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

[1887] (Claim 3)

[1888] 2. The system according to claim 1, wherein the generating means generates "why" questions using a generating AI. [Explanation of symbols]

[1889] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for providing an interface for inputting accident information; means for receiving the accident information and storing it in a database; generating means for generating a first "why" question based on the accident information; means for presenting the generated first "why" question; means for collecting answers to said first "why" question; generating means for generating more detailed "why" questions based on the collected answers; means for posing said further detailed "why" questions; A means for analyzing all of the questions and answers to identify the true cause of the accident; means for presenting the identified root cause in the form of a report; A system including:

2. 2. The system of claim 1, wherein the accident information includes the date, time, location, and circumstances of the accident.

3. 2. The system according to claim 1, wherein the generating means generates "why" questions using a generating AI.

Images