System

Abstract

A system is provided.SOLUTION: The system includes a means for inputting near-miss incident cases, a means for storing the input near-miss incident cases in a database, a means for analyzing the stored near-miss incident cases and retrieving similar cases, a means for specifying the root cause based on the retrieval result of the similar cases and proposing measures, and a means for executing the proposed measures and inputting the feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Near-miss incidents at worksites provide important information for preventing accidents and problems, but it is difficult to effectively accumulate and analyze this information and take appropriate countermeasures. While near-miss incidents are reported individually at many worksites, they are rarely managed and analyzed in an organized manner. Furthermore, because searching for similar incidents and identifying the root cause is time-consuming, countermeasures are often delayed. This results in a risk of reduced safety at the worksite. The present invention aims to solve these problems and provide a system that enables efficient management of near-miss incidents and rapid countermeasures. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by the following means. Specifically, it provides a system including a means for inputting near-miss incident cases, a means for storing the input near-miss incident cases in a database, a means for analyzing the stored near-miss incident cases and searching for similar incident cases, a means for identifying the root cause based on the search results of similar incidents and proposing countermeasures, and a means for implementing the proposed countermeasures and inputting feedback. This system has a means for inputting near-miss incident cases using a dedicated terminal, and also has a means for learning patterns and trends from the feedback data using machine learning to improve the accuracy of the analysis and countermeasure proposals. This allows on-site staff to perform everything from inputting near-miss incident cases to implementing and evaluating countermeasures in a unified manner, thereby effectively improving on-site safety.

[0006] "Near miss cases" refer to cases where an accident or trouble almost occurred at a work site, but was prevented from happening.

[0007] "Means of input" refers to the tools and interfaces that allow users to record details of near-miss incidents and provide them to the system.

[0008] A "database" refers to an information management system that systematically stores accumulated information such as near-miss incidents, allowing it to be searched and analyzed later.

[0009] "Means of analysis" refers to tools and algorithms that perform analysis such as searching for similar cases and identifying the root cause based on information accumulated in a database.

[0010] "Similar cases" refer to past cases that have specific characteristics or patterns similar to the input near-miss case.

[0011] "Means for identifying root causes" refers to analytical tools and algorithms used to identify the underlying causes of near-miss incidents.

[0012] "Means for proposing countermeasures" refers to tools and systems that suggest appropriate preventive measures and solutions to users based on the identified root causes.

[0013] "Means for inputting feedback" refers to tools or interfaces for re-entering the effects of implemented measures and on-site conditions into the system and accumulating them as feedback information.

[0014] A "dedicated terminal" refers to a device on which a specific application is installed for inputting near-miss incidents and recording feedback.

[0015] "Machine learning" refers to a technology that automatically learns important patterns and trends from feedback data, improving the accuracy of system analysis and countermeasure proposals. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system for carrying out the present invention is for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Below, each component of the system and its operation will be explained.

[0038] System configuration

[0039] 1. User Device

[0040] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0041] Users operate the system through a dedicated application or a web browser.

[0042] 2. Server

[0043] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0044] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[0045] 3. Database

[0046] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback.

[0047] Program processing explanation

[0048] User terminal operation

[0049] Users input examples of near misses that occurred on-site into a dedicated application or web form. For example, they can input an example such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[0050] Server Operation

[0051] The server receives near-miss incidents sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0052] Search and suggest similar cases

[0053] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to an expert, who can then conduct a more detailed analysis or ask questions to the user. For example, the expert can provide the user with information such as "what countermeasures have been effective in similar situations."

[0054] Identifying the root cause and proposing countermeasures

[0055] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[0056] Implementation and feedback input

[0057] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0058] Learning and accuracy improvement through machine learning

[0059] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0060] Operation example

[0061] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[0062] In this way, the system of the present invention is able to quickly and effectively manage and take action against near miss incidents.

[0063] The processing flow will be explained below.

[0064] Step 1: User Action

[0065] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[0066] Users input details of near miss incidents that occurred on-site. For example, they record an incident such as "While working at height, a tool slipped from my hand and nearly hit a worker below."

[0067] The user completes the data entry and presses the submit button.

[0068] Step 2: Device Operation

[0069] The terminal performs a format check on the entered data to ensure there are no missing items or formatting errors.

[0070] After verifying that the data has been entered correctly, convert the input data into an appropriate data format, such as JSON.

[0071] The converted data is sent to the server.

[0072] Step 3: Receiving and storing on the server

[0073] The server receives the data sent from the terminal.

[0074] The received data is analyzed and stored in a database in an appropriate format.

[0075] Step 4: Search for similar cases

[0076] The server searches the accumulated database and extracts past cases similar to the received near-miss case.

[0077] Apply natural language processing (NLP) techniques and machine learning algorithms to identify similar cases and their countermeasures.

[0078] Step 5: Initial analysis and expert notification

[0079] The server performs an initial analysis based on the search results for similar cases.

[0080] The initial analysis results are notified to experts via email or a dedicated application.

[0081] Step 6: Expert Operation

[0082] The expert accesses the server and checks the notified initial analysis results.

[0083] If necessary, the user is asked additional questions.

[0084] Step 7: Provide additional user information

[0085] The user answers questions from the expert and provides additional information.

[0086] Enter additional information and submit it to the server.

[0087] Step 8: Identify the root cause and propose a solution

[0088] The server identifies the true cause of near miss incidents based on the expert analysis results and additional information from users.

[0089] Based on the root cause, appropriate countermeasures are searched for in the database.

[0090] A countermeasure plan is generated and notified to the user.

[0091] Step 9: User implementation and feedback

[0092] The user implements the notified countermeasures on-site.

[0093] The effects of the implemented measures and the situation at the site are again entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of tools being dropped has decreased."

[0094] The user terminal transmits the feedback to the server.

[0095] Step 10: Gather feedback and learn

[0096] The server stores the received feedback in a database.

[0097] The machine learning model is updated based on the feedback data and learns to improve the accuracy of system analysis and countermeasure proposals.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Near miss incidents at work sites are important information for preventing serious accidents, but their collection, management, and analysis depend on human resources, making them inefficient and difficult to contribute to overall safety measures.In particular, when there are a large number of near miss incidents, it is difficult to search for similar incidents and propose appropriate countermeasures, which results in the problem of accident prevention measures not being implemented effectively.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes: means for a user to input near-miss incident cases; means for checking the format of the near-miss incident cases input at the terminal; means for transmitting data to the server; means for storing the data received by the server in a database; means for analyzing the stored data using natural language processing technology and machine learning algorithms and searching for similar incidents; means for notifying experts based on the search results; means for the experts to analyze the analysis results in detail and provide feedback to the user; means for the server to identify the root cause based on the expert feedback and propose countermeasures; means for the user to implement the proposed countermeasures and provide feedback on the results; and means for the server to update the machine learning model based on newly accumulated feedback information to improve accuracy. This makes it possible to efficiently collect, manage, and analyze near-miss incident cases and take prompt and appropriate countermeasures.

[0103] A "user terminal" is a device for transmitting near-miss incidents input by a user to a server.

[0104] A "near miss case" refers to an incident that nearly developed into an accident or disaster but was prevented.

[0105] "Format check" is the process of checking whether near miss cases are entered in the correct format.

[0106] A "server" is a central data processing system that receives, analyzes, and stores data sent from user terminals.

[0107] The "database" is a system that systematically stores and manages information such as near-miss incident cases, countermeasures, and feedback.

[0108] "Natural language processing technology" is a technology for understanding and analyzing human language.

[0109] A "machine learning algorithm" is a mathematical method for learning from data and making predictions or classifications.

[0110] "Searching for similar cases" is the process of finding past cases similar to a newly entered near-miss case from the database.

[0111] "Expert notification" is the process of communicating the results of the initial analysis to experts and soliciting their feedback for further analysis.

[0112] "Root cause identification" is an analytical process for identifying the underlying cause of a near miss incident.

[0113] "Proposing measures" is the process of presenting appropriate measures to eliminate the identified root causes.

[0114] "Feedback input" is the process in which the user inputs the results and effects of the measures they have implemented back into the system.

[0115] "Machine learning model updating" is the process of retraining machine learning algorithms based on newly accumulated data to improve the accuracy of the system.

[0116] The present invention is a system for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Each component of the present invention and its operation will be specifically described below.

[0117] System configuration

[0118] 1. User Device

[0119] A device that allows users to input near-miss incidents and provide feedback on countermeasures. Specifically, it can be a tablet, smartphone, or PC.

[0120] Users operate the system through a dedicated application or a web browser.

[0121] 2. Server

[0122] This is a central system that receives data sent from user devices and stores and analyzes it. Specifically, it uses cloud services. A server with high-performance data processing capabilities is desirable.

[0123] 3. Database

[0124] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback. SQL databases and NoSQL databases are used.

[0125] System processing flow

[0126] User terminal

[0127] Users can input near-miss incidents into a dedicated application or web form at the work site. For example, they can input an incident such as, "While working at height, a tool slipped from my hand and nearly hit a worker below." The terminal checks the format of this input information and, if there are no problems, sends it to the server.

[0128] server

[0129] The server receives near-miss incidents sent from user devices and stores them in a database. It then applies natural language processing (NLP) technology and machine learning algorithms to analyze the stored data and search for similar incidents. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts, and provides relevant information to the user.

[0130] Search and suggest similar cases

[0131] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to experts, who can then conduct further detailed analysis or ask questions to the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations."

[0132] Identifying the root cause and proposing countermeasures

[0133] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[0134] Implementation and feedback input

[0135] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0136] Learning and accuracy improvement through machine learning

[0137] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0138] Operation example

[0139] For example, if a tool slips from a user's hand while working at height in a manufacturing plant and nearly hits a worker below, the user can input the incident into a terminal. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and receives feedback on the results, allowing the entire system to be improved over time. In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures.

[0140] Prompt Sentence Examples

[0141] An example of a prompt sentence to be input to the generative AI model is shown below.

[0142] Please explain how you propose specific countermeasures when a user enters a near-miss incident case. Case: "While working at height, a tool slips from your hand and may hit a worker below."

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] The user inputs a near-miss incident.

[0146] Users use a dedicated application or web form to enter near-miss incidents that have occurred at the work site. The data entered includes details of the incident, date and time, and on-site conditions. An example of an input would be, "While working at height, a tool slipped from my hand and nearly hit a worker below."

[0147] Input: Near miss incidents input by users

[0148] Output: Input data that passes format check

[0149] Step 2:

[0150] Check the format of input data on the terminal

[0151] The terminal checks whether the data entered by the user conforms to the specified format. For example, it checks whether all required fields have been entered and whether the data format is correct. If the check passes, it proceeds to the next process.

[0152] Input: Near miss incidents entered by the user

[0153] Output: Data formatted to be sent to the server

[0154] Example: Check whether required fields such as "Event details," "Date and time," and "On-site situation" have been entered, and if any fields are missing, display a warning to the user.

[0155] Step 3:

[0156] Sending data to the server

[0157] The terminal sends data that has passed the format check to the server. At this time, the data is encrypted and sent using the HTTPS protocol to ensure data security.

[0158] Input: Data that passes format check

[0159] Output: Data sent to the server

[0160] Example: Encrypting data entered by a user and sending it to a server using HTTPS.

[0161] Step 4:

[0162] Receiving and storing data on the server

[0163] The server receives the data sent from the device and stores it in a database, which can then be used for analysis and retrieval.

[0164] Input: Data sent from the terminal

[0165] Output: Near miss cases stored in the database

[0166] Example: The server immediately stores the received data in the database and simultaneously backs up the data.

[0167] Step 5:

[0168] Data analysis and similar case search on the server

[0169] The server analyzes the accumulated data and searches for similar cases using natural language processing technology and machine learning algorithms. Specifically, it finds past cases of "tools falling while working at height" and analyzes countermeasures and impacts.

[0170] Input: Near miss cases stored in the database

[0171] Output: Search results for similar cases and initial analysis results

[0172] Example: Natural language processing techniques are used to tokenize example sentences entered by users, and similarities are calculated using machine learning algorithms.

[0173] Step 6:

[0174] Notification from the server to the expert

[0175] The server performs an initial analysis based on the search results for similar cases and notifies the experts of the results, who then begin a detailed analysis.

[0176] Input: Search results for similar cases and initial analysis results

[0177] Output:Notify Expert

[0178] Example: The server notifies the search results and initial analysis results via email or an expert dashboard.

[0179] Step 7:

[0180] Detailed analysis and feedback from experts

[0181] The expert conducts a detailed analysis and provides additional questions and feedback to the user, such as "What kind of gloves were you using?", and develops detailed countermeasures.

[0182] Input: Expert notification data and initial analysis results

[0183] Output: Detailed analysis by experts and feedback to users

[0184] Example: Experts can view detailed case studies from the dashboard and use the comments feature to ask users additional questions.

[0185] Step 8:

[0186] Identifying the root cause and proposing countermeasures by the server

[0187] Based on the expert analysis and past data, the server uses an algorithm to identify the root cause of near-miss incidents and develop appropriate countermeasures. For example, if the root cause is "inappropriate glove selection," the server will suggest specific countermeasures such as "using non-slip gloves."

[0188] Input: Expert detailed analysis and historical data

[0189] Output: Identified root cause and proposed measures

[0190] Example: The server runs an algorithm to find the root cause of the case and generate effective countermeasures as a proposal.

[0191] Step 9:

[0192] User implementation of measures and feedback

[0193] The user implements the measures in the field and then inputs the results and effects into the terminal again. For example, the user inputs feedback such as, "Using non-slip gloves has reduced the number of tools being dropped."

[0194] Input: Countermeasure proposals from the server, user feedback

[0195] Output: Data on the results of measures taken

[0196] Example: The user inputs the situation after implementing the countermeasures into the terminal and sends it to the server.

[0197] Step 10:

[0198] Server-based feedback accumulation and machine learning model updates

[0199] The server accumulates the feedback sent by users in a database and updates the machine learning model based on that information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0200] Input: Feedback data from users

[0201] Output: Updated machine learning model

[0202] Example: The server uses new feedback data to retrain the machine learning model, improving the accuracy of the system.

[0203] (Application example 1)

[0204] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0205] Near miss incidents can occur frequently in work environments such as factories. It is necessary to properly manage and analyze these incidents and implement appropriate countermeasures, but conventional systems often fail to adequately detect dangers in real time or propose specific countermeasures. Furthermore, there is a lack of means to efficiently collect worker feedback and improve the system based on the analysis results. Therefore, there is a need for a system that can efficiently manage and address near miss incidents while ensuring worker safety.

[0206] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0207] In this invention, the server includes means for inputting near-miss incident cases, means for storing the input near-miss incident cases in a database, means for analyzing the stored near-miss incident cases and searching for similar incidents, means for identifying root causes and proposing countermeasures based on the search results for similar incidents, means for implementing the proposed countermeasures and inputting feedback thereon, means for performing real-time monitoring in the factory, automatically detecting potential hazards, and proposing countermeasures, and means for functioning as an application installed on automated work equipment in the factory. This enables efficient management of near-miss incident cases and countermeasures in the work environment, and ensures the safety of workers.

[0208] A "near miss case" is an incident that comes just short of causing an accident or trouble in the work environment.

[0209] A "database" is a system that manages accumulated information in an organized manner and enables it to be quickly searched and used.

[0210] "Real-time monitoring" is a method of constantly monitoring the working environment and equipment status, and acquiring and analyzing data instantly.

[0211] "Automated work equipment" refers to machines and robots used in work environments such as factories.

[0212] "Feedback" refers to reports and opinions about the results and effectiveness of measures.

[0213] "Similar cases" refer to past near-miss cases that are similar to the current case.

[0214] A "remedy proposal" is a recommendation of appropriate action or change in response to a specific problem or risk.

[0215] "Analysis" involves examining accumulated data in detail and deriving meaning and patterns from it.

[0216] An "input device" is a device used by a user to input data.

[0217] "Machine learning" is a technology that allows computers to autonomously learn based on empirical data and make judgments and predictions.

[0218] A "pattern or trend" is a recurring characteristic or direction of variation in data.

[0219] MODE FOR CARRYING OUT THE INVENTION

[0220] The system for implementing the present invention efficiently manages and analyzes near-miss incidents in a factory environment and automatically suggests appropriate countermeasures. Each component of the system and its operation will be described below.

[0221] System configuration

[0222] 1. User Device

[0223] This device allows workers to input near-miss incidents and provides feedback on countermeasures.

[0224] Users operate the device through a dedicated application or a web browser, such as on a tablet or smartphone.

[0225] 2. Real-time monitoring device

[0226] The working environment and working conditions are constantly monitored using sensors and cameras mounted on automated work equipment (robot arms, etc.) in the factory.

[0227] The acquired data is immediately sent to a server where it is analyzed to detect potential dangers.

[0228] 3. Server

[0229] It is a central system that receives, stores, and analyzes data sent from user terminals and real-time monitoring devices.

[0230] Using cloud services (e.g., Amazon Web Services, Google Cloud Platform) enables high-speed data processing and large-volume data storage.

[0231] 4. Database

[0232] It is an information management system connected to a server that stores data such as near-miss incidents, real-time monitoring data, countermeasures, and feedback.

[0233] Program processing explanation

[0234] User terminal operation

[0235] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they can enter an incident such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[0236] Real-time monitoring in action

[0237] Sensors and cameras attached to the automated work equipment monitor the work situation in real time and send the acquired data to a server, which analyzes the data and detects potential hazards.

[0238] Server Operation

[0239] The server receives data sent from user devices and real-time monitoring devices and stores it in a database. It then analyzes the stored data and searches for similar cases, applying natural language processing (NLP) techniques (e.g., spaCy, NLTK) and machine learning algorithms (e.g., Scikit-learn, TensorFlow). For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0240] Search and suggest similar cases

[0241] The server searches for similar cases and performs an initial analysis based on the results. Based on the results of this initial analysis, the server proposes countermeasures. For example, it provides information to the user such as "In similar situations, using non-slip gloves was effective."

[0242] Implementation and feedback input

[0243] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0244] Learning and accuracy improvement through machine learning

[0245] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0246] Examples of concrete examples and prompts

[0247] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[0248] Example prompt sentence:

[0249] "While working at height, a tool almost slipped out of my hand. Please suggest the best solution to this problem based on similar cases from the past. We will also collect feedback to evaluate the effectiveness of the solution."

[0250] In this way, the system of the present invention is able to quickly and effectively manage and address near miss incidents in a factory environment.

[0251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0252] Step 1:

[0253] Input of near miss cases from user terminals

[0254] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they might enter, "While working at height, a tool nearly slipped out of my hand." The entered data is checked for format and, if there are no problems, is sent to the server.

[0255] Input: Near miss incident details

[0256] Output: Send data to the server

[0257] Step 2:

[0258] Data reception and storage on the server

[0259] The server stores the near-miss incident data received from the user terminal in a database, which also stores past near-miss incidents and countermeasures.

[0260] Input: Near miss incident data sent from the user's device

[0261] Output: Data accumulation in database

[0262] Step 3:

[0263] Real-time monitoring data collection

[0264] Sensors and cameras installed on automated work equipment in factories monitor the work situation and environment in real time, and the acquired data is immediately sent to a server.

[0265] Input: Real-time data from automated work equipment

[0266] Output: Send data to the server

[0267] Step 4:

[0268] Data analysis on the server

[0269] The server uses natural language processing (NLP) techniques and machine learning algorithms (e.g., spaCy, NLTK, Scikit-learn, TensorFlow) to analyze the accumulated data and real-time monitoring data, search for similar past cases, and extract information to propose appropriate countermeasures.

[0270] Input: Database and real-time monitoring data

[0271] Output: Analysis results and countermeasures

[0272] Step 5:

[0273] Search for similar cases and propose countermeasures

[0274] The server searches for similar cases based on the analysis results and provides the user with countermeasures and their impact. For example, if there is a case of a tool slipping off in the past, the server will suggest the use of non-slip gloves.

[0275] Input: Analysis results

[0276] Output: Countermeasure proposal

[0277] Step 6:

[0278] Implementing measures and providing feedback

[0279] The user implements the proposed measures on-site and then inputs the results and effects into the terminal again. For example, the user might input, "Using non-slip gloves resulted in fewer dropped tools." The feedback information is then sent to the server.

[0280] Input: Feedback after implementing measures

[0281] Output: Send feedback to the server

[0282] Step 7:

[0283] Accumulating feedback data and updating machine learning models

[0284] The server accumulates the feedback data in a database and updates the machine learning model, enabling more accurate countermeasure proposals the next time a near-miss occurs.

[0285] Input: Feedback data

[0286] Output: Database and machine learning model updates

[0287] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0288] The system for implementing this invention efficiently manages and analyzes near-miss incidents and takes more effective measures by combining it with an emotion engine that recognizes the user's emotions. Each component of this system and its operation will be explained below.

[0289] System configuration

[0290] 1. User Device

[0291] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0292] Users operate the system through a dedicated application or a web browser.

[0293] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[0294] 2. Server

[0295] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0296] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[0297] 3. Database

[0298] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[0299] Program processing explanation

[0300] User terminal operation

[0301] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[0302] Server Operation

[0303] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. It then analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0304] Search and suggest similar cases

[0305] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then reported to experts, who can then conduct further detailed analysis or ask questions of the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations." The server also determines the importance of the case based on the user's emotions evaluated by the emotion engine, and adjusts the priority of countermeasures as necessary.

[0306] Identifying the root cause and proposing countermeasures

[0307] The server runs an algorithm to identify the root cause of near-miss incidents based on the expert analysis, past data, and emotion data. For example, it might determine that "the cause of a tool slipping from the hand was improper use of gloves." It then suggests appropriate countermeasures, such as "use non-slip gloves for certain tasks." If the user feels anxious, it will provide further detailed explanations and encourage them to take countermeasures.

[0308] Implementation and feedback input

[0309] The user implements the notified measures on-site and again inputs the results and effects into the device. For example, the user might input feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." At this time, the emotion engine recognizes the user's emotions again and collects emotional data after the measures have been implemented. The device then sends this information to the server, which stores it in a database.

[0310] Learning and accuracy improvement through machine learning

[0311] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information and emotional data. This allows for more accurate countermeasure proposals the next time a near-miss occurs, while also enabling flexible responses based on the user's emotional state.

[0312] Operation example

[0313] For example, if a tool slips from a user's hand while working at a height in a manufacturing plant, nearly hitting a worker below, the user can input the incident into their device. At the same time, the emotion engine recognizes the user's anxious facial expression. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures, providing feedback on the results while also recollecting their emotional state at the time. As new feedback and emotion data accumulate, the entire system is continually improved, resulting in more effective countermeasures and responses that take the user's psychology into consideration.

[0314] In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures, and also to respond flexibly while taking into consideration the user's feelings.

[0315] The processing flow will be explained below.

[0316] Step 1: User Action

[0317] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[0318] Users input details of near miss incidents that occurred on-site. For example, they might record an incident such as "While working at height, a tool slipped from their hand and nearly hit a worker below."

[0319] The user enters the importance and scope of the case, and then presses the send button.

[0320] Step 2: Device Operation

[0321] The terminal performs a format check on the data entered by the user to ensure there are no missing items or formatting errors.

[0322] Once you have verified that the data has been entered accurately, convert the input data into an appropriate data format, such as JSON.

[0323] The converted data is sent to the server.

[0324] Step 3: Receiving and storing on the server

[0325] The server receives the data sent from the terminal.

[0326] The received near-miss incident data is analyzed and stored in a database in an appropriate format.

[0327] Step 4: Emotion Engine in Action

[0328] The emotion engine installed in the device simultaneously recognizes the user's facial expressions and voice to collect emotional data.

[0329] If the user is expressing emotions such as anxiety, impatience, or anger, this information is also sent to the server.

[0330] Step 5: Search for similar cases

[0331] The server searches past near-miss incident cases stored in a database and extracts cases similar to the received case.

[0332] Natural language processing (NLP) techniques and machine learning algorithms are used to extract relevant information from similar cases.

[0333] Step 6: Initial analysis and expert notification

[0334] The server performs an initial analysis based on the search results for similar cases.

[0335] The initial analysis results and the user's emotional data are notified to the experts via email or a dedicated application.

[0336] Step 7: Expert Operation

[0337] The expert accesses the server and checks the notified initial analysis results and emotion data.

[0338] If necessary, the user is asked additional questions.

[0339] Step 8: Provide additional user information

[0340] The user answers questions from the expert and provides additional information.

[0341] Enter additional information and submit it to the server.

[0342] Step 9: Identify the root cause and propose a solution

[0343] The server identifies the root cause of near miss incidents based on the expert analysis results, additional information from users, and emotional data.

[0344] Based on the root cause, appropriate countermeasures are searched for in the database.

[0345] A countermeasure plan is generated and notified to the user.

[0346] Step 10: User implementation and feedback

[0347] The user implements the notified countermeasures on-site.

[0348] The effects of the implemented measures and the situation at the site are entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of dropped tools has decreased."

[0349] The emotion engine installed on the device will again recognize the user's emotions and collect emotional data after the activity.

[0350] Step 11: Gather feedback and learn

[0351] The server stores the received feedback and emotion data in a database.

[0352] Based on feedback data and sentiment data, the machine learning model is updated to improve the accuracy of analysis and countermeasure proposals.

[0353] Example 2

[0354] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0355] Conventional near-miss incident management systems primarily focus on case input and analysis, and lack the ability to propose flexible countermeasures that take into account the user's emotional state and learning from feedback. This makes it difficult to propose more effective countermeasures while reducing the user's psychological anxiety and stress. Furthermore, technology for retrieving similar cases and improving analysis accuracy is insufficient, limiting the effectiveness of the proposed countermeasures.

[0356] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0357] In this invention, the server includes a means for inputting near-miss incidents, a means for storing the input near-miss incidents in a database, and a means for analyzing the stored near-miss incidents and searching for similar incidents. This enables the user to quickly collect and manage near-miss incidents experienced in the field, and enables highly accurate analysis and searching for similar incidents. Furthermore, by including a means for recognizing the user's emotional state and collecting emotional data, a means for analyzing the collected emotional data and adjusting the importance of countermeasures, and a means for using machine learning to learn patterns and trends from feedback data and emotional data and improve the accuracy of countermeasure proposals, flexible and effective countermeasure proposals that take the user's psychological state into consideration are possible.

[0358] A "near miss incident" refers to an event or occurrence that did not result in a serious accident or trouble, but could have caused danger or problems.

[0359] "Input means" refers to the means by which a user inputs near-miss incidents and feedback information into a device.

[0360] "Database" refers to a system that centrally manages accumulated data such as near-miss incident cases, emotional data, and feedback information.

[0361] "Analysis means" refers to the technology used to analyze accumulated data, search for similar cases, and discover patterns.

[0362] The "similar case search means" refers to a means for searching for cases similar to the currently input case from data accumulated in the past.

[0363] "Means for identifying the root cause" refers to a means for analyzing the causes of the near-miss incidents entered and identifying the underlying problems.

[0364] "Measure suggestion means" refers to a means for proposing optimal solutions or measures to users based on the identified root cause.

[0365] The "feedback input means" refers to a means for a user to input the results and impressions of implementing the proposed measures back into the system.

[0366] "Emotion recognition means" refers to technology for collecting and analyzing emotional data from a user's facial expressions and voice.

[0367] "Emotion data analysis means" refers to a means for analyzing collected emotion data and adjusting the importance of countermeasures based on the analysis.

[0368] "Machine learning methods" refers to technologies that learn patterns and trends from feedback data and emotion data to improve the accuracy of analysis and countermeasure proposals.

[0369] "Natural language processing technology" refers to technology that analyzes accumulated text data to search for similar cases and assist in problem-solving.

[0370] This invention is a system that efficiently manages and analyzes near-miss incidents and takes more effective countermeasures by combining it with an emotion engine that recognizes user emotions. The system includes a series of processes from user input to countermeasure proposals, and also uses user emotion data for analysis.

[0371] System configuration

[0372] 1. User Device

[0373] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0374] Users operate the system through a dedicated application or a web browser.

[0375] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[0376] 2. Server

[0377] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0378] Cloud services may also be used, enabling high-speed data processing and large-volume data storage. The server uses Python and machine learning frameworks (such as TensorFlow and PyTorch) to analyze and visualize the data.

[0379] 3. Database

[0380] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[0381] Using a database system such as PostgreSQL enables efficient data management and fast searches.

[0382] Program operation explanation

[0383] User terminal operation

[0384] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[0385] Server Operation

[0386] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar cases by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact and provides relevant information to the user. Specifically, the analysis is performed using Python's NLTK library and Scikit-learn.

[0387] Examples of prompt statements

[0388] Here are some example prompts to input to a generative AI model:

[0389] When a user inputs a near-miss incident into a terminal, such as when a tool nearly fell while working at height, the emotion engine also identifies the user's anxious facial expression at the time. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using non-slip gloves" or "wearing a tool wristband." Please explain how this near-miss incident management system, which incorporates emotion engineering, works.

[0390] Using this prompt, the generative AI model is expected to provide a detailed explanation of the behavior of a specific system.

[0391] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0392] Step 1:

[0393] User input of near miss incidents

[0394] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. Specifically, they use the device's keyboard or touch panel to enter details of the incident, such as "While working at height, a tool slipped from my hand and nearly hit a worker below." The input in this step is a record of the actual near-miss incident, and the output is the entered text data.

[0395] Step 2:

[0396] Emotion data collection using an emotion engine

[0397] An emotion engine built into the user device recognizes the user's facial expressions and voice. Specifically, the device's camera captures facial expressions, and the microphone analyzes the voice. For example, if the user looks anxious, that facial expression data is collected. The input for this step is the user's facial expression and voice, and the output is analyzed emotion data.

[0398] Step 3:

[0399] Data check and transmission by user terminal

[0400] The terminal performs a format check on the input near-miss case information and collected emotion data. Specifically, it checks the required fields on the input form and the consistency of the data format, and if there are no problems, it sends the data to the server. For example, it checks that the input case meets all required fields. The input for this step is near-miss case data and emotion data, and the output is the checked data.

[0401] Step 4:

[0402] Receiving and storing data by the server

[0403] The server receives the near-miss incidents and emotion data sent from the user's device and stores them in a database. Specifically, it receives the data using a RESTful API and saves it in a database such as PostgreSQL. The input to this step is the sent near-miss incident data and emotion data, and the output is the data saved in the database.

[0404] Step 5:

[0405] Server-based search and analysis of similar cases

[0406] The server analyzes the accumulated data and searches for similar cases. Specifically, it applies natural language processing (NLP) techniques and machine learning algorithms using Python's NLTK library and Scikit-learn. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact. The input for this step is the accumulated data, and the output is the search results and analysis results for similar cases.

[0407] Step 6:

[0408] Initial analysis and suggestions based on search results for similar cases

[0409] The server performs an initial analysis based on the search results. The results of this initial analysis are then sent to experts, who then conduct further detailed analysis or ask questions of the user. Specifically, the data is displayed in real time using a dedicated dashboard. The input to this step is the search results for similar cases, and the output is the initial analysis results.

[0410] Step 7:

[0411] Identifying the root cause by the server and proposing detailed countermeasures

[0412] The server runs an algorithm to identify the root cause based on the expert's analysis results, past data, and emotion data. For example, it might determine that "the cause of the tool slipping from the hand is improper use of gloves." It then suggests an appropriate countermeasure: "Use non-slip gloves for certain tasks." The input for this step is the initial analysis results and past data, and the output is a detailed countermeasure proposal.

[0413] Step 8:

[0414] User implementation and feedback

[0415] The user implements the notified countermeasures on-site and again inputs the results and effects into the terminal. Specifically, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The input in this step is the result of implementing the countermeasures, and the output is feedback data.

[0416] Step 9:

[0417] Server receives feedback and updates the learning model

[0418] The server updates the machine learning model using the newly accumulated feedback information and emotion data. Specifically, it retrains the model using TensorFlow and PyTorch. This enables more accurate countermeasure proposals for the next near-miss incident. The input for this step is the feedback data and emotion data, and the output is an updated machine learning model.

[0419] (Application example 2)

[0420] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] Conventional near-miss incident analysis systems did not efficiently manage and analyze near-miss incidents, nor did they evaluate risks or respond to them while taking into account user emotional data. As a result, countermeasure proposals did not take into account the user's psychological state, making it difficult to achieve both improved safety and user satisfaction. In addition, the effective use of feedback data was limited, limiting the improvement of accuracy.

[0422] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting near-miss incident cases; means for storing the input near-miss incident cases in a database; means for analyzing the stored near-miss incident cases and searching for similar incident cases; means for identifying root causes and proposing countermeasures based on the search results for similar incident cases; means for implementing the proposed countermeasures and inputting feedback therefrom; means for collecting and analyzing emotional data and reflecting it in risk assessment; means for adjusting priorities based on the user's emotional state when proposing countermeasures; and means for learning patterns and trends from the feedback data and emotional data using machine learning to improve the accuracy of the analysis and countermeasure proposals. This enables safety countermeasures based on near-miss incident cases to be flexible and effective, taking into account the user's emotions, enabling more accurate countermeasure proposals and improved user satisfaction.

[0423] A "near miss incident" refers to an incident that did not actually result in an accident in a factory or work site, but was potentially dangerous.

[0424] "Storage" means systematically collecting and storing data and information.

[0425] "Analysis" refers to the process of investigating and analyzing data in detail to clarify its background and causes.

[0426] A "similar case" refers to an event that occurred in the past that shares characteristics or patterns with the current case.

[0427] A "root cause" is the direct cause that causes a certain phenomenon or result.

[0428] "Countermeasures" refer to the means or methods taken to prevent or resolve a specific problem or danger.

[0429] "Feedback" refers to the process of providing an evaluation or response to actions or results, and making improvements or corrections based on that information.

[0430] "Emotional data" refers to data that expresses the user's emotions and psychological state, which can be obtained from facial expressions, voice, posture, etc.

[0431] "Risk assessment" refers to the process of evaluating the dangers of specific tasks or situations and determining the necessity and priority of countermeasures.

[0432] "Priority" refers to determining the order in which multiple tasks or problems should be processed first.

[0433] "Machine learning" is a field of artificial intelligence that allows computers to find patterns in data and make predictions and decisions based on them.

[0434] The system for implementing this invention is designed to manage near-miss incidents and improve safety within factories. This system consists of a series of steps: inputting near-miss incidents, collecting and analyzing emotional data, storing and searching data, proposing appropriate countermeasures, and collecting feedback.

[0435] System Components

[0436] 1. User Device

[0437] Input method: Users input near-miss incidents that occur in the factory. For example, they can input an incident such as "a tool slipped out of the hand while working at height" using a smartphone, tablet, or dedicated device.

[0438] Emotion data collection method: A camera and microphone are built into the device to collect the user's facial expressions and voice. This allows for the collection of emotional data such as the user's anxiety or surprise. Software such as Affectiva is used as the emotion engine.

[0439] 2. Server

[0440] Data storage method: Near-miss incidents and emotion data sent from user devices are received and stored in a database. Large volumes of data can be stored using cloud services.

[0441] Similar case search method: Apply natural language processing (NLP) technology and machine learning algorithms to past near-miss case data to search for similar cases. For example, libraries such as Scikit-learn and TensorFlow are used.

[0442] Countermeasure suggestion method: Based on the search results of similar cases, appropriate countermeasures are suggested. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[0443] Feedback collection method: The results of the measures taken by users and their emotional data at the time are collected again and stored in a database.

[0444] 3. Machine Learning Models

[0445] Learning and accuracy improvement measures: Using accumulated feedback data and sentiment data, the machine learning model is updated to improve the accuracy of countermeasure proposals.

[0446] Examples and prompts

[0447] Example 1:

[0448] Near miss case: While working at height, a tool slipped from the worker's hand and nearly hit a worker below.

[0449] Emotion data collection: Detecting "anxiety" from the operator's voice

[0450] Similar case search results: Past cases and the suggestion to "use non-slip gloves" as a countermeasure

[0451] Example 2:

[0452] Near miss case: Heavy machinery almost crashed into a wall due to an operating error

[0453] Emotion data collection: Detecting "surprise" from the operator's facial expression

[0454] Similar case search results: Past cases and suggested countermeasures such as "sensor readjustment" and "additional safety checks"

[0455] Prompt Sentence Examples

[0456] User submitted example: While working at height, a tool slipped from his hand and nearly hit a worker below.

[0457] Emotional data: Anxiety

[0458] Analysis result: This is a case where a tool fell while working at height. Based on past data, we suggest using non-slip gloves. Please take this measure.

[0459] In this way, this system efficiently carries out a series of steps, from managing near-miss incidents to proposing countermeasures and providing feedback, in order to improve safety within the factory, and is able to take flexible and effective measures that take into account the user's emotional state.

[0460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0461] Step 1:

[0462] Users input cases of near misses that have occurred in the factory into the terminal. For example, they can use a smartphone, tablet, or dedicated terminal to input an example in text format, such as "A tool slipped out of my hand while working at height."

[0463] Input: Near miss incidents entered by the user

[0464] Output: Near miss incidents in text format

[0465] Step 2:

[0466] The user's facial expressions and voice are collected using a camera and microphone installed on the user's device, and the emotional data is analyzed by an emotion engine (e.g., Affectiva), which obtains emotional data such as the user's anxiety or surprise.

[0467] Input: User facial and voice data

[0468] Output: Parsed emotion data

[0469] Step 3:

[0470] The device sends the collected near-miss incidents and emotion data to a server, where the data is checked for consistency in data format and then stored in a database.

[0471] Input: Submitted near-miss incident and emotion data

[0472] Output: Near miss incidents and emotion data stored in a database

[0473] Step 4:

[0474] The server analyzes the accumulated near-miss incident cases and searches for similar cases using natural language processing (NLP) technology and machine learning algorithms (such as Scikit-learn and TensorFlow). For example, it searches to see if there have been any past cases of "a tool falling while working at height."

[0475] Input: Near miss incidents stored in the database

[0476] Output: Search results for similar cases

[0477] Step 5:

[0478] The server identifies the root cause based on the search results of similar cases and suggests appropriate countermeasures. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[0479] Input: Search results for similar cases and sentiment data

[0480] Output: List of proposed measures

[0481] Step 6:

[0482] The server notifies the user of the proposed measures. The user implements the measures and inputs the results into the terminal again. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of tools dropped has decreased."

[0483] Input: Proposed measures and user implementation results

[0484] Output: Input feedback data

[0485] Step 7:

[0486] The device then sends the collected feedback and emotion data to a server where it is stored in a database. The stored data is used for further analysis and countermeasure proposals.

[0487] Input: Feedback and emotion data

[0488] Output: Feedback and emotion data stored in a database

[0489] Step 8:

[0490] The server updates the machine learning model using the accumulated new feedback and emotion data, improving the accuracy of the next countermeasure proposal and enabling flexible countermeasures to be taken based on the user's emotional state.

[0491] Input: Newly accumulated feedback and emotion data

[0492] Output: An updated machine learning model

[0493] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0494] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0495] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0496] [Second embodiment]

[0497] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0498] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0500] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0504] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0507] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0508] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0509] The system for carrying out the present invention is for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Below, each component of the system and its operation will be explained.

[0510] System configuration

[0511] 1. User Device

[0512] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0513] Users operate the system through a dedicated application or a web browser.

[0514] 2. Server

[0515] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0516] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[0517] 3. Database

[0518] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback.

[0519] Program processing explanation

[0520] User terminal operation

[0521] Users input examples of near misses that occurred on-site into a dedicated application or web form. For example, they can input an example such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[0522] Server Operation

[0523] The server receives near-miss incidents sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0524] Search and suggest similar cases

[0525] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to an expert, who can then conduct a more detailed analysis or ask questions to the user. For example, the expert can provide the user with information such as "what countermeasures have been effective in similar situations."

[0526] Identifying the root cause and proposing countermeasures

[0527] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[0528] Implementation and feedback input

[0529] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0530] Learning and accuracy improvement through machine learning

[0531] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0532] Operation example

[0533] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[0534] In this way, the system of the present invention is able to quickly and effectively manage and take action against near miss incidents.

[0535] The processing flow will be explained below.

[0536] Step 1: User Action

[0537] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[0538] Users input details of near miss incidents that occurred on-site. For example, they record an incident such as "While working at height, a tool slipped from my hand and nearly hit a worker below."

[0539] The user completes the data entry and presses the submit button.

[0540] Step 2: Device Operation

[0541] The terminal performs a format check on the entered data to ensure there are no missing items or formatting errors.

[0542] After verifying that the data has been entered correctly, convert the input data into an appropriate data format, such as JSON.

[0543] The converted data is sent to the server.

[0544] Step 3: Receiving and storing on the server

[0545] The server receives the data sent from the terminal.

[0546] The received data is analyzed and stored in a database in an appropriate format.

[0547] Step 4: Search for similar cases

[0548] The server searches the accumulated database and extracts past cases similar to the received near-miss case.

[0549] Apply natural language processing (NLP) techniques and machine learning algorithms to identify similar cases and their countermeasures.

[0550] Step 5: Initial analysis and expert notification

[0551] The server performs an initial analysis based on the search results for similar cases.

[0552] The initial analysis results are notified to experts via email or a dedicated application.

[0553] Step 6: Expert Operation

[0554] The expert accesses the server and checks the notified initial analysis results.

[0555] If necessary, the user is asked additional questions.

[0556] Step 7: Provide additional user information

[0557] The user answers questions from the expert and provides additional information.

[0558] Enter additional information and submit it to the server.

[0559] Step 8: Identify the root cause and propose a solution

[0560] The server identifies the true cause of near miss incidents based on the expert analysis results and additional information from users.

[0561] Based on the root cause, appropriate countermeasures are searched for in the database.

[0562] A countermeasure plan is generated and notified to the user.

[0563] Step 9: User implementation and feedback

[0564] The user implements the notified countermeasures on-site.

[0565] The effects of the implemented measures and the situation at the site are again entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of tools being dropped has decreased."

[0566] The user terminal transmits the feedback to the server.

[0567] Step 10: Gather feedback and learn

[0568] The server stores the received feedback in a database.

[0569] The machine learning model is updated based on the feedback data and learns to improve the accuracy of system analysis and countermeasure proposals.

[0570] Example 1

[0571] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0572] Near miss incidents at work sites are important information for preventing serious accidents, but their collection, management, and analysis depend on human resources, making them inefficient and difficult to contribute to overall safety measures.In particular, when there are a large number of near miss incidents, it is difficult to search for similar incidents and propose appropriate countermeasures, which results in the problem of accident prevention measures not being implemented effectively.

[0573] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0574] In this invention, the server includes: means for a user to input near-miss incident cases; means for checking the format of the near-miss incident cases input at the terminal; means for transmitting data to the server; means for storing the data received by the server in a database; means for analyzing the stored data using natural language processing technology and machine learning algorithms and searching for similar incidents; means for notifying experts based on the search results; means for the experts to analyze the analysis results in detail and provide feedback to the user; means for the server to identify the root cause based on the expert feedback and propose countermeasures; means for the user to implement the proposed countermeasures and provide feedback on the results; and means for the server to update the machine learning model based on newly accumulated feedback information to improve accuracy. This makes it possible to efficiently collect, manage, and analyze near-miss incident cases and take prompt and appropriate countermeasures.

[0575] A "user terminal" is a device for transmitting near-miss incidents input by a user to a server.

[0576] A "near miss case" refers to an incident that nearly developed into an accident or disaster but was prevented.

[0577] "Format check" is the process of checking whether near miss cases are entered in the correct format.

[0578] A "server" is a central data processing system that receives, analyzes, and stores data sent from user terminals.

[0579] The "database" is a system that systematically stores and manages information such as near-miss incident cases, countermeasures, and feedback.

[0580] "Natural language processing technology" is a technology for understanding and analyzing human language.

[0581] A "machine learning algorithm" is a mathematical method for learning from data and making predictions or classifications.

[0582] "Searching for similar cases" is the process of finding past cases similar to a newly entered near-miss case from the database.

[0583] "Expert notification" is the process of communicating the results of the initial analysis to experts and soliciting their feedback for further analysis.

[0584] "Root cause identification" is an analytical process for identifying the underlying cause of a near miss incident.

[0585] "Proposing measures" is the process of presenting appropriate measures to eliminate the identified root causes.

[0586] "Feedback input" is the process in which the user inputs the results and effects of the measures they have implemented back into the system.

[0587] "Machine learning model updating" is the process of retraining machine learning algorithms based on newly accumulated data to improve the accuracy of the system.

[0588] The present invention is a system for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Each component of the present invention and its operation will be specifically described below.

[0589] System configuration

[0590] 1. User Device

[0591] A device that allows users to input near-miss incidents and provide feedback on countermeasures. Specifically, it can be a tablet, smartphone, or PC.

[0592] Users operate the system through a dedicated application or a web browser.

[0593] 2. Server

[0594] This is a central system that receives data sent from user devices and stores and analyzes it. Specifically, it uses cloud services. A server with high-performance data processing capabilities is desirable.

[0595] 3. Database

[0596] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback. SQL databases and NoSQL databases are used.

[0597] System processing flow

[0598] User terminal

[0599] Users can input near-miss incidents into a dedicated application or web form at the work site. For example, they can input an incident such as, "While working at height, a tool slipped from my hand and nearly hit a worker below." The device checks the format of this input information and, if there are no problems, sends it to the server.

[0600] server

[0601] The server receives near-miss incidents sent from user devices and stores them in a database. It then applies natural language processing (NLP) technology and machine learning algorithms to analyze the stored data and search for similar incidents. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts, and provides relevant information to the user.

[0602] Search and suggest similar cases

[0603] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to experts, who can then conduct further detailed analysis or ask questions to the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations."

[0604] Identifying the root cause and proposing countermeasures

[0605] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[0606] Implementation and feedback input

[0607] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0608] Learning and accuracy improvement through machine learning

[0609] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0610] Operation example

[0611] For example, if a tool slips from a user's hand while working at height in a manufacturing plant and nearly hits a worker below, the user can input the incident into a terminal. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and receives feedback on the results, allowing the entire system to be improved over time. In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures.

[0612] Prompt Sentence Examples

[0613] An example of a prompt sentence to be input to the generative AI model is shown below.

[0614] Please explain how you propose specific countermeasures when a user enters a near-miss incident case. Case: "While working at height, a tool slips from your hand and may hit a worker below."

[0615] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0616] Step 1:

[0617] The user inputs a near-miss incident.

[0618] Users use a dedicated application or web form to enter near-miss incidents that have occurred at the work site. The data entered includes details of the incident, date and time, and on-site conditions. An example of an input would be, "While working at height, a tool slipped from my hand and nearly hit a worker below."

[0619] Input: Near miss incidents input by users

[0620] Output: Input data that passes format check

[0621] Step 2:

[0622] Check the format of input data on the terminal

[0623] The terminal checks whether the data entered by the user conforms to the specified format. For example, it checks whether all required fields have been entered and whether the data format is correct. If the check passes, it proceeds to the next process.

[0624] Input: Near miss incidents entered by the user

[0625] Output: Data formatted to be sent to the server

[0626] Example: Check whether required fields such as "Event details," "Date and time," and "On-site situation" have been entered, and if any fields are missing, display a warning to the user.

[0627] Step 3:

[0628] Sending data to the server

[0629] The terminal sends data that has passed the format check to the server. At this time, the data is encrypted and sent using the HTTPS protocol to ensure data security.

[0630] Input: Data that passes format check

[0631] Output: Data sent to the server

[0632] Example: Encrypting data entered by a user and sending it to a server using HTTPS.

[0633] Step 4:

[0634] Receiving and storing data on the server

[0635] The server receives the data sent from the device and stores it in a database, which can then be used for analysis and retrieval.

[0636] Input: Data sent from the terminal

[0637] Output: Near miss cases stored in the database

[0638] Example: The server immediately stores the received data in the database and simultaneously backs up the data.

[0639] Step 5:

[0640] Data analysis and similar case search on the server

[0641] The server analyzes the accumulated data and searches for similar cases using natural language processing technology and machine learning algorithms. Specifically, it finds past cases of "tools falling while working at height" and analyzes countermeasures and impacts.

[0642] Input: Near miss cases stored in the database

[0643] Output: Search results for similar cases and initial analysis results

[0644] Example: Natural language processing techniques are used to tokenize example sentences entered by users, and similarities are calculated using machine learning algorithms.

[0645] Step 6:

[0646] Notification from the server to the expert

[0647] The server performs an initial analysis based on the search results for similar cases and notifies the experts of the results, who then begin a detailed analysis.

[0648] Input: Search results for similar cases and initial analysis results

[0649] Output:Notify Expert

[0650] Example: The server notifies the search results and initial analysis results via email or an expert dashboard.

[0651] Step 7:

[0652] Detailed analysis and feedback from experts

[0653] The expert conducts a detailed analysis and provides additional questions and feedback to the user, such as "What kind of gloves were you using?", and develops detailed countermeasures.

[0654] Input: Expert notification data and initial analysis results

[0655] Output: Detailed analysis by experts and feedback to users

[0656] Example: Experts can review detailed case studies from the dashboard and use the comments feature to ask users additional questions.

[0657] Step 8:

[0658] Identifying the root cause and proposing countermeasures by the server

[0659] Based on the expert analysis and past data, the server uses an algorithm to identify the root cause of near-miss incidents and develop appropriate countermeasures. For example, if the root cause is "inappropriate glove selection," the server will suggest specific countermeasures such as "using non-slip gloves."

[0660] Input: Expert detailed analysis and historical data

[0661] Output: Identified root cause and proposed measures

[0662] Example: The server runs an algorithm to find the root cause of the case and generate effective countermeasures as a proposal.

[0663] Step 9:

[0664] User implementation of measures and feedback

[0665] The user implements the measures in the field and then inputs the results and effects into the terminal again. For example, the user inputs feedback such as, "Using non-slip gloves has reduced the number of tools being dropped."

[0666] Input: Countermeasure proposals from the server, user feedback

[0667] Output: Data on the results of measures taken

[0668] Example: The user inputs the situation after implementing the countermeasures into the terminal and sends it to the server.

[0669] Step 10:

[0670] Server-based feedback accumulation and machine learning model updates

[0671] The server accumulates the feedback sent by users in a database and updates the machine learning model based on that information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0672] Input: Feedback data from users

[0673] Output: Updated machine learning model

[0674] Example: The server uses new feedback data to retrain the machine learning model, improving the accuracy of the system.

[0675] (Application example 1)

[0676] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0677] Near miss incidents can occur frequently in work environments such as factories. It is necessary to properly manage and analyze these incidents and implement appropriate countermeasures, but conventional systems often fail to adequately detect dangers in real time or propose specific countermeasures. Furthermore, there is a lack of means to efficiently collect worker feedback and improve the system based on the analysis results. Therefore, there is a need for a system that can efficiently manage and address near miss incidents while ensuring worker safety.

[0678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0679] In this invention, the server includes means for inputting near-miss incident cases, means for storing the input near-miss incident cases in a database, means for analyzing the stored near-miss incident cases and searching for similar incidents, means for identifying root causes and proposing countermeasures based on the search results for similar incidents, means for implementing the proposed countermeasures and inputting feedback thereon, means for performing real-time monitoring in the factory, automatically detecting potential hazards, and proposing countermeasures, and means for functioning as an application installed on automated work equipment in the factory. This enables efficient management of near-miss incident cases and countermeasures in the work environment, and ensures the safety of workers.

[0680] A "near miss case" is an incident that comes just short of causing an accident or trouble in the work environment.

[0681] A "database" is a system that manages accumulated information in an organized manner and enables it to be quickly searched and used.

[0682] "Real-time monitoring" is a method of constantly monitoring the working environment and equipment status, and acquiring and analyzing data instantly.

[0683] "Automated work equipment" refers to machines and robots used in work environments such as factories.

[0684] "Feedback" refers to reports and opinions about the results and effectiveness of measures.

[0685] "Similar cases" refer to past near-miss cases that are similar to the current case.

[0686] A "remedy proposal" is a recommendation of appropriate action or change in response to a specific problem or risk.

[0687] "Analysis" involves examining accumulated data in detail and deriving meaning and patterns from it.

[0688] An "input device" is a device used by a user to input data.

[0689] "Machine learning" is a technology that allows computers to autonomously learn based on empirical data and make judgments and predictions.

[0690] A "pattern or trend" is a recurring characteristic or direction of variation in data.

[0691] MODE FOR CARRYING OUT THE INVENTION

[0692] The system for implementing the present invention efficiently manages and analyzes near-miss incidents in a factory environment and automatically suggests appropriate countermeasures. Each component of the system and its operation will be described below.

[0693] System configuration

[0694] 1. User Device

[0695] This device allows workers to input near-miss incidents and provides feedback on countermeasures.

[0696] Users operate the device through a dedicated application or a web browser, such as on a tablet or smartphone.

[0697] 2. Real-time monitoring device

[0698] The working environment and working conditions are constantly monitored using sensors and cameras mounted on automated work equipment (robot arms, etc.) in the factory.

[0699] The acquired data is immediately sent to a server where it is analyzed to detect potential dangers.

[0700] 3. Server

[0701] It is a central system that receives, stores, and analyzes data sent from user terminals and real-time monitoring devices.

[0702] Using cloud services (e.g., Amazon Web Services, Google Cloud Platform) enables high-speed data processing and large-volume data storage.

[0703] 4. Database

[0704] It is an information management system connected to a server that stores data such as near-miss incidents, real-time monitoring data, countermeasures, and feedback.

[0705] Program processing explanation

[0706] User terminal operation

[0707] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they can enter an incident such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[0708] Real-time monitoring in action

[0709] Sensors and cameras attached to the automated work equipment monitor the work situation in real time and send the acquired data to a server, which analyzes the data and detects potential hazards.

[0710] Server Operation

[0711] The server receives data sent from user devices and real-time monitoring devices and stores it in a database. It then analyzes the stored data and searches for similar cases, applying natural language processing (NLP) techniques (e.g., spaCy, NLTK) and machine learning algorithms (e.g., Scikit-learn, TensorFlow). For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0712] Search and suggest similar cases

[0713] The server searches for similar cases and performs an initial analysis based on the results. Based on the results of this initial analysis, the server proposes countermeasures. For example, it provides information to the user such as "In similar situations, using non-slip gloves was effective."

[0714] Implementation and feedback input

[0715] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[0716] Learning and accuracy improvement through machine learning

[0717] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[0718] Examples of concrete examples and prompts

[0719] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[0720] Example prompt sentence:

[0721] "While working at height, a tool almost slipped out of my hand. Please suggest the best solution to this problem based on similar cases from the past. We will also collect feedback to evaluate the effectiveness of the solution."

[0722] In this way, the system of the present invention is able to quickly and effectively manage and address near miss incidents in a factory environment.

[0723] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0724] Step 1:

[0725] Input of near miss cases from user terminals

[0726] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they might enter, "While working at height, a tool nearly slipped out of my hand." The entered data is checked for format and, if there are no problems, is sent to the server.

[0727] Input: Near miss incident details

[0728] Output: Send data to the server

[0729] Step 2:

[0730] Data reception and storage on the server

[0731] The server stores the near-miss incident data received from the user terminal in a database, which also stores past near-miss incidents and countermeasures.

[0732] Input: Near miss incident data sent from the user's device

[0733] Output: Data accumulation in database

[0734] Step 3:

[0735] Real-time monitoring data collection

[0736] Sensors and cameras installed on automated work equipment in factories monitor the work situation and environment in real time, and the acquired data is immediately sent to a server.

[0737] Input: Real-time data from automated work equipment

[0738] Output: Send data to the server

[0739] Step 4:

[0740] Data analysis on the server

[0741] The server uses natural language processing (NLP) techniques and machine learning algorithms (e.g., spaCy, NLTK, Scikit-learn, TensorFlow) to analyze the accumulated data and real-time monitoring data, search for similar past cases, and extract information to propose appropriate countermeasures.

[0742] Input: Database and real-time monitoring data

[0743] Output: Analysis results and countermeasures

[0744] Step 5:

[0745] Search for similar cases and propose countermeasures

[0746] The server searches for similar cases based on the analysis results and provides the user with countermeasures and their impact. For example, if there is a case of a tool slipping off in the past, the server will suggest the use of non-slip gloves.

[0747] Input: Analysis results

[0748] Output: Countermeasure proposal

[0749] Step 6:

[0750] Implementing measures and providing feedback

[0751] The user implements the proposed measures on-site and then inputs the results and effects into the terminal again. For example, the user might input, "Using non-slip gloves resulted in fewer dropped tools." The feedback information is then sent to the server.

[0752] Input: Feedback after implementing measures

[0753] Output: Send feedback to the server

[0754] Step 7:

[0755] Accumulating feedback data and updating machine learning models

[0756] The server accumulates the feedback data in a database and updates the machine learning model, enabling more accurate countermeasure proposals the next time a near-miss occurs.

[0757] Input: Feedback data

[0758] Output: Database and machine learning model updates

[0759] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0760] The system for implementing this invention efficiently manages and analyzes near-miss incidents and takes more effective measures by combining it with an emotion engine that recognizes the user's emotions. Each component of this system and its operation will be explained below.

[0761] System configuration

[0762] 1. User Device

[0763] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0764] Users operate the system through a dedicated application or a web browser.

[0765] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[0766] 2. Server

[0767] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0768] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[0769] 3. Database

[0770] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[0771] Program processing explanation

[0772] User terminal operation

[0773] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[0774] Server Operation

[0775] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. It then analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0776] Search and suggest similar cases

[0777] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then reported to experts, who can then conduct further detailed analysis or ask questions of the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations." The server also determines the importance of the case based on the user's emotions evaluated by the emotion engine, and adjusts the priority of countermeasures as necessary.

[0778] Identifying the root cause and proposing countermeasures

[0779] The server runs an algorithm to identify the root cause of near-miss incidents based on the expert analysis, past data, and emotion data. For example, it might determine that "the cause of a tool slipping from the hand was improper use of gloves." It then suggests appropriate countermeasures, such as "use non-slip gloves for certain tasks." If the user feels anxious, it will provide further detailed explanations and encourage them to take countermeasures.

[0780] Implementation and feedback input

[0781] The user implements the notified measures on-site and again inputs the results and effects into the device. For example, the user might input feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." At this time, the emotion engine recognizes the user's emotions again and collects emotional data after the measures have been implemented. The device then sends this information to the server, which stores it in a database.

[0782] Learning and accuracy improvement through machine learning

[0783] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information and emotional data. This allows for more accurate countermeasure proposals the next time a near-miss occurs, while also enabling flexible responses based on the user's emotional state.

[0784] Operation example

[0785] For example, if a tool slips from a user's hand while working at a height in a manufacturing plant, nearly hitting a worker below, the user can input the incident into their device. At the same time, the emotion engine recognizes the user's anxious facial expression. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures, providing feedback on the results while also recollecting their emotional state at the time. As new feedback and emotion data accumulate, the entire system is continually improved, resulting in more effective countermeasures and responses that take the user's psychology into consideration.

[0786] In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures, and also to respond flexibly while taking into consideration the user's feelings.

[0787] The processing flow will be explained below.

[0788] Step 1: User Action

[0789] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[0790] Users input details of near miss incidents that occurred on-site. For example, they might record an incident such as "While working at height, a tool slipped from their hand and nearly hit a worker below."

[0791] The user enters the importance and scope of the case, and then presses the send button.

[0792] Step 2: Device Operation

[0793] The terminal performs a format check on the data entered by the user to ensure there are no missing items or formatting errors.

[0794] Once you have verified that the data has been entered accurately, convert the input data into an appropriate data format, such as JSON.

[0795] The converted data is sent to the server.

[0796] Step 3: Receiving and storing on the server

[0797] The server receives the data sent from the terminal.

[0798] The received near-miss incident data is analyzed and stored in a database in an appropriate format.

[0799] Step 4: Emotion Engine in Action

[0800] The emotion engine installed in the device simultaneously recognizes the user's facial expressions and voice to collect emotional data.

[0801] If the user is expressing emotions such as anxiety, impatience, or anger, this information is also sent to the server.

[0802] Step 5: Search for similar cases

[0803] The server searches past near-miss incident cases stored in a database and extracts cases similar to the received case.

[0804] Natural language processing (NLP) techniques and machine learning algorithms are used to extract relevant information from similar cases.

[0805] Step 6: Initial analysis and expert notification

[0806] The server performs an initial analysis based on the search results for similar cases.

[0807] The initial analysis results and the user's emotional data are notified to the experts via email or a dedicated application.

[0808] Step 7: Expert Operation

[0809] The expert accesses the server and checks the notified initial analysis results and emotion data.

[0810] If necessary, the user is asked additional questions.

[0811] Step 8: Provide additional user information

[0812] The user answers questions from the expert and provides additional information.

[0813] Enter additional information and submit it to the server.

[0814] Step 9: Identify the root cause and propose a solution

[0815] The server identifies the root cause of near miss incidents based on the expert analysis results, additional information from users, and emotional data.

[0816] Based on the root cause, appropriate countermeasures are searched for in the database.

[0817] A countermeasure plan is generated and notified to the user.

[0818] Step 10: User implementation and feedback

[0819] The user implements the notified countermeasures on-site.

[0820] The effects of the implemented measures and the situation at the site are entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of dropped tools has decreased."

[0821] The emotion engine installed on the device will again recognize the user's emotions and collect emotional data after the activity.

[0822] Step 11: Gather feedback and learn

[0823] The server stores the received feedback and emotion data in a database.

[0824] Based on feedback data and sentiment data, the machine learning model is updated to improve the accuracy of analysis and countermeasure proposals.

[0825] Example 2

[0826] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] Conventional near-miss incident management systems primarily focus on case input and analysis, and lack the ability to propose flexible countermeasures that take into account the user's emotional state and learning from feedback. This makes it difficult to propose more effective countermeasures while reducing the user's psychological anxiety and stress. Furthermore, technology for retrieving similar cases and improving analysis accuracy is insufficient, limiting the effectiveness of the proposed countermeasures.

[0828] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0829] In this invention, the server includes a means for inputting near-miss incidents, a means for storing the input near-miss incidents in a database, and a means for analyzing the stored near-miss incidents and searching for similar incidents. This enables the user to quickly collect and manage near-miss incidents experienced in the field, and enables highly accurate analysis and searching for similar incidents. Furthermore, by including a means for recognizing the user's emotional state and collecting emotional data, a means for analyzing the collected emotional data and adjusting the importance of countermeasures, and a means for using machine learning to learn patterns and trends from feedback data and emotional data and improve the accuracy of countermeasure proposals, flexible and effective countermeasure proposals that take the user's psychological state into consideration are possible.

[0830] A "near miss incident" refers to an event or occurrence that did not result in a serious accident or trouble, but could have caused danger or problems.

[0831] "Input means" refers to the means by which a user inputs near-miss incidents and feedback information into a device.

[0832] "Database" refers to a system that centrally manages accumulated data such as near-miss incident cases, emotional data, and feedback information.

[0833] "Analysis means" refers to the technology used to analyze accumulated data, search for similar cases, and discover patterns.

[0834] The "similar case search means" refers to a means for searching for cases similar to the currently input case from data accumulated in the past.

[0835] "Means for identifying the root cause" refers to a means for analyzing the causes of the near-miss incidents entered and identifying the underlying problems.

[0836] "Measure suggestion means" refers to a means for proposing optimal solutions or measures to users based on the identified root cause.

[0837] The "feedback input means" refers to a means for a user to input the results and impressions of implementing the proposed measures back into the system.

[0838] "Emotion recognition means" refers to technology for collecting and analyzing emotional data from a user's facial expressions and voice.

[0839] "Emotion data analysis means" refers to a means for analyzing collected emotion data and adjusting the importance of countermeasures based on the analysis.

[0840] "Machine learning methods" refers to technologies that learn patterns and trends from feedback data and emotion data to improve the accuracy of analysis and countermeasure proposals.

[0841] "Natural language processing technology" refers to technology that analyzes accumulated text data to search for similar cases and assist in problem-solving.

[0842] This invention is a system that efficiently manages and analyzes near-miss incidents and takes more effective countermeasures by combining it with an emotion engine that recognizes user emotions. The system includes a series of processes from user input to countermeasure proposals, and also uses user emotion data for analysis.

[0843] System configuration

[0844] 1. User Device

[0845] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0846] Users operate the system through a dedicated application or a web browser.

[0847] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[0848] 2. Server

[0849] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0850] Cloud services may also be used, enabling high-speed data processing and large-volume data storage. The server uses Python and machine learning frameworks (such as TensorFlow and PyTorch) to analyze and visualize the data.

[0851] 3. Database

[0852] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[0853] Using a database system such as PostgreSQL enables efficient data management and fast searches.

[0854] Program operation explanation

[0855] User terminal operation

[0856] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[0857] Server Operation

[0858] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar cases by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact and provides relevant information to the user. Specifically, the analysis is performed using Python's NLTK library and Scikit-learn.

[0859] Examples of prompt statements

[0860] Here are some example prompts to input to a generative AI model:

[0861] When a user inputs a near-miss incident into a terminal, such as when a tool nearly fell while working at height, the emotion engine also identifies the user's anxious facial expression at the time. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using non-slip gloves" or "wearing a tool wristband." Please explain how this near-miss incident management system, which incorporates emotion engineering, works.

[0862] Using this prompt, the generative AI model is expected to provide a detailed explanation of the behavior of a specific system.

[0863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0864] Step 1:

[0865] User input of near miss incidents

[0866] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. Specifically, they use the device's keyboard or touch panel to enter details of the incident, such as "While working at height, a tool slipped from my hand and nearly hit a worker below." The input in this step is a record of the actual near-miss incident, and the output is the entered text data.

[0867] Step 2:

[0868] Emotion data collection using an emotion engine

[0869] An emotion engine built into the user device recognizes the user's facial expressions and voice. Specifically, the device's camera captures facial expressions, and the microphone analyzes the voice. For example, if the user looks anxious, that facial expression data is collected. The input for this step is the user's facial expression and voice, and the output is analyzed emotion data.

[0870] Step 3:

[0871] Data check and transmission by user terminal

[0872] The terminal performs a format check on the input near-miss case information and collected emotion data. Specifically, it checks the required fields on the input form and the consistency of the data format, and if there are no problems, it sends the data to the server. For example, it checks that the input case meets all required fields. The input for this step is near-miss case data and emotion data, and the output is the checked data.

[0873] Step 4:

[0874] Receiving and storing data by the server

[0875] The server receives the near-miss incidents and emotion data sent from the user's device and stores them in a database. Specifically, it receives the data using a RESTful API and saves it in a database such as PostgreSQL. The input to this step is the sent near-miss incident data and emotion data, and the output is the data saved in the database.

[0876] Step 5:

[0877] Server-based search and analysis of similar cases

[0878] The server analyzes the accumulated data and searches for similar cases. Specifically, it applies natural language processing (NLP) techniques and machine learning algorithms using Python's NLTK library and Scikit-learn. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact. The input for this step is the accumulated data, and the output is the search results and analysis results for similar cases.

[0879] Step 6:

[0880] Initial analysis and suggestions based on search results for similar cases

[0881] The server performs an initial analysis based on the search results. The results of this initial analysis are then sent to experts, who then conduct further detailed analysis or ask questions of the user. Specifically, the data is displayed in real time using a dedicated dashboard. The input to this step is the search results for similar cases, and the output is the initial analysis results.

[0882] Step 7:

[0883] Identifying the root cause by the server and proposing detailed countermeasures

[0884] The server runs an algorithm to identify the root cause based on the expert's analysis results, past data, and emotion data. For example, it might determine that "the cause of the tool slipping from the hand is improper use of gloves." It then suggests an appropriate countermeasure: "Use non-slip gloves for certain tasks." The input for this step is the initial analysis results and past data, and the output is a detailed countermeasure proposal.

[0885] Step 8:

[0886] User implementation and feedback

[0887] The user implements the notified countermeasures on-site and again inputs the results and effects into the terminal. Specifically, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The input in this step is the result of implementing the countermeasures, and the output is feedback data.

[0888] Step 9:

[0889] Server receives feedback and updates the learning model

[0890] The server updates the machine learning model using the newly accumulated feedback information and emotion data. Specifically, it retrains the model using TensorFlow and PyTorch. This enables more accurate countermeasure proposals for the next near-miss incident. The input for this step is the feedback data and emotion data, and the output is an updated machine learning model.

[0891] (Application example 2)

[0892] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0893] Conventional near-miss incident analysis systems did not efficiently manage and analyze near-miss incidents, nor did they evaluate risks or respond to them while taking into account user emotional data. As a result, countermeasure proposals did not take into account the user's psychological state, making it difficult to achieve both improved safety and user satisfaction. In addition, the effective use of feedback data was limited, limiting the improvement of accuracy.

[0894] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting near-miss incident cases; means for storing the input near-miss incident cases in a database; means for analyzing the stored near-miss incident cases and searching for similar incident cases; means for identifying root causes and proposing countermeasures based on the search results for similar incident cases; means for implementing the proposed countermeasures and inputting feedback therefrom; means for collecting and analyzing emotional data and reflecting it in risk assessment; means for adjusting priorities based on the user's emotional state when proposing countermeasures; and means for learning patterns and trends from the feedback data and emotional data using machine learning to improve the accuracy of the analysis and countermeasure proposals. This enables safety countermeasures based on near-miss incident cases to be flexible and effective, taking into account the user's emotions, enabling more accurate countermeasure proposals and improved user satisfaction.

[0895] A "near miss incident" refers to an incident that did not actually result in an accident in a factory or work site, but was potentially dangerous.

[0896] "Storage" means systematically collecting and storing data and information.

[0897] "Analysis" refers to the process of investigating and analyzing data in detail to clarify its background and causes.

[0898] A "similar case" refers to an event that occurred in the past that shares characteristics or patterns with the current case.

[0899] A "root cause" is the direct cause that causes a certain phenomenon or result.

[0900] "Countermeasures" refer to the means or methods taken to prevent or resolve a specific problem or danger.

[0901] "Feedback" refers to the process of providing an evaluation or response to actions or results, and making improvements or corrections based on that information.

[0902] "Emotional data" refers to data that expresses the user's emotions and psychological state, which can be obtained from facial expressions, voice, posture, etc.

[0903] "Risk assessment" refers to the process of evaluating the dangers of specific tasks or situations and determining the necessity and priority of countermeasures.

[0904] "Priority" refers to determining the order in which multiple tasks or problems should be processed first.

[0905] "Machine learning" is a field of artificial intelligence that allows computers to find patterns in data and make predictions and decisions based on them.

[0906] The system for implementing this invention is designed to manage near-miss incidents and improve safety within factories. This system consists of a series of steps: inputting near-miss incidents, collecting and analyzing emotional data, storing and searching data, proposing appropriate countermeasures, and collecting feedback.

[0907] System Components

[0908] 1. User Device

[0909] Input method: Users input near-miss incidents that occur in the factory. For example, they can input an incident such as "a tool slipped out of the hand while working at height" using a smartphone, tablet, or dedicated device.

[0910] Emotion data collection method: A camera and microphone are built into the device to collect the user's facial expressions and voice. This allows for the collection of emotional data such as the user's anxiety or surprise. Software such as Affectiva is used as the emotion engine.

[0911] 2. Server

[0912] Data storage method: Near-miss incidents and emotion data sent from user devices are received and stored in a database. Large volumes of data can be stored using cloud services.

[0913] Similar case search method: Apply natural language processing (NLP) technology and machine learning algorithms to past near-miss case data to search for similar cases. For example, libraries such as Scikit-learn and TensorFlow are used.

[0914] Countermeasure suggestion method: Based on the search results of similar cases, appropriate countermeasures are suggested. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[0915] Feedback collection method: The results of the measures taken by users and their emotional data at the time are collected again and stored in a database.

[0916] 3. Machine Learning Models

[0917] Learning and accuracy improvement measures: Using accumulated feedback data and sentiment data, the machine learning model is updated to improve the accuracy of countermeasure proposals.

[0918] Examples and prompts

[0919] Example 1:

[0920] Near miss case: While working at height, a tool slipped from the worker's hand and nearly hit a worker below.

[0921] Emotion data collection: Detecting "anxiety" from the operator's voice

[0922] Similar case search results: Past cases and the suggestion to "use non-slip gloves" as a countermeasure

[0923] Example 2:

[0924] Near miss case: Heavy machinery almost crashed into a wall due to an operating error

[0925] Emotion data collection: Detecting "surprise" from the operator's facial expression

[0926] Similar case search results: Past cases and suggested countermeasures such as "sensor readjustment" and "additional safety checks"

[0927] Prompt Sentence Examples

[0928] User submitted example: While working at height, a tool slipped from his hand and nearly hit a worker below.

[0929] Emotional data: Anxiety

[0930] Analysis result: This is a case where a tool fell while working at height. Based on past data, we suggest using non-slip gloves. Please take this measure.

[0931] In this way, this system efficiently carries out a series of steps, from managing near-miss incidents to proposing countermeasures and providing feedback, in order to improve safety within the factory, and is able to take flexible and effective measures that take into account the user's emotional state.

[0932] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0933] Step 1:

[0934] Users input cases of near misses that have occurred in the factory into the terminal. For example, they can use a smartphone, tablet, or dedicated terminal to input an example in text format, such as "A tool slipped out of my hand while working at height."

[0935] Input: Near miss incidents entered by the user

[0936] Output: Near miss incidents in text format

[0937] Step 2:

[0938] The user's facial expressions and voice are collected using a camera and microphone installed on the user's device, and the emotional data is analyzed by an emotion engine (e.g., Affectiva), which obtains emotional data such as the user's anxiety or surprise.

[0939] Input: User facial and voice data

[0940] Output: Parsed emotion data

[0941] Step 3:

[0942] The device sends the collected near-miss incidents and emotion data to a server, where the data is checked for consistency in data format and then stored in a database.

[0943] Input: Submitted near-miss incident and emotion data

[0944] Output: Near miss incidents and emotion data stored in a database

[0945] Step 4:

[0946] The server analyzes the accumulated near-miss incident cases and searches for similar cases using natural language processing (NLP) technology and machine learning algorithms (such as Scikit-learn and TensorFlow). For example, it searches to see if there have been any past cases of "a tool falling while working at height."

[0947] Input: Near miss incidents stored in the database

[0948] Output: Search results for similar cases

[0949] Step 5:

[0950] The server identifies the root cause based on the search results of similar cases and suggests appropriate countermeasures. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[0951] Input: Search results for similar cases and sentiment data

[0952] Output: List of proposed measures

[0953] Step 6:

[0954] The server notifies the user of the proposed measures. The user implements the measures and inputs the results into the terminal again. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of tools dropped has decreased."

[0955] Input: Proposed measures and user implementation results

[0956] Output: Input feedback data

[0957] Step 7:

[0958] The device then sends the collected feedback and emotion data to a server where it is stored in a database. The stored data is used for further analysis and countermeasure proposals.

[0959] Input: Feedback and emotion data

[0960] Output: Feedback and emotion data stored in a database

[0961] Step 8:

[0962] The server updates the machine learning model using the accumulated new feedback and emotion data, improving the accuracy of the next countermeasure proposal and enabling flexible countermeasures to be taken based on the user's emotional state.

[0963] Input: Newly accumulated feedback and emotion data

[0964] Output: An updated machine learning model

[0965] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0966] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0967] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0968] [Third embodiment]

[0969] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0970] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0971] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0972] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0973] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0974] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0975] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0976] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0977] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0978] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0979] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0980] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0981] The system for carrying out the present invention is for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Below, each component of the system and its operation will be explained.

[0982] System configuration

[0983] 1. User Device

[0984] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[0985] Users operate the system through a dedicated application or a web browser.

[0986] 2. Server

[0987] This is a central system that receives data sent from user terminals and stores and analyzes it.

[0988] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[0989] 3. Database

[0990] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback.

[0991] Program processing explanation

[0992] User terminal operation

[0993] Users input examples of near misses that occurred on-site into a dedicated application or web form. For example, they can input an example such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[0994] Server Operation

[0995] The server receives near-miss incidents sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[0996] Search and suggest similar cases

[0997] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to an expert, who can then conduct a more detailed analysis or ask questions to the user. For example, the expert can provide the user with information such as "what countermeasures have been effective in similar situations."

[0998] Identifying the root cause and proposing countermeasures

[0999] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[1000] Implementation and feedback input

[1001] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1002] Learning and accuracy improvement through machine learning

[1003] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1004] Operation example

[1005] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[1006] In this way, the system of the present invention is able to quickly and effectively manage and take action against near miss incidents.

[1007] The processing flow will be explained below.

[1008] Step 1: User Action

[1009] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[1010] Users input details of near miss incidents that occurred on-site. For example, they record an incident such as "While working at height, a tool slipped from my hand and nearly hit a worker below."

[1011] The user completes the data entry and presses the submit button.

[1012] Step 2: Device Operation

[1013] The terminal performs a format check on the entered data to ensure there are no missing items or formatting errors.

[1014] After verifying that the data has been entered correctly, convert the input data into an appropriate data format, such as JSON.

[1015] The converted data is sent to the server.

[1016] Step 3: Receiving and storing on the server

[1017] The server receives the data sent from the terminal.

[1018] The received data is analyzed and stored in a database in an appropriate format.

[1019] Step 4: Search for similar cases

[1020] The server searches the accumulated database and extracts past cases similar to the received near-miss case.

[1021] Apply natural language processing (NLP) techniques and machine learning algorithms to identify similar cases and their countermeasures.

[1022] Step 5: Initial analysis and expert notification

[1023] The server performs an initial analysis based on the search results for similar cases.

[1024] The initial analysis results are notified to experts via email or a dedicated application.

[1025] Step 6: Expert Operation

[1026] The expert accesses the server and checks the notified initial analysis results.

[1027] If necessary, the user is asked additional questions.

[1028] Step 7: Provide additional user information

[1029] The user answers questions from the expert and provides additional information.

[1030] Enter additional information and submit it to the server.

[1031] Step 8: Identify the root cause and propose a solution

[1032] The server identifies the true cause of near miss incidents based on the expert analysis results and additional information from users.

[1033] Based on the root cause, appropriate countermeasures are searched for in the database.

[1034] A countermeasure plan is generated and notified to the user.

[1035] Step 9: User implementation and feedback

[1036] The user implements the notified countermeasures on-site.

[1037] The effects of the implemented measures and the situation at the site are again entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of tools being dropped has decreased."

[1038] The user terminal transmits the feedback to the server.

[1039] Step 10: Gather feedback and learn

[1040] The server stores the received feedback in a database.

[1041] The machine learning model is updated based on the feedback data and learns to improve the accuracy of system analysis and countermeasure proposals.

[1042] Example 1

[1043] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1044] Near miss incidents at work sites are important information for preventing serious accidents, but their collection, management, and analysis depend on human resources, making them inefficient and difficult to contribute to overall safety measures.In particular, when there are a large number of near miss incidents, it is difficult to search for similar incidents and propose appropriate countermeasures, which results in the problem of accident prevention measures not being implemented effectively.

[1045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1046] In this invention, the server includes: means for a user to input near-miss incident cases; means for checking the format of the near-miss incident cases input at the terminal; means for transmitting data to the server; means for storing the data received by the server in a database; means for analyzing the stored data using natural language processing technology and machine learning algorithms and searching for similar incidents; means for notifying experts based on the search results; means for the experts to analyze the analysis results in detail and provide feedback to the user; means for the server to identify the root cause based on the expert feedback and propose countermeasures; means for the user to implement the proposed countermeasures and provide feedback on the results; and means for the server to update the machine learning model based on newly accumulated feedback information to improve accuracy. This makes it possible to efficiently collect, manage, and analyze near-miss incident cases and take prompt and appropriate countermeasures.

[1047] A "user terminal" is a device for transmitting near-miss incidents input by a user to a server.

[1048] A "near miss case" refers to an incident that nearly developed into an accident or disaster but was prevented.

[1049] "Format check" is the process of checking whether near miss cases are entered in the correct format.

[1050] A "server" is a central data processing system that receives, analyzes, and stores data sent from user terminals.

[1051] The "database" is a system that systematically stores and manages information such as near-miss incident cases, countermeasures, and feedback.

[1052] "Natural language processing technology" is a technology for understanding and analyzing human language.

[1053] A "machine learning algorithm" is a mathematical method for learning from data and making predictions or classifications.

[1054] "Searching for similar cases" is the process of finding past cases similar to a newly entered near-miss case from the database.

[1055] "Expert notification" is the process of communicating the results of the initial analysis to experts and soliciting their feedback for further analysis.

[1056] "Root cause identification" is an analytical process for identifying the underlying cause of a near miss incident.

[1057] "Proposing measures" is the process of presenting appropriate measures to eliminate the identified root causes.

[1058] "Feedback input" is the process in which the user inputs the results and effects of the measures they have implemented back into the system.

[1059] "Machine learning model updating" is the process of retraining machine learning algorithms based on newly accumulated data to improve the accuracy of the system.

[1060] The present invention is a system for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Each component of the present invention and its operation will be specifically described below.

[1061] System configuration

[1062] 1. User Device

[1063] A device that allows users to input near-miss incidents and provide feedback on countermeasures. Specifically, it can be a tablet, smartphone, or PC.

[1064] Users operate the system through a dedicated application or a web browser.

[1065] 2. Server

[1066] This is a central system that receives data sent from user devices and stores and analyzes it. Specifically, it uses cloud services. A server with high-performance data processing capabilities is desirable.

[1067] 3. Database

[1068] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback. SQL databases and NoSQL databases are used.

[1069] System processing flow

[1070] User terminal

[1071] Users can input near-miss incidents into a dedicated application or web form at the work site. For example, they can input an incident such as, "While working at height, a tool slipped from my hand and nearly hit a worker below." The device checks the format of this input information and, if there are no problems, sends it to the server.

[1072] server

[1073] The server receives near-miss incidents sent from user devices and stores them in a database. It then applies natural language processing (NLP) technology and machine learning algorithms to analyze the stored data and search for similar incidents. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts, and provides relevant information to the user.

[1074] Search and suggest similar cases

[1075] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to experts, who can then conduct further detailed analysis or ask questions to the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations."

[1076] Identifying the root cause and proposing countermeasures

[1077] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[1078] Implementation and feedback input

[1079] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1080] Learning and accuracy improvement through machine learning

[1081] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1082] Operation example

[1083] For example, if a tool slips from a user's hand while working at height in a manufacturing plant and nearly hits a worker below, the user can input the incident into a terminal. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and receives feedback on the results, allowing the entire system to be improved over time. In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures.

[1084] Prompt Sentence Examples

[1085] An example of a prompt sentence to be input to the generative AI model is shown below.

[1086] Please explain how you propose specific countermeasures when a user enters a near-miss incident case. Case: "While working at height, a tool slips from your hand and may hit a worker below."

[1087] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1088] Step 1:

[1089] The user inputs a near-miss incident.

[1090] Users use a dedicated application or web form to enter near-miss incidents that have occurred at the work site. The data entered includes details of the incident, date and time, and on-site conditions. An example of an input would be, "While working at height, a tool slipped from my hand and nearly hit a worker below."

[1091] Input: Near miss incidents input by users

[1092] Output: Input data that passes format check

[1093] Step 2:

[1094] Check the format of input data on the terminal

[1095] The terminal checks whether the data entered by the user conforms to the specified format. For example, it checks whether all required fields have been entered and whether the data format is correct. If the check passes, it proceeds to the next process.

[1096] Input: Near miss incidents entered by the user

[1097] Output: Data formatted to be sent to the server

[1098] Example: Check whether required fields such as "Event details," "Date and time," and "On-site situation" have been entered, and if any fields are missing, display a warning to the user.

[1099] Step 3:

[1100] Sending data to the server

[1101] The terminal sends data that has passed the format check to the server. At this time, the data is encrypted and sent using the HTTPS protocol to ensure data security.

[1102] Input: Data that passes format check

[1103] Output: Data sent to the server

[1104] Example: Encrypting data entered by a user and sending it to a server using HTTPS.

[1105] Step 4:

[1106] Receiving and storing data on the server

[1107] The server receives the data sent from the device and stores it in a database, which can then be used for analysis and retrieval.

[1108] Input: Data sent from the terminal

[1109] Output: Near miss cases stored in the database

[1110] Example: The server immediately stores the received data in the database and simultaneously backs up the data.

[1111] Step 5:

[1112] Data analysis and similar case search on the server

[1113] The server analyzes the accumulated data and searches for similar cases using natural language processing technology and machine learning algorithms. Specifically, it finds past cases of "tools falling while working at height" and analyzes countermeasures and impacts.

[1114] Input: Near miss cases stored in the database

[1115] Output: Search results for similar cases and initial analysis results

[1116] Example: Natural language processing techniques are used to tokenize example sentences entered by users, and similarities are calculated using machine learning algorithms.

[1117] Step 6:

[1118] Notification from the server to the expert

[1119] The server performs an initial analysis based on the search results for similar cases and notifies the experts of the results, who then begin a detailed analysis.

[1120] Input: Search results for similar cases and initial analysis results

[1121] Output:Notify Expert

[1122] Example: The server notifies the search results and initial analysis results via email or an expert dashboard.

[1123] Step 7:

[1124] Detailed analysis and feedback from experts

[1125] The expert conducts a detailed analysis and provides additional questions and feedback to the user, such as "What kind of gloves were you using?", and develops detailed countermeasures.

[1126] Input: Expert notification data and initial analysis results

[1127] Output: Detailed analysis by experts and feedback to users

[1128] Example: Experts can review detailed case studies from the dashboard and use the comments feature to ask users additional questions.

[1129] Step 8:

[1130] Identifying the root cause and proposing countermeasures by the server

[1131] Based on the expert analysis and past data, the server uses an algorithm to identify the root cause of near-miss incidents and develop appropriate countermeasures. For example, if the root cause is "inappropriate glove selection," the server will suggest specific countermeasures such as "using non-slip gloves."

[1132] Input: Expert detailed analysis and historical data

[1133] Output: Identified root cause and proposed measures

[1134] Example: The server runs an algorithm to find the root cause of the case and generate effective countermeasures as a proposal.

[1135] Step 9:

[1136] User implementation of measures and feedback

[1137] The user implements the measures in the field and then inputs the results and effects into the terminal again. For example, the user inputs feedback such as, "Using non-slip gloves has reduced the number of tools being dropped."

[1138] Input: Countermeasure proposals from the server, user feedback

[1139] Output: Data on the results of measures taken

[1140] Example: The user inputs the situation after implementing the countermeasures into the terminal and sends it to the server.

[1141] Step 10:

[1142] Server-based feedback accumulation and machine learning model updates

[1143] The server accumulates the feedback sent by users in a database and updates the machine learning model based on that information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1144] Input: Feedback data from users

[1145] Output: Updated machine learning model

[1146] Example: The server uses new feedback data to retrain the machine learning model, improving the accuracy of the system.

[1147] (Application example 1)

[1148] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1149] Near miss incidents can occur frequently in work environments such as factories. It is necessary to properly manage and analyze these incidents and implement appropriate countermeasures, but conventional systems often fail to adequately detect dangers in real time or propose specific countermeasures. Furthermore, there is a lack of means to efficiently collect worker feedback and improve the system based on the analysis results. Therefore, there is a need for a system that can efficiently manage and address near miss incidents while ensuring worker safety.

[1150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1151] In this invention, the server includes means for inputting near-miss incident cases, means for storing the input near-miss incident cases in a database, means for analyzing the stored near-miss incident cases and searching for similar incidents, means for identifying root causes and proposing countermeasures based on the search results for similar incidents, means for implementing the proposed countermeasures and inputting feedback thereon, means for performing real-time monitoring in the factory, automatically detecting potential hazards, and proposing countermeasures, and means for functioning as an application installed on automated work equipment in the factory. This enables efficient management of near-miss incident cases and countermeasures in the work environment, and ensures the safety of workers.

[1152] A "near miss case" is an incident that comes just short of causing an accident or trouble in the work environment.

[1153] A "database" is a system that manages accumulated information in an organized manner and enables it to be quickly searched and used.

[1154] "Real-time monitoring" is a method of constantly monitoring the working environment and equipment status, and acquiring and analyzing data instantly.

[1155] "Automated work equipment" refers to machines and robots used in work environments such as factories.

[1156] "Feedback" refers to reports and opinions about the results and effectiveness of measures.

[1157] "Similar cases" refer to past near-miss cases that are similar to the current case.

[1158] A "remedy proposal" is a recommendation of appropriate action or change in response to a specific problem or risk.

[1159] "Analysis" involves examining accumulated data in detail and deriving meaning and patterns from it.

[1160] An "input device" is a device used by a user to input data.

[1161] "Machine learning" is a technology that allows computers to autonomously learn based on empirical data and make judgments and predictions.

[1162] A "pattern or trend" is a recurring characteristic or direction of variation in data.

[1163] MODE FOR CARRYING OUT THE INVENTION

[1164] The system for implementing the present invention efficiently manages and analyzes near-miss incidents in a factory environment and automatically suggests appropriate countermeasures. Each component of the system and its operation will be described below.

[1165] System configuration

[1166] 1. User Device

[1167] This device allows workers to input near-miss incidents and provides feedback on countermeasures.

[1168] Users operate the device through a dedicated application or a web browser, such as on a tablet or smartphone.

[1169] 2. Real-time monitoring device

[1170] The working environment and working conditions are constantly monitored using sensors and cameras mounted on automated work equipment (robot arms, etc.) in the factory.

[1171] The acquired data is immediately sent to a server where it is analyzed to detect potential dangers.

[1172] 3. Server

[1173] It is a central system that receives, stores, and analyzes data sent from user terminals and real-time monitoring devices.

[1174] Using cloud services (e.g., Amazon Web Services, Google Cloud Platform) enables high-speed data processing and large-volume data storage.

[1175] 4. Database

[1176] It is an information management system connected to a server that stores data such as near-miss incidents, real-time monitoring data, countermeasures, and feedback.

[1177] Program processing explanation

[1178] User terminal operation

[1179] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they can enter an incident such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[1180] Real-time monitoring in action

[1181] Sensors and cameras attached to the automated work equipment monitor the work situation in real time and send the acquired data to a server, which analyzes the data and detects potential hazards.

[1182] Server Operation

[1183] The server receives data sent from user devices and real-time monitoring devices and stores it in a database. It then analyzes the stored data and searches for similar cases, applying natural language processing (NLP) techniques (e.g., spaCy, NLTK) and machine learning algorithms (e.g., Scikit-learn, TensorFlow). For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[1184] Search and suggest similar cases

[1185] The server searches for similar cases and performs an initial analysis based on the results. Based on the results of this initial analysis, the server proposes countermeasures. For example, it provides information to the user such as "In similar situations, using non-slip gloves was effective."

[1186] Implementation and feedback input

[1187] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1188] Learning and accuracy improvement through machine learning

[1189] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1190] Examples of concrete examples and prompts

[1191] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[1192] Example prompt sentence:

[1193] "While working at height, a tool almost slipped out of my hand. Please suggest the best solution to this problem based on similar cases from the past. We will also collect feedback to evaluate the effectiveness of the solution."

[1194] In this way, the system of the present invention is able to quickly and effectively manage and address near miss incidents in a factory environment.

[1195] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1196] Step 1:

[1197] Input of near miss cases from user terminals

[1198] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they might enter, "While working at height, a tool nearly slipped out of my hand." The entered data is checked for format and, if there are no problems, is sent to the server.

[1199] Input: Near miss incident details

[1200] Output: Send data to the server

[1201] Step 2:

[1202] Data reception and storage on the server

[1203] The server stores the near-miss incident data received from the user terminal in a database, which also stores past near-miss incidents and countermeasures.

[1204] Input: Near miss incident data sent from the user's device

[1205] Output: Data accumulation in database

[1206] Step 3:

[1207] Real-time monitoring data collection

[1208] Sensors and cameras installed on automated work equipment in factories monitor the work situation and environment in real time, and the acquired data is immediately sent to a server.

[1209] Input: Real-time data from automated work equipment

[1210] Output: Send data to the server

[1211] Step 4:

[1212] Data analysis on the server

[1213] The server uses natural language processing (NLP) techniques and machine learning algorithms (e.g., spaCy, NLTK, Scikit-learn, TensorFlow) to analyze the accumulated data and real-time monitoring data, search for similar past cases, and extract information to propose appropriate countermeasures.

[1214] Input: Database and real-time monitoring data

[1215] Output: Analysis results and countermeasures

[1216] Step 5:

[1217] Search for similar cases and propose countermeasures

[1218] The server searches for similar cases based on the analysis results and provides the user with countermeasures and their impact. For example, if there is a case of a tool slipping off in the past, the server will suggest the use of non-slip gloves.

[1219] Input: Analysis results

[1220] Output: Countermeasure proposal

[1221] Step 6:

[1222] Implementing measures and providing feedback

[1223] The user implements the proposed measures on-site and then inputs the results and effects into the terminal again. For example, the user might input, "Using non-slip gloves resulted in fewer dropped tools." The feedback information is then sent to the server.

[1224] Input: Feedback after implementing measures

[1225] Output: Send feedback to the server

[1226] Step 7:

[1227] Accumulating feedback data and updating machine learning models

[1228] The server accumulates the feedback data in a database and updates the machine learning model, enabling more accurate countermeasure proposals the next time a near-miss occurs.

[1229] Input: Feedback data

[1230] Output: Database and machine learning model updates

[1231] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1232] The system for implementing this invention efficiently manages and analyzes near-miss incidents and takes more effective measures by combining it with an emotion engine that recognizes the user's emotions. Each component of this system and its operation will be explained below.

[1233] System configuration

[1234] 1. User Device

[1235] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[1236] Users operate the system through a dedicated application or a web browser.

[1237] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[1238] 2. Server

[1239] This is a central system that receives data sent from user terminals and stores and analyzes it.

[1240] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[1241] 3. Database

[1242] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[1243] Program processing explanation

[1244] User terminal operation

[1245] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[1246] Server Operation

[1247] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. It then analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[1248] Search and suggest similar cases

[1249] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then reported to experts, who can then conduct further detailed analysis or ask questions of the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations." The server also determines the importance of the case based on the user's emotions evaluated by the emotion engine, and adjusts the priority of countermeasures as necessary.

[1250] Identifying the root cause and proposing countermeasures

[1251] The server runs an algorithm to identify the root cause of near-miss incidents based on the expert analysis, past data, and emotion data. For example, it might determine that "the cause of a tool slipping from the hand was improper use of gloves." It then suggests appropriate countermeasures, such as "use non-slip gloves for certain tasks." If the user feels anxious, it will provide further detailed explanations and encourage them to take countermeasures.

[1252] Implementation and feedback input

[1253] The user implements the notified measures on-site and again inputs the results and effects into the device. For example, the user might input feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." At this time, the emotion engine recognizes the user's emotions again and collects emotional data after the measures have been implemented. The device then sends this information to the server, which stores it in a database.

[1254] Learning and accuracy improvement through machine learning

[1255] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information and emotional data. This allows for more accurate countermeasure proposals the next time a near-miss occurs, while also enabling flexible responses based on the user's emotional state.

[1256] Operation example

[1257] For example, if a tool slips from a user's hand while working at a height in a manufacturing plant, nearly hitting a worker below, the user can input the incident into their device. At the same time, the emotion engine recognizes the user's anxious facial expression. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures, providing feedback on the results while also recollecting their emotional state at the time. As new feedback and emotion data accumulate, the entire system is continually improved, resulting in more effective countermeasures and responses that take the user's psychology into consideration.

[1258] In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures, and also to respond flexibly while taking into consideration the user's feelings.

[1259] The processing flow will be explained below.

[1260] Step 1: User Action

[1261] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[1262] Users input details of near miss incidents that occurred on-site. For example, they might record an incident such as "While working at height, a tool slipped from their hand and nearly hit a worker below."

[1263] The user enters the importance and scope of the case, and then presses the send button.

[1264] Step 2: Device Operation

[1265] The terminal performs a format check on the data entered by the user to ensure there are no missing items or formatting errors.

[1266] Once you have verified that the data has been entered accurately, convert the input data into an appropriate data format, such as JSON.

[1267] The converted data is sent to the server.

[1268] Step 3: Receiving and storing on the server

[1269] The server receives the data sent from the terminal.

[1270] The received near-miss incident data is analyzed and stored in a database in an appropriate format.

[1271] Step 4: Emotion Engine in Action

[1272] The emotion engine installed in the device simultaneously recognizes the user's facial expressions and voice to collect emotional data.

[1273] If the user is expressing emotions such as anxiety, impatience, or anger, this information is also sent to the server.

[1274] Step 5: Search for similar cases

[1275] The server searches past near-miss incident cases stored in a database and extracts cases similar to the received case.

[1276] Natural language processing (NLP) techniques and machine learning algorithms are used to extract relevant information from similar cases.

[1277] Step 6: Initial analysis and expert notification

[1278] The server performs an initial analysis based on the search results for similar cases.

[1279] The initial analysis results and the user's emotional data are notified to the experts via email or a dedicated application.

[1280] Step 7: Expert Operation

[1281] The expert accesses the server and checks the notified initial analysis results and emotion data.

[1282] If necessary, the user is asked additional questions.

[1283] Step 8: Provide additional user information

[1284] The user answers questions from the expert and provides additional information.

[1285] Enter additional information and submit it to the server.

[1286] Step 9: Identify the root cause and propose a solution

[1287] The server identifies the root cause of near miss incidents based on the expert analysis results, additional information from users, and emotional data.

[1288] Based on the root cause, appropriate countermeasures are searched for in the database.

[1289] A countermeasure plan is generated and notified to the user.

[1290] Step 10: User implementation and feedback

[1291] The user implements the notified countermeasures on-site.

[1292] The effects of the implemented measures and the situation at the site are entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of dropped tools has decreased."

[1293] The emotion engine installed on the device will again recognize the user's emotions and collect emotional data after the activity.

[1294] Step 11: Gather feedback and learn

[1295] The server stores the received feedback and emotion data in a database.

[1296] Based on feedback data and sentiment data, the machine learning model is updated to improve the accuracy of analysis and countermeasure proposals.

[1297] Example 2

[1298] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1299] Conventional near-miss incident management systems primarily focus on case input and analysis, and lack the ability to propose flexible countermeasures that take into account the user's emotional state and learning from feedback. This makes it difficult to propose more effective countermeasures while reducing the user's psychological anxiety and stress. Furthermore, technology for retrieving similar cases and improving analysis accuracy is insufficient, limiting the effectiveness of the proposed countermeasures.

[1300] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1301] In this invention, the server includes a means for inputting near-miss incidents, a means for storing the input near-miss incidents in a database, and a means for analyzing the stored near-miss incidents and searching for similar incidents. This enables the user to quickly collect and manage near-miss incidents experienced in the field, and enables highly accurate analysis and searching for similar incidents. Furthermore, by including a means for recognizing the user's emotional state and collecting emotional data, a means for analyzing the collected emotional data and adjusting the importance of countermeasures, and a means for using machine learning to learn patterns and trends from feedback data and emotional data and improve the accuracy of countermeasure proposals, flexible and effective countermeasure proposals that take the user's psychological state into consideration are possible.

[1302] A "near miss incident" refers to an event or occurrence that did not result in a serious accident or trouble, but could have caused danger or problems.

[1303] "Input means" refers to the means by which a user inputs near-miss incidents and feedback information into a device.

[1304] "Database" refers to a system that centrally manages accumulated data such as near-miss incident cases, emotional data, and feedback information.

[1305] "Analysis means" refers to the technology used to analyze accumulated data, search for similar cases, and discover patterns.

[1306] The "similar case search means" refers to a means for searching for cases similar to the currently input case from data accumulated in the past.

[1307] "Means for identifying the root cause" refers to a means for analyzing the causes of the near-miss incidents entered and identifying the underlying problems.

[1308] "Measure suggestion means" refers to a means for proposing optimal solutions or measures to users based on the identified root cause.

[1309] The "feedback input means" refers to a means for a user to input the results and impressions of implementing the proposed measures back into the system.

[1310] "Emotion recognition means" refers to technology for collecting and analyzing emotional data from a user's facial expressions and voice.

[1311] "Emotion data analysis means" refers to a means for analyzing collected emotion data and adjusting the importance of countermeasures based on the analysis.

[1312] "Machine learning methods" refers to technologies that learn patterns and trends from feedback data and emotion data to improve the accuracy of analysis and countermeasure proposals.

[1313] "Natural language processing technology" refers to technology that analyzes accumulated text data to search for similar cases and assist in problem-solving.

[1314] This invention is a system that efficiently manages and analyzes near-miss incidents and takes more effective countermeasures by combining it with an emotion engine that recognizes user emotions. The system includes a series of processes from user input to countermeasure proposals, and also uses user emotion data for analysis.

[1315] System configuration

[1316] 1. User Device

[1317] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[1318] Users operate the system through a dedicated application or a web browser.

[1319] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[1320] 2. Server

[1321] This is a central system that receives data sent from user terminals and stores and analyzes it.

[1322] Cloud services may also be used, enabling high-speed data processing and large-volume data storage. The server uses Python and machine learning frameworks (such as TensorFlow and PyTorch) to analyze and visualize the data.

[1323] 3. Database

[1324] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[1325] Using a database system such as PostgreSQL enables efficient data management and fast searches.

[1326] Program operation explanation

[1327] User terminal operation

[1328] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[1329] Server Operation

[1330] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar cases by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact and provides relevant information to the user. Specifically, the analysis is performed using Python's NLTK library and Scikit-learn.

[1331] Examples of prompt statements

[1332] Here are some example prompts to input to a generative AI model:

[1333] When a user inputs a near-miss incident into a terminal, such as when a tool nearly fell while working at height, the emotion engine also identifies the user's anxious facial expression at the time. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using non-slip gloves" or "wearing a tool wristband." Please explain how this near-miss incident management system, which incorporates emotion engineering, works.

[1334] Using this prompt, the generative AI model is expected to provide a detailed explanation of the behavior of a specific system.

[1335] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1336] Step 1:

[1337] User input of near miss incidents

[1338] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. Specifically, they use the device's keyboard or touch panel to enter details of the incident, such as "While working at height, a tool slipped from my hand and nearly hit a worker below." The input in this step is a record of the actual near-miss incident, and the output is the entered text data.

[1339] Step 2:

[1340] Emotion data collection using an emotion engine

[1341] An emotion engine built into the user device recognizes the user's facial expressions and voice. Specifically, the device's camera captures facial expressions, and the microphone analyzes the voice. For example, if the user looks anxious, that facial expression data is collected. The input for this step is the user's facial expression and voice, and the output is analyzed emotion data.

[1342] Step 3:

[1343] Data check and transmission by user terminal

[1344] The terminal performs a format check on the input near-miss case information and collected emotion data. Specifically, it checks the required fields on the input form and the consistency of the data format, and if there are no problems, it sends the data to the server. For example, it checks that the input case meets all required fields. The input for this step is near-miss case data and emotion data, and the output is the checked data.

[1345] Step 4:

[1346] Receiving and storing data by the server

[1347] The server receives the near-miss incidents and emotion data sent from the user's device and stores them in a database. Specifically, it receives the data using a RESTful API and saves it in a database such as PostgreSQL. The input to this step is the sent near-miss incident data and emotion data, and the output is the data saved in the database.

[1348] Step 5:

[1349] Server-based search and analysis of similar cases

[1350] The server analyzes the accumulated data and searches for similar cases. Specifically, it applies natural language processing (NLP) techniques and machine learning algorithms using Python's NLTK library and Scikit-learn. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact. The input for this step is the accumulated data, and the output is the search results and analysis results for similar cases.

[1351] Step 6:

[1352] Initial analysis and suggestions based on search results for similar cases

[1353] The server performs an initial analysis based on the search results. The results of this initial analysis are then sent to experts, who then conduct further detailed analysis or ask questions of the user. Specifically, the data is displayed in real time using a dedicated dashboard. The input to this step is the search results for similar cases, and the output is the initial analysis results.

[1354] Step 7:

[1355] Identifying the root cause by the server and proposing detailed countermeasures

[1356] The server runs an algorithm to identify the root cause based on the expert's analysis results, past data, and emotion data. For example, it might determine that "the cause of the tool slipping from the hand is improper use of gloves." It then suggests an appropriate countermeasure: "Use non-slip gloves for certain tasks." The input for this step is the initial analysis results and past data, and the output is a detailed countermeasure proposal.

[1357] Step 8:

[1358] User implementation and feedback

[1359] The user implements the notified countermeasures on-site and again inputs the results and effects into the terminal. Specifically, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The input in this step is the result of implementing the countermeasures, and the output is feedback data.

[1360] Step 9:

[1361] Server receives feedback and updates the learning model

[1362] The server updates the machine learning model from the newly accumulated feedback information and emotion data. Specifically, it retrains the model using TensorFlow and PyTorch. This enables more accurate countermeasure proposals for the next near-miss incident. The input for this step is the feedback data and emotion data, and the output is an updated machine learning model.

[1363] (Application example 2)

[1364] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1365] Conventional near-miss incident analysis systems did not efficiently manage and analyze near-miss incidents, nor did they evaluate risks or respond to them while taking into account user emotional data. As a result, countermeasure proposals did not take into account the user's psychological state, making it difficult to achieve both improved safety and user satisfaction. In addition, the effective use of feedback data was limited, limiting the improvement of accuracy.

[1366] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting near-miss incident cases; means for storing the input near-miss incident cases in a database; means for analyzing the stored near-miss incident cases and searching for similar incident cases; means for identifying root causes and proposing countermeasures based on the search results for similar incident cases; means for implementing the proposed countermeasures and inputting feedback therefrom; means for collecting and analyzing emotional data and reflecting it in risk assessment; means for adjusting priorities based on the user's emotional state when proposing countermeasures; and means for learning patterns and trends from the feedback data and emotional data using machine learning to improve the accuracy of the analysis and countermeasure proposals. This enables safety countermeasures based on near-miss incident cases to be flexible and effective, taking into account the user's emotions, enabling more accurate countermeasure proposals and improved user satisfaction.

[1367] A "near miss incident" refers to an incident that did not actually result in an accident in a factory or work site, but was potentially dangerous.

[1368] "Storage" means systematically collecting and storing data and information.

[1369] "Analysis" refers to the process of investigating and analyzing data in detail to clarify its background and causes.

[1370] A "similar case" refers to an event that occurred in the past that shares characteristics or patterns with the current case.

[1371] A "root cause" is the direct cause that causes a certain phenomenon or result.

[1372] "Countermeasures" refer to the means or methods taken to prevent or resolve a specific problem or danger.

[1373] "Feedback" refers to the process of providing an evaluation or response to actions or results, and making improvements or corrections based on that information.

[1374] "Emotional data" refers to data that expresses the user's emotions and psychological state, which can be obtained from facial expressions, voice, posture, etc.

[1375] "Risk assessment" refers to the process of evaluating the dangers of specific tasks or situations and determining the necessity and priority of countermeasures.

[1376] "Priority" refers to determining the order in which multiple tasks or problems should be processed first.

[1377] "Machine learning" is a field of artificial intelligence that allows computers to find patterns in data and make predictions and decisions based on them.

[1378] The system for implementing this invention is designed to manage near-miss incidents and improve safety within factories. This system consists of a series of steps: inputting near-miss incidents, collecting and analyzing emotional data, storing and searching data, proposing appropriate countermeasures, and collecting feedback.

[1379] System Components

[1380] 1. User Device

[1381] Input method: Users input near-miss incidents that occur in the factory. For example, they can input an incident such as "a tool slipped out of the hand while working at height" using a smartphone, tablet, or dedicated device.

[1382] Emotion data collection method: A camera and microphone are built into the device to collect the user's facial expressions and voice. This allows for the collection of emotional data such as the user's anxiety or surprise. Software such as Affectiva is used as the emotion engine.

[1383] 2. Server

[1384] Data storage method: Near-miss incidents and emotion data sent from user devices are received and stored in a database. Large volumes of data can be stored using cloud services.

[1385] Similar case search method: Apply natural language processing (NLP) technology and machine learning algorithms to past near-miss case data to search for similar cases. For example, libraries such as Scikit-learn and TensorFlow are used.

[1386] Countermeasure suggestion method: Based on the search results of similar cases, appropriate countermeasures are suggested. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[1387] Feedback collection method: The results of the measures taken by users and their emotional data at the time are collected again and stored in a database.

[1388] 3. Machine Learning Models

[1389] Learning and accuracy improvement measures: Using accumulated feedback data and sentiment data, the machine learning model is updated to improve the accuracy of countermeasure proposals.

[1390] Examples and prompts

[1391] Example 1:

[1392] Near miss case: While working at height, a tool slipped from the worker's hand and nearly hit a worker below.

[1393] Emotion data collection: Detecting "anxiety" from the operator's voice

[1394] Similar case search results: Past cases and the suggestion to "use non-slip gloves" as a countermeasure

[1395] Example 2:

[1396] Near miss case: Heavy machinery almost crashed into a wall due to an operating error

[1397] Emotion data collection: Detecting "surprise" from the operator's facial expression

[1398] Similar case search results: Past cases and suggested countermeasures such as "sensor readjustment" and "additional safety checks"

[1399] Prompt Sentence Examples

[1400] User submitted example: While working at height, a tool slipped from his hand and nearly hit a worker below.

[1401] Emotional data: Anxiety

[1402] Analysis result: This is a case where a tool fell while working at height. Based on past data, we suggest using non-slip gloves. Please take this measure.

[1403] In this way, this system efficiently carries out a series of steps, from managing near-miss incidents to proposing countermeasures and providing feedback, in order to improve safety within the factory, and is able to take flexible and effective measures that take into account the user's emotional state.

[1404] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1405] Step 1:

[1406] Users input cases of near misses that have occurred in the factory into the terminal. For example, they can use a smartphone, tablet, or dedicated terminal to input an example in text format, such as "A tool slipped out of my hand while working at height."

[1407] Input: Near miss incidents entered by the user

[1408] Output: Near miss incidents in text format

[1409] Step 2:

[1410] The user's facial expressions and voice are collected using a camera and microphone installed on the user's device, and the emotional data is analyzed by an emotion engine (e.g., Affectiva), which obtains emotional data such as the user's anxiety or surprise.

[1411] Input: User facial and voice data

[1412] Output: Parsed emotion data

[1413] Step 3:

[1414] The device sends the collected near-miss incidents and emotion data to a server, where the data format is checked for consistency and then stored in a database.

[1415] Input: Submitted near-miss incident and emotion data

[1416] Output: Near miss incidents and emotion data stored in a database

[1417] Step 4:

[1418] The server analyzes the accumulated near-miss incident cases and searches for similar cases using natural language processing (NLP) technology and machine learning algorithms (such as Scikit-learn and TensorFlow). For example, it searches to see if there have been any past cases of "a tool falling while working at height."

[1419] Input: Near miss incidents stored in the database

[1420] Output: Search results for similar cases

[1421] Step 5:

[1422] The server identifies the root cause based on the search results of similar cases and suggests appropriate countermeasures. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[1423] Input: Search results for similar cases and sentiment data

[1424] Output: List of proposed measures

[1425] Step 6:

[1426] The server notifies the user of the proposed measures. The user implements the measures and inputs the results into the terminal again. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of tools dropped has decreased."

[1427] Input: Proposed measures and user implementation results

[1428] Output: Input feedback data

[1429] Step 7:

[1430] The device then sends the collected feedback and emotion data to a server where it is stored in a database. The stored data is used for further analysis and countermeasure proposals.

[1431] Input: Feedback and emotion data

[1432] Output: Feedback and emotion data stored in a database

[1433] Step 8:

[1434] The server updates the machine learning model using the accumulated new feedback and emotion data, improving the accuracy of the next countermeasure proposal and enabling flexible countermeasures to be taken based on the user's emotional state.

[1435] Input: Newly accumulated feedback and emotion data

[1436] Output: An updated machine learning model

[1437] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1438] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1439] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1440] [Fourth embodiment]

[1441] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1442] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1443] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1444] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1445] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1446] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1447] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1448] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1449] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1450] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1451] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1452] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1453] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1454] The system for carrying out the present invention is for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Below, each component of the system and its operation will be explained.

[1455] System configuration

[1456] 1. User Device

[1457] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[1458] Users operate the system through a dedicated application or a web browser.

[1459] 2. Server

[1460] This is a central system that receives data sent from user terminals and stores and analyzes it.

[1461] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[1462] 3. Database

[1463] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback.

[1464] Program processing explanation

[1465] User terminal operation

[1466] Users input examples of near misses that occurred on-site into a dedicated application or web form. For example, they can input an example such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[1467] Server Operation

[1468] The server receives near-miss incidents sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[1469] Search and suggest similar cases

[1470] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to an expert, who can then conduct a more detailed analysis or ask questions to the user. For example, the expert can provide the user with information such as "what countermeasures have been effective in similar situations."

[1471] Identifying the root cause and proposing countermeasures

[1472] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[1473] Implementation and feedback input

[1474] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1475] Learning and accuracy improvement through machine learning

[1476] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1477] Operation example

[1478] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[1479] In this way, the system of the present invention is able to quickly and effectively manage and take action against near miss incidents.

[1480] The processing flow will be explained below.

[1481] Step 1: User Action

[1482] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[1483] Users input details of near miss incidents that occurred on-site. For example, they record an incident such as "While working at height, a tool slipped from my hand and nearly hit a worker below."

[1484] The user completes the data entry and presses the submit button.

[1485] Step 2: Device Operation

[1486] The terminal performs a format check on the entered data to ensure there are no missing items or formatting errors.

[1487] After verifying that the data has been entered correctly, convert the input data into an appropriate data format, such as JSON.

[1488] The converted data is sent to the server.

[1489] Step 3: Receiving and storing on the server

[1490] The server receives the data sent from the terminal.

[1491] The received data is analyzed and stored in a database in an appropriate format.

[1492] Step 4: Search for similar cases

[1493] The server searches the accumulated database and extracts past cases similar to the received near-miss case.

[1494] Apply natural language processing (NLP) techniques and machine learning algorithms to identify similar cases and their countermeasures.

[1495] Step 5: Initial analysis and expert notification

[1496] The server performs an initial analysis based on the search results for similar cases.

[1497] The initial analysis results are notified to experts via email or a dedicated application.

[1498] Step 6: Expert Operation

[1499] The expert accesses the server and checks the notified initial analysis results.

[1500] If necessary, the user is asked additional questions.

[1501] Step 7: Provide additional user information

[1502] The user answers questions from the expert and provides additional information.

[1503] Enter additional information and submit it to the server.

[1504] Step 8: Identify the root cause and propose a solution

[1505] The server identifies the true cause of near miss incidents based on the expert analysis results and additional information from users.

[1506] Based on the root cause, appropriate countermeasures are searched for in the database.

[1507] A countermeasure plan is generated and notified to the user.

[1508] Step 9: User implementation and feedback

[1509] The user implements the notified countermeasures on-site.

[1510] The effects of the implemented measures and the situation at the site are again entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of tools being dropped has decreased."

[1511] The user terminal transmits the feedback to the server.

[1512] Step 10: Gather feedback and learn

[1513] The server stores the received feedback in a database.

[1514] The machine learning model is updated based on the feedback data and learns to improve the accuracy of system analysis and countermeasure proposals.

[1515] Example 1

[1516] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1517] Near miss incidents at work sites are important information for preventing serious accidents, but their collection, management, and analysis depend on human resources, making them inefficient and difficult to contribute to overall safety measures.In particular, when there are a large number of near miss incidents, it is difficult to search for similar incidents and propose appropriate countermeasures, which results in the problem of accident prevention measures not being implemented effectively.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1519] In this invention, the server includes: means for a user to input near-miss incident cases; means for checking the format of the near-miss incident cases input at the terminal; means for transmitting data to the server; means for storing the data received by the server in a database; means for analyzing the stored data using natural language processing technology and machine learning algorithms and searching for similar incidents; means for notifying experts based on the search results; means for the experts to analyze the analysis results in detail and provide feedback to the user; means for the server to identify the root cause based on the expert feedback and propose countermeasures; means for the user to implement the proposed countermeasures and provide feedback on the results; and means for the server to update the machine learning model based on newly accumulated feedback information to improve accuracy. This makes it possible to efficiently collect, manage, and analyze near-miss incident cases and take prompt and appropriate countermeasures.

[1520] A "user terminal" is a device for transmitting near-miss incidents input by a user to a server.

[1521] A "near miss case" refers to an incident that nearly developed into an accident or disaster but was prevented.

[1522] "Format check" is the process of checking whether near miss cases are entered in the correct format.

[1523] A "server" is a central data processing system that receives, analyzes, and stores data sent from user terminals.

[1524] The "database" is a system that systematically stores and manages information such as near-miss incident cases, countermeasures, and feedback.

[1525] "Natural language processing technology" is a technology for understanding and analyzing human language.

[1526] A "machine learning algorithm" is a mathematical method for learning from data and making predictions or classifications.

[1527] "Searching for similar cases" is the process of finding past cases similar to a newly entered near-miss case from the database.

[1528] "Expert notification" is the process of communicating the results of the initial analysis to experts and soliciting their feedback for further analysis.

[1529] "Root cause identification" is an analytical process for identifying the underlying cause of a near miss incident.

[1530] "Proposing measures" is the process of presenting appropriate measures to eliminate the identified root causes.

[1531] "Feedback input" is the process in which the user inputs the results and effects of the measures they have implemented back into the system.

[1532] "Machine learning model updating" is the process of retraining machine learning algorithms based on newly accumulated data to improve the accuracy of the system.

[1533] The present invention is a system for efficiently managing and analyzing near-miss incidents and taking appropriate measures. Each component of the present invention and its operation will be specifically described below.

[1534] System configuration

[1535] 1. User Device

[1536] A device that allows users to input near-miss incidents and provide feedback on countermeasures. Specifically, it can be a tablet, smartphone, or PC.

[1537] Users operate the system through a dedicated application or a web browser.

[1538] 2. Server

[1539] This is a central system that receives data sent from user devices and stores and analyzes it. Specifically, it uses cloud services. A server with high-performance data processing capabilities is desirable.

[1540] 3. Database

[1541] It is an information management system connected to a server that stores data such as near-miss incidents, countermeasures, and feedback. SQL databases and NoSQL databases are used.

[1542] System processing flow

[1543] User terminal

[1544] Users can input near-miss incidents into a dedicated application or web form at the work site. For example, they can input an incident such as, "While working at height, a tool slipped from my hand and nearly hit a worker below." The terminal checks the format of this input information and, if there are no problems, sends it to the server.

[1545] server

[1546] The server receives near-miss incidents sent from user devices and stores them in a database. It then applies natural language processing (NLP) technology and machine learning algorithms to analyze the stored data and search for similar incidents. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts, and provides relevant information to the user.

[1547] Search and suggest similar cases

[1548] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then sent to experts, who can then conduct further detailed analysis or ask questions to the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations."

[1549] Identifying the root cause and proposing countermeasures

[1550] The server runs an algorithm based on the expert analysis and past data to identify the root cause of near-miss incidents. For example, it might determine that the cause of a tool slipping from a hand was improper use of gloves. It then suggests appropriate countermeasures, such as using non-slip gloves for specific tasks.

[1551] Implementation and feedback input

[1552] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1553] Learning and accuracy improvement through machine learning

[1554] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1555] Operation example

[1556] For example, if a tool slips from a user's hand while working at height in a manufacturing plant and nearly hits a worker below, the user can input the incident into a terminal. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and receives feedback on the results, allowing the entire system to be improved over time. In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures.

[1557] Prompt Sentence Examples

[1558] An example of a prompt sentence to be input to the generative AI model is shown below.

[1559] Please explain how you propose specific countermeasures when a user enters a near-miss incident case. Case: "While working at height, a tool slips from your hand and may hit a worker below."

[1560] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1561] Step 1:

[1562] The user inputs a near-miss incident.

[1563] Users use a dedicated application or web form to enter near-miss incidents that have occurred at the work site. The data entered includes details of the incident, date and time, and on-site conditions. An example of an input would be, "While working at height, a tool slipped from my hand and nearly hit a worker below."

[1564] Input: Near miss incidents input by users

[1565] Output: Input data that passes format check

[1566] Step 2:

[1567] Check the format of input data on the terminal

[1568] The terminal checks whether the data entered by the user conforms to the specified format. For example, it checks whether all required fields have been entered and whether the data format is correct. If the check passes, it proceeds to the next process.

[1569] Input: Near miss incidents entered by the user

[1570] Output: Data formatted to be sent to the server

[1571] Example: Check whether required fields such as "Event details," "Date and time," and "On-site situation" have been entered, and if any fields are missing, display a warning to the user.

[1572] Step 3:

[1573] Sending data to the server

[1574] The terminal sends data that has passed the format check to the server. At this time, the data is encrypted and sent using the HTTPS protocol to ensure data security.

[1575] Input: Data that passes format check

[1576] Output: Data sent to the server

[1577] Example: Encrypting data entered by a user and sending it to a server using HTTPS.

[1578] Step 4:

[1579] Receiving and storing data on the server

[1580] The server receives the data sent from the device and stores it in a database, which can then be used for analysis and retrieval.

[1581] Input: Data sent from the terminal

[1582] Output: Near miss cases stored in the database

[1583] Example: The server immediately stores the received data in the database and simultaneously backs up the data.

[1584] Step 5:

[1585] Data analysis and similar case search on the server

[1586] The server analyzes the accumulated data and searches for similar cases using natural language processing technology and machine learning algorithms. Specifically, it finds past cases of "tools falling while working at height" and analyzes countermeasures and impacts.

[1587] Input: Near miss cases stored in the database

[1588] Output: Search results for similar cases and initial analysis results

[1589] Example: Natural language processing techniques are used to tokenize example sentences entered by users, and similarities are calculated using machine learning algorithms.

[1590] Step 6:

[1591] Notification from the server to the expert

[1592] The server performs an initial analysis based on the search results for similar cases and notifies the experts of the results, who then begin a detailed analysis.

[1593] Input: Search results for similar cases and initial analysis results

[1594] Output:Notify Expert

[1595] Example: The server notifies the search results and initial analysis results via email or an expert dashboard.

[1596] Step 7:

[1597] Detailed analysis and feedback from experts

[1598] The expert conducts a detailed analysis and provides additional questions and feedback to the user, such as "What kind of gloves were you using?", and develops detailed countermeasures.

[1599] Input: Expert notification data and initial analysis results

[1600] Output: Detailed analysis by experts and feedback to users

[1601] Example: Experts can view detailed case studies from the dashboard and use the comments feature to ask users additional questions.

[1602] Step 8:

[1603] Identifying the root cause and proposing countermeasures by the server

[1604] Based on the expert analysis and past data, the server uses an algorithm to identify the root cause of near-miss incidents and develop appropriate countermeasures. For example, if the root cause is "inappropriate glove selection," the server will suggest specific countermeasures such as "using non-slip gloves."

[1605] Input: Expert detailed analysis and historical data

[1606] Output: Identified root cause and proposed measures

[1607] Example: The server runs an algorithm to find the root cause of the case and generate effective countermeasures as a proposal.

[1608] Step 9:

[1609] User implementation of measures and feedback

[1610] The user implements the measures in the field and then inputs the results and effects into the terminal again. For example, the user inputs feedback such as, "Using non-slip gloves has reduced the number of tools being dropped."

[1611] Input: Countermeasure proposals from the server, user feedback

[1612] Output: Data on the results of measures taken

[1613] Example: The user inputs the situation after implementing the countermeasures into the terminal and sends it to the server.

[1614] Step 10:

[1615] Server-based feedback accumulation and machine learning model updates

[1616] The server accumulates the feedback sent by users in a database and updates the machine learning model based on that information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1617] Input: Feedback data from users

[1618] Output: Updated machine learning model

[1619] Example: The server uses new feedback data to retrain the machine learning model, improving the accuracy of the system.

[1620] (Application example 1)

[1621] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1622] Near miss incidents can occur frequently in work environments such as factories. It is necessary to properly manage and analyze these incidents and implement appropriate countermeasures, but conventional systems often fail to adequately detect dangers in real time or propose specific countermeasures. Furthermore, there is a lack of means to efficiently collect worker feedback and improve the system based on the analysis results. Therefore, there is a need for a system that can efficiently manage and address near miss incidents while ensuring worker safety.

[1623] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1624] In this invention, the server includes means for inputting near-miss incident cases, means for storing the input near-miss incident cases in a database, means for analyzing the stored near-miss incident cases and searching for similar incidents, means for identifying root causes and proposing countermeasures based on the search results for similar incidents, means for implementing the proposed countermeasures and inputting feedback thereon, means for performing real-time monitoring in the factory, automatically detecting potential hazards, and proposing countermeasures, and means for functioning as an application installed on automated work equipment in the factory. This enables efficient management of near-miss incident cases and countermeasures in the work environment, and ensures the safety of workers.

[1625] A "near miss case" is an incident that comes just short of causing an accident or trouble in the work environment.

[1626] A "database" is a system that manages accumulated information in an organized manner and enables it to be quickly searched and used.

[1627] "Real-time monitoring" is a method of constantly monitoring the working environment and equipment status, and acquiring and analyzing data instantly.

[1628] "Automated work equipment" refers to machines and robots used in work environments such as factories.

[1629] "Feedback" refers to reports and opinions about the results and effectiveness of measures.

[1630] "Similar cases" refer to past near-miss cases that are similar to the current case.

[1631] A "remedy proposal" is a recommendation of appropriate action or change in response to a specific problem or risk.

[1632] "Analysis" involves examining accumulated data in detail and deriving meaning and patterns from it.

[1633] An "input device" is a device used by a user to input data.

[1634] "Machine learning" is a technology that allows computers to autonomously learn based on empirical data and make judgments and predictions.

[1635] A "pattern or trend" is a recurring characteristic or direction of variation in data.

[1636] MODE FOR CARRYING OUT THE INVENTION

[1637] The system for implementing the present invention efficiently manages and analyzes near-miss incidents in a factory environment and automatically suggests appropriate countermeasures. Each component of the system and its operation will be described below.

[1638] System configuration

[1639] 1. User Device

[1640] This device allows workers to input near-miss incidents and provides feedback on countermeasures.

[1641] Users operate the device through a dedicated application or a web browser, such as on a tablet or smartphone.

[1642] 2. Real-time monitoring device

[1643] The working environment and working conditions are constantly monitored using sensors and cameras mounted on automated work equipment (robot arms, etc.) in the factory.

[1644] The acquired data is immediately sent to a server where it is analyzed to detect potential dangers.

[1645] 3. Server

[1646] It is a central system that receives, stores, and analyzes data sent from user terminals and real-time monitoring devices.

[1647] Using cloud services (e.g., Amazon Web Services, Google Cloud Platform) enables high-speed data processing and large-volume data storage.

[1648] 4. Database

[1649] It is an information management system connected to a server that stores data such as near-miss incidents, real-time monitoring data, countermeasures, and feedback.

[1650] Program processing explanation

[1651] User terminal operation

[1652] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they can enter an incident such as, "While working at height, a tool slipped from my hand and may have hit a worker below." The terminal checks the format of this input information, and if there are no problems, it is sent to the server.

[1653] Real-time monitoring in action

[1654] Sensors and cameras attached to the automated work equipment monitor the work situation in real time and send the acquired data to a server, which analyzes the data and detects potential hazards.

[1655] Server Operation

[1656] The server receives data sent from user devices and real-time monitoring devices and stores it in a database. It then analyzes the stored data and searches for similar cases, applying natural language processing (NLP) techniques (e.g., spaCy, NLTK) and machine learning algorithms (e.g., Scikit-learn, TensorFlow). For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[1657] Search and suggest similar cases

[1658] The server searches for similar cases and performs an initial analysis based on the results. Based on the results of this initial analysis, the server proposes countermeasures. For example, it provides information to the user such as "In similar situations, using non-slip gloves was effective."

[1659] Implementation and feedback input

[1660] The user implements the notified measures on-site and then inputs the results and effects back into the terminal. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The user terminal sends this information to the server, which stores it in a database.

[1661] Learning and accuracy improvement through machine learning

[1662] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information, enabling it to propose more accurate countermeasures the next time a near-miss occurs.

[1663] Examples of concrete examples and prompts

[1664] For example, if a tool slips from a user's hand while working at height in a manufacturing plant, almost hitting a worker below, the user can input the incident into their terminal. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures and provides feedback on the results, allowing the entire system to be improved over time.

[1665] Example prompt sentence:

[1666] "While working at height, a tool almost slipped out of my hand. Please suggest the best solution to this problem based on similar cases from the past. We will also collect feedback to evaluate the effectiveness of the solution."

[1667] In this way, the system of the present invention is able to quickly and effectively manage and address near miss incidents in a factory environment.

[1668] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1669] Step 1:

[1670] Input of near miss cases from user terminals

[1671] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, they might enter, "While working at height, a tool nearly slipped out of my hand." The entered data is checked for format and, if there are no problems, is sent to the server.

[1672] Input: Near miss incident details

[1673] Output: Send data to the server

[1674] Step 2:

[1675] Data reception and storage on the server

[1676] The server stores the near-miss incident data received from the user terminal in a database, which also stores past near-miss incidents and countermeasures.

[1677] Input: Near miss incident data sent from the user's device

[1678] Output: Data accumulation in database

[1679] Step 3:

[1680] Real-time monitoring data collection

[1681] Sensors and cameras installed on automated work equipment in factories monitor the work situation and environment in real time, and the acquired data is immediately sent to a server.

[1682] Input: Real-time data from automated work equipment

[1683] Output: Send data to the server

[1684] Step 4:

[1685] Data analysis on the server

[1686] The server uses natural language processing (NLP) techniques and machine learning algorithms (e.g., spaCy, NLTK, Scikit-learn, TensorFlow) to analyze the accumulated data and real-time monitoring data, search for similar past cases, and extract information to propose appropriate countermeasures.

[1687] Input: Database and real-time monitoring data

[1688] Output: Analysis results and countermeasures

[1689] Step 5:

[1690] Search for similar cases and propose countermeasures

[1691] The server searches for similar cases based on the analysis results and provides the user with countermeasures and their impact. For example, if there is a case of a tool slipping off in the past, the server will suggest the use of non-slip gloves.

[1692] Input: Analysis results

[1693] Output: Countermeasure proposal

[1694] Step 6:

[1695] Implementing measures and providing feedback

[1696] The user implements the proposed measures on-site and then inputs the results and effects into the terminal again. For example, the user might input, "Using non-slip gloves resulted in fewer dropped tools." The feedback information is then sent to the server.

[1697] Input: Feedback after implementing measures

[1698] Output: Send feedback to the server

[1699] Step 7:

[1700] Accumulating feedback data and updating machine learning models

[1701] The server accumulates the feedback data in a database and updates the machine learning model, enabling more accurate countermeasure proposals the next time a near-miss occurs.

[1702] Input: Feedback data

[1703] Output: Database and machine learning model updates

[1704] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1705] The system for implementing this invention efficiently manages and analyzes near-miss incidents and takes more effective measures by combining it with an emotion engine that recognizes the user's emotions. Each component of this system and its operation will be explained below.

[1706] System configuration

[1707] 1. User Device

[1708] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[1709] Users operate the system through a dedicated application or a web browser.

[1710] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[1711] 2. Server

[1712] This is a central system that receives data sent from user terminals and stores and analyzes it.

[1713] Cloud services may also be used, allowing for high-speed data processing and large-volume data storage.

[1714] 3. Database

[1715] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[1716] Program processing explanation

[1717] User terminal operation

[1718] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[1719] Server Operation

[1720] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. It then analyzes the stored data and searches for similar incidents by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past incident of a tool falling while working at height, it analyzes the countermeasures and impacts involved and provides relevant information to the user.

[1721] Search and suggest similar cases

[1722] The server searches for similar cases and performs an initial analysis based on the results. The results of this initial analysis are then reported to experts, who can then conduct further detailed analysis or ask questions of the user. For example, the experts can provide the user with information such as "what countermeasures have been effective in similar situations." The server also determines the importance of the case based on the user's emotions evaluated by the emotion engine, and adjusts the priority of countermeasures as necessary.

[1723] Identifying the root cause and proposing countermeasures

[1724] The server runs an algorithm to identify the root cause of near-miss incidents based on the expert analysis, past data, and emotion data. For example, it might determine that "the cause of a tool slipping from the hand was improper use of gloves." It then suggests appropriate countermeasures, such as "use non-slip gloves for certain tasks." If the user feels anxious, it will provide further detailed explanations and encourage them to take countermeasures.

[1725] Implementation and feedback input

[1726] The user implements the notified measures on-site and again inputs the results and effects into the device. For example, the user might input feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." At this time, the emotion engine recognizes the user's emotions again and collects emotional data after the measures have been implemented. The device then sends this information to the server, which stores it in a database.

[1727] Learning and accuracy improvement through machine learning

[1728] The server updates the machine learning model to learn important patterns and trends from newly accumulated feedback information and emotional data. This allows for more accurate countermeasure proposals the next time a near-miss occurs, while also enabling flexible responses based on the user's emotional state.

[1729] Operation example

[1730] For example, if a tool slips from a user's hand while working at a height in a manufacturing plant, nearly hitting a worker below, the user can input the incident into their device. At the same time, the emotion engine recognizes the user's anxious facial expression. The server uses this information to search for similar cases and, based on past countermeasures, suggests measures such as "using special anti-slip gloves" or "wearing a tool wristband." The user then implements the suggested countermeasures, providing feedback on the results while also recollecting their emotional state at the time. As new feedback and emotion data accumulate, the entire system is continually improved, resulting in more effective countermeasures and responses that take the user's psychology into consideration.

[1731] In this way, the system of the present invention is able to quickly and effectively manage near-miss incidents and take countermeasures, and also to respond flexibly while taking into consideration the user's feelings.

[1732] The processing flow will be explained below.

[1733] Step 1: User Action

[1734] The user starts up a dedicated terminal and opens an application or web form for recording near-miss incidents.

[1735] Users input details of near miss incidents that occurred on-site. For example, they might record an incident such as "While working at height, a tool slipped from their hand and nearly hit a worker below."

[1736] The user enters the importance and scope of the case, and then presses the send button.

[1737] Step 2: Device Operation

[1738] The terminal performs a format check on the data entered by the user to ensure there are no missing items or formatting errors.

[1739] Once you have verified that the data has been entered accurately, convert the input data into an appropriate data format, such as JSON.

[1740] The converted data is sent to the server.

[1741] Step 3: Receiving and storing on the server

[1742] The server receives the data sent from the terminal.

[1743] The received near-miss incident data is analyzed and stored in a database in an appropriate format.

[1744] Step 4: Emotion Engine in Action

[1745] The emotion engine installed in the device simultaneously recognizes the user's facial expressions and voice to collect emotional data.

[1746] If the user is expressing emotions such as anxiety, impatience, or anger, this information is also sent to the server.

[1747] Step 5: Search for similar cases

[1748] The server searches past near-miss incident cases stored in a database and extracts cases similar to the received case.

[1749] Natural language processing (NLP) techniques and machine learning algorithms are used to extract relevant information from similar cases.

[1750] Step 6: Initial analysis and expert notification

[1751] The server performs an initial analysis based on the search results for similar cases.

[1752] The initial analysis results and the user's emotional data are notified to the experts via email or a dedicated application.

[1753] Step 7: Expert Operation

[1754] The expert accesses the server and checks the notified initial analysis results and emotion data.

[1755] If necessary, the user is asked additional questions.

[1756] Step 8: Provide additional user information

[1757] The user answers questions from the expert and provides additional information.

[1758] Enter additional information and submit it to the server.

[1759] Step 9: Identify the root cause and propose a solution

[1760] The server identifies the root cause of near miss incidents based on the expert analysis results, additional information from users, and emotional data.

[1761] Based on the root cause, appropriate countermeasures are searched for in the database.

[1762] A countermeasure plan is generated and notified to the user.

[1763] Step 10: User implementation and feedback

[1764] The user implements the notified countermeasures on-site.

[1765] The effects of the implemented measures and the situation at the site are entered into the terminal. For example, feedback such as "As a result of using non-slip gloves, the number of dropped tools has decreased."

[1766] The emotion engine installed on the device will again recognize the user's emotions and collect emotional data after the activity.

[1767] Step 11: Gather feedback and learn

[1768] The server stores the received feedback and emotion data in a database.

[1769] Based on feedback data and sentiment data, the machine learning model is updated to improve the accuracy of analysis and countermeasure proposals.

[1770] Example 2

[1771] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1772] Conventional near-miss incident management systems primarily focus on case input and analysis, and lack the ability to propose flexible countermeasures that take into account the user's emotional state and learning from feedback. This makes it difficult to propose more effective countermeasures while reducing the user's psychological anxiety and stress. Furthermore, technology for retrieving similar cases and improving analysis accuracy is insufficient, limiting the effectiveness of the proposed countermeasures.

[1773] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1774] In this invention, the server includes a means for inputting near-miss incidents, a means for storing the input near-miss incidents in a database, and a means for analyzing the stored near-miss incidents and searching for similar incidents. This enables the user to quickly collect and manage near-miss incidents experienced in the field, and enables highly accurate analysis and searching for similar incidents. Furthermore, by including a means for recognizing the user's emotional state and collecting emotional data, a means for analyzing the collected emotional data and adjusting the importance of countermeasures, and a means for using machine learning to learn patterns and trends from feedback data and emotional data and improve the accuracy of countermeasure proposals, flexible and effective countermeasure proposals that take the user's psychological state into consideration are possible.

[1775] A "near miss incident" refers to an event or occurrence that did not result in a serious accident or trouble, but could have caused danger or problems.

[1776] "Input means" refers to the means by which a user inputs near-miss incidents and feedback information into a device.

[1777] "Database" refers to a system that centrally manages accumulated data such as near-miss incident cases, emotional data, and feedback information.

[1778] "Analysis means" refers to the technology used to analyze accumulated data, search for similar cases, and discover patterns.

[1779] The "similar case search means" refers to a means for searching for cases similar to the currently input case from data accumulated in the past.

[1780] "Means for identifying the root cause" refers to a means for analyzing the causes of the near-miss incidents entered and identifying the underlying problems.

[1781] "Measure suggestion means" refers to a means for proposing optimal solutions or measures to users based on the identified root cause.

[1782] The "feedback input means" refers to a means for a user to input the results and impressions of implementing the proposed measures back into the system.

[1783] "Emotion recognition means" refers to technology for collecting and analyzing emotional data from a user's facial expressions and voice.

[1784] "Emotion data analysis means" refers to a means for analyzing collected emotion data and adjusting the importance of countermeasures based on the analysis.

[1785] "Machine learning methods" refers to technologies that learn patterns and trends from feedback data and emotion data to improve the accuracy of analysis and countermeasure proposals.

[1786] "Natural language processing technology" refers to technology that analyzes accumulated text data to search for similar cases and assist in problem-solving.

[1787] This invention is a system that efficiently manages and analyzes near-miss incidents and takes more effective countermeasures by combining it with an emotion engine that recognizes user emotions. The system includes a series of processes from user input to countermeasure proposals, and also uses user emotion data for analysis.

[1788] System configuration

[1789] 1. User Device

[1790] This is a device that allows users to input near-miss incidents and provide feedback on countermeasures.

[1791] Users operate the system through a dedicated application or a web browser.

[1792] The device is equipped with an emotion engine that recognizes the user's emotions and collects emotional data from the user's facial expressions and voice when inputting.

[1793] 2. Server

[1794] This is a central system that receives data sent from user terminals and stores and analyzes it.

[1795] Cloud services may also be used, enabling high-speed data processing and large-volume data storage. The server uses Python and machine learning frameworks (such as TensorFlow and PyTorch) to analyze and visualize the data.

[1796] 3. Database

[1797] It is an information management system connected to a server that stores information such as near-miss incidents, countermeasures, feedback, and emotional data.

[1798] Using a database system such as PostgreSQL enables efficient data management and fast searches.

[1799] Program operation explanation

[1800] User terminal operation

[1801] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. For example, a user might record an incident such as "While working at height, a tool slipped from his / her hand and nearly hit a worker below." At the same time, an emotion engine built into the device recognizes the user's facial expressions and voice and collects emotional data. For example, if the user looks anxious, that emotional data is also collected. The device checks the format of the input information and emotional data, and if there are no problems, sends the data to the server.

[1802] Server Operation

[1803] The server receives near-miss incidents and emotion data sent from user devices and stores them in a database. Next, it analyzes the stored data and searches for similar cases by applying natural language processing (NLP) technology and machine learning algorithms to past data. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact and provides relevant information to the user. Specifically, the analysis is performed using Python's NLTK library and Scikit-learn.

[1804] Examples of prompt statements

[1805] Here are some example prompts to input to a generative AI model:

[1806] When a user inputs a near-miss incident into a terminal, such as when a tool nearly fell while working at height, the emotion engine also identifies the user's anxious facial expression at the time. The server uses this information to search for similar incidents and, based on past countermeasures, suggests measures such as "using non-slip gloves" or "wearing a tool wristband." Please explain how this near-miss incident management system, which incorporates emotion engineering, works.

[1807] Using this prompt, the generative AI model is expected to provide a detailed explanation of the behavior of a specific system.

[1808] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1809] Step 1:

[1810] User input of near miss incidents

[1811] Users enter near-miss incidents that occurred on-site into a dedicated application or web form. Specifically, they use the device's keyboard or touch panel to enter details of the incident, such as "While working at height, a tool slipped from my hand and nearly hit a worker below." The input in this step is a record of the actual near-miss incident, and the output is the entered text data.

[1812] Step 2:

[1813] Emotion data collection using an emotion engine

[1814] An emotion engine built into the user device recognizes the user's facial expressions and voice. Specifically, the device's camera captures facial expressions, and the microphone analyzes the voice. For example, if the user looks anxious, that facial expression data is collected. The input for this step is the user's facial expression and voice, and the output is analyzed emotion data.

[1815] Step 3:

[1816] Data check and transmission by user terminal

[1817] The terminal performs a format check on the input near-miss case information and collected emotion data. Specifically, it checks the required fields on the input form and the consistency of the data format, and if there are no problems, it sends the data to the server. For example, it checks that the input case meets all required fields. The input for this step is near-miss case data and emotion data, and the output is the checked data.

[1818] Step 4:

[1819] Receiving and storing data by the server

[1820] The server receives the near-miss incidents and emotion data sent from the user's device and stores them in a database. Specifically, it receives the data using a RESTful API and saves it in a database such as PostgreSQL. The input to this step is the sent near-miss incident data and emotion data, and the output is the data saved in the database.

[1821] Step 5:

[1822] Server-based search and analysis of similar cases

[1823] The server analyzes the accumulated data and searches for similar cases. Specifically, it applies natural language processing (NLP) techniques and machine learning algorithms using Python's NLTK library and Scikit-learn. For example, if there is a past case of a tool falling while working at height, it analyzes the countermeasures and impact. The input for this step is the accumulated data, and the output is the search results and analysis results for similar cases.

[1824] Step 6:

[1825] Initial analysis and suggestions based on search results for similar cases

[1826] The server performs an initial analysis based on the search results. The results of this initial analysis are then sent to experts, who then conduct further detailed analysis or ask questions of the user. Specifically, the data is displayed in real time using a dedicated dashboard. The input to this step is the search results for similar cases, and the output is the initial analysis results.

[1827] Step 7:

[1828] Identifying the root cause by the server and proposing detailed countermeasures

[1829] The server runs an algorithm to identify the root cause based on the expert's analysis results, past data, and emotion data. For example, it might determine that "the cause of the tool slipping from the hand is improper use of gloves." It then suggests an appropriate countermeasure: "Use non-slip gloves for certain tasks." The input for this step is the initial analysis results and past data, and the output is a detailed countermeasure proposal.

[1830] Step 8:

[1831] User implementation and feedback

[1832] The user implements the notified countermeasures on-site and again inputs the results and effects into the terminal. Specifically, the user inputs feedback such as, "As a result of using non-slip gloves, the number of dropped tools has decreased." The input in this step is the result of implementing the countermeasures, and the output is feedback data.

[1833] Step 9:

[1834] Server receives feedback and updates the learning model

[1835] The server updates the machine learning model from the newly accumulated feedback information and emotion data. Specifically, it retrains the model using TensorFlow and PyTorch. This enables more accurate countermeasure proposals for the next near-miss incident. The input for this step is the feedback data and emotion data, and the output is an updated machine learning model.

[1836] (Application example 2)

[1837] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1838] Conventional near-miss incident analysis systems did not efficiently manage and analyze near-miss incidents, nor did they evaluate risks or respond to them while taking into account user emotional data. As a result, countermeasure proposals did not take into account the user's psychological state, making it difficult to achieve both improved safety and user satisfaction. In addition, the effective use of feedback data was limited, limiting the improvement of accuracy.

[1839] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting near-miss incident cases; means for storing the input near-miss incident cases in a database; means for analyzing the stored near-miss incident cases and searching for similar incident cases; means for identifying root causes and proposing countermeasures based on the search results for similar incident cases; means for implementing the proposed countermeasures and inputting feedback therefrom; means for collecting and analyzing emotional data and reflecting it in risk assessment; means for adjusting priorities based on the user's emotional state when proposing countermeasures; and means for learning patterns and trends from the feedback data and emotional data using machine learning to improve the accuracy of the analysis and countermeasure proposals. This enables safety countermeasures based on near-miss incident cases to be flexible and effective, taking into account the user's emotions, enabling more accurate countermeasure proposals and improved user satisfaction.

[1840] A "near miss incident" refers to an incident that did not actually result in an accident in a factory or work site, but was potentially dangerous.

[1841] "Storage" means systematically collecting and storing data and information.

[1842] "Analysis" refers to the process of investigating and analyzing data in detail to clarify its background and causes.

[1843] A "similar case" refers to an event that occurred in the past that shares characteristics or patterns with the current case.

[1844] A "root cause" is the direct cause that causes a certain phenomenon or result.

[1845] "Countermeasures" refer to the means or methods taken to prevent or resolve a specific problem or danger.

[1846] "Feedback" refers to the process of providing an evaluation or response to actions or results, and making improvements or corrections based on that information.

[1847] "Emotional data" refers to data that expresses the user's emotions and psychological state, which can be obtained from facial expressions, voice, posture, etc.

[1848] "Risk assessment" refers to the process of evaluating the dangers of specific tasks or situations and determining the necessity and priority of countermeasures.

[1849] "Priority" refers to determining the order in which multiple tasks or problems should be processed first.

[1850] "Machine learning" is a field of artificial intelligence that allows computers to find patterns in data and make predictions and decisions based on them.

[1851] The system for implementing this invention is designed to manage near-miss incidents and improve safety within factories. This system consists of a series of steps: inputting near-miss incidents, collecting and analyzing emotional data, storing and searching data, proposing appropriate countermeasures, and collecting feedback.

[1852] System Components

[1853] 1. User Device

[1854] Input method: Users input near-miss incidents that occur in the factory. For example, they can input an incident such as "a tool slipped out of the hand while working at height" using a smartphone, tablet, or dedicated device.

[1855] Emotion data collection method: A camera and microphone are built into the device to collect the user's facial expressions and voice. This allows for the collection of emotional data such as the user's anxiety or surprise. Software such as Affectiva is used as the emotion engine.

[1856] 2. Server

[1857] Data storage method: Near-miss incidents and emotion data sent from user devices are received and stored in a database. Large volumes of data can be stored using cloud services.

[1858] Similar case search method: Apply natural language processing (NLP) technology and machine learning algorithms to past near-miss case data to search for similar cases. For example, libraries such as Scikit-learn and TensorFlow are used.

[1859] Countermeasure suggestion method: Based on the search results of similar cases, appropriate countermeasures are suggested. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[1860] Feedback collection method: The results of the measures taken by users and their emotional data at the time are collected again and stored in a database.

[1861] 3. Machine Learning Models

[1862] Learning and accuracy improvement measures: Using accumulated feedback data and sentiment data, the machine learning model is updated to improve the accuracy of countermeasure proposals.

[1863] Examples and prompts

[1864] Example 1:

[1865] Near miss case: While working at height, a tool slipped from the worker's hand and nearly hit a worker below.

[1866] Emotion data collection: Detecting "anxiety" from the operator's voice

[1867] Similar case search results: Past cases and the suggestion to "use non-slip gloves" as a countermeasure

[1868] Example 2:

[1869] Near miss case: Heavy machinery almost crashed into a wall due to an operating error

[1870] Emotion data collection: Detecting "surprise" from the operator's facial expression

[1871] Similar case search results: Past cases and suggested countermeasures such as "sensor readjustment" and "additional safety checks"

[1872] Prompt Sentence Examples

[1873] User submitted example: While working at height, a tool slipped from his hand and nearly hit a worker below.

[1874] Emotional data: Anxiety

[1875] Analysis result: This is a case where a tool fell while working at height. Based on past data, we suggest using non-slip gloves. Please take this measure.

[1876] In this way, this system efficiently carries out a series of steps, from managing near-miss incidents to proposing countermeasures and providing feedback, in order to improve safety within the factory, and is able to take flexible and effective measures that take into account the user's emotional state.

[1877] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1878] Step 1:

[1879] Users input cases of near misses that have occurred in the factory into the terminal. For example, they can use a smartphone, tablet, or dedicated terminal to input an example in text format, such as "A tool slipped out of my hand while working at height."

[1880] Input: Near miss incidents entered by the user

[1881] Output: Near miss incidents in text format

[1882] Step 2:

[1883] The user's facial expressions and voice are collected using a camera and microphone installed on the user's device, and the emotional data is analyzed by an emotion engine (e.g., Affectiva), which obtains emotional data such as the user's anxiety or surprise.

[1884] Input: User facial and voice data

[1885] Output: Parsed emotion data

[1886] Step 3:

[1887] The device sends the collected near-miss incidents and emotion data to a server, where the data format is checked for consistency and then stored in a database.

[1888] Input: Submitted near-miss incident and emotion data

[1889] Output: Near miss incidents and emotion data stored in a database

[1890] Step 4:

[1891] The server analyzes the accumulated near-miss incident cases and searches for similar cases using natural language processing (NLP) technology and machine learning algorithms (such as Scikit-learn and TensorFlow). For example, it searches to see if there have been any past cases of "a tool falling while working at height."

[1892] Input: Near miss incidents stored in the database

[1893] Output: Search results for similar cases

[1894] Step 5:

[1895] The server identifies the root cause based on the search results of similar cases and suggests appropriate countermeasures. The priority of countermeasures is adjusted based on the user's emotional data. For example, suggestions such as "use non-slip gloves" or "strengthen safety checks" are made.

[1896] Input: Search results for similar cases and sentiment data

[1897] Output: List of proposed measures

[1898] Step 6:

[1899] The server notifies the user of the proposed measures. The user implements the measures and inputs the results into the terminal again. For example, the user inputs feedback such as, "As a result of using non-slip gloves, the number of tools dropped has decreased."

[1900] Input: Proposed measures and user implementation results

[1901] Output: Input feedback data

[1902] Step 7:

[1903] The device then sends the collected feedback and emotion data to a server where it is stored in a database. The stored data is used for further analysis and countermeasure proposals.

[1904] Input: Feedback and emotion data

[1905] Output: Feedback and emotion data stored in a database

[1906] Step 8:

[1907] The server updates the machine learning model using the accumulated new feedback and emotion data, improving the accuracy of the next countermeasure proposal and enabling flexible countermeasures to be taken based on the user's emotional state.

[1908] Input: Newly accumulated feedback and emotion data

[1909] Output: An updated machine learning model

[1910] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1911] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1912] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1913] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1914] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1915] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1916] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1917] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1918] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1919] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1920] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1921] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1922] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1923] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1924] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1925] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the...

Claims

1. A means of inputting near miss cases; A means for storing the input near miss cases in a database; A means to analyze the accumulated near-miss cases and search for similar cases; A method to identify the root cause based on the search results of similar cases and propose countermeasures, A system that includes a means to implement the proposed measures and enter feedback on them.

2. The system according to claim 1, further comprising a means for inputting near-miss incidents using a dedicated terminal.

3. 10. The system of claim 1, further comprising means for using machine learning to learn patterns and trends from the feedback data to improve the accuracy of the analysis and countermeasure proposals.

Images