Workplace-customized occupational safety and health metacognition assessment and training system

Abstract

The present invention relates to a workplace-customized industrial safety and health metacognitive evaluation and education system. The present invention relates to a technology capable of providing an integrated metacognitive evaluation and education system that searches for similar workplace accident cases based on multidimensional workplace information to construct risk scenarios, generates and scores metacognitive evaluation items using a large-scale language model based on these scenarios, and quantifies individual vulnerable areas based on the evaluation results to enable customized education.

Description

Technology Field

[0001] The present invention relates to a workplace-customized industrial safety and health metacognitive evaluation and education system, and more specifically, to a workplace-customized industrial safety and health metacognitive evaluation and education system capable of constructing risk scenarios by searching for similar workplace accident cases based on multidimensional workplace information, generating metacognitive evaluation questions using a large-scale language model based on this, and quantifying individual vulnerable areas through answers to the evaluation questions to provide customized educational materials. Background Technology

[0002] Despite the steady distribution of legal and institutional measures and technical protective equipment in the field of industrial safety and health over the decades, the overall industrial accident mortality rate in Korea has entered a range where it no longer decreases below a certain level.

[0003] In particular, in high-risk industries such as construction, manufacturing, and logistics, where risk factors fluctuate constantly depending on the project, process, season, and work pattern, the extent to which workers accurately perceive and appropriately respond to these risks determines whether an accident occurs.

[0004] Existing competency management systems have repeatedly conducted checklist-based training and evaluations, primarily focusing on "what is known" and "how well rules are followed." However, this approach has the problem of failing to quantitatively identify the gap between "what one believes they know" and "what one actually knows and practices."

[0005] As a result, the reality is that even managers and workers who have completed legal training and hold qualifications frequently miss key risk factors or fail to respond due to overconfidence or underconfidence.

[0006] The concept of metacognition, as defined in education and psychology, is a powerful theoretical tool for explaining or resolving this gap; metacognition refers to the cognitive process of self-monitoring one's knowledge, judgments, and behaviors, and regulating the degree of certainty.

[0007] When metacognition is introduced into the field of industrial safety and health, workplace stakeholders, including workers, supervisors (those being evaluated), and management, can simultaneously measure and improve the accuracy and confidence regarding their ability to identify risk factors, select risk control measures, and understand relevant laws and regulations.

[0008] However, since conditions vary significantly by workplace and accident scenarios involve complex multiple variables, manually designing and scoring the scenario-based open-ended questions required for metacognitive assessment poses a problem that demands an enormous amount of time and specialized personnel. Consequently, customized metacognitive assessments for actual workplaces have remained in the research stage.

[0009] In this regard, Korean registered patent No. 10-2528768 ("Foreign Worker Industrial Safety Education Management Platform") discloses a technology capable of systematically managing the education and labor of foreign workers. Prior art literature

[0010] Korean Registered Patent No. 10-2528768 (Registration Date: April 28, 2023) The problem to be solved

[0011] Accordingly, the present invention has been devised to solve the problems of the prior art as described above. The objective of the present invention is to provide a workplace-customized industrial safety and health metacognitive evaluation and education system capable of constructing risk scenarios by searching for similar workplace accident cases based on multidimensional workplace information, generating metacognitive evaluation questions using a large-scale language model based on this, and quantifying individual vulnerable areas through answers to the evaluation questions to provide customized educational materials. means of solving the problem

[0012] The workplace-customized industrial safety and health metacognition evaluation and education system of the present invention for achieving the above-mentioned purpose preferably comprises: an information processing unit that receives workplace characteristic information corresponding to a preset item, processes it, and converts it into a semantic vector; a matching processing unit that extracts accident-related data matching the vectorized workplace characteristic information using a first DB that stores and manages accident-related data that occurred in the past at an industrial site; a question processing unit that generates question information for metacognition evaluation using at least one of the vectorized workplace characteristic information and the extracted accident-related data; a scoring processing unit that receives response information corresponding to the generated question information and generates scoring result information; and a metacognition processing unit that analyzes the received response information and the generated scoring result information to generate metacognition evaluation information.

[0013] Furthermore, it is desirable for the matching processing unit to extract multiple accident-related data matched from the first DB based on vectorized workplace characteristic information, and to calculate a relevance score by performing a relevance evaluation for each extracted accident-related data.

[0014] Furthermore, it is preferable that the above-mentioned question processing unit includes a statute processing unit that extracts industrial safety and health statute provisions and related rules that match vectorized workplace characteristic information using a second DB storing and managing the Industrial Safety and Health Act and related rules, and a first question generation unit that generates question information for evaluating statute comprehension based on the extracted provisions and rules using a Large Language Model (LLM).

[0015] Furthermore, it is preferable that the item processing unit further includes a basic generation unit that generates a basic scenario containing a core risk factor for an accident using the accident-related data with the highest relevance score based on the relevance score for each accident-related data extracted by the matching processing unit; a final generation unit that generates a confusion risk factor using the remaining accident-related data excluding the accident-related data with the highest relevance score among the multiple accident-related data extracted by the matching processing unit, and generates a final scenario by including the confusion risk factor in the generated basic scenario; and a second item generation unit that generates item information for evaluating the ability to identify risk factors according to the final scenario using the large-scale language model.

[0016] Furthermore, it is preferable that the above-mentioned item processing unit further includes a control processing unit that extracts control information matching a risk factor according to accident-related data with the highest relevance score by utilizing a third DB that stores and manages control information regarding various risk factors, and a third item generation unit that generates item information for evaluating the ability to derive risk control based on the extracted control information by utilizing the above-mentioned large-scale language model.

[0017] Furthermore, it is desirable for the scoring processing unit to receive as input the answer information corresponding to the question information and the degree of confidence regarding the answer information as the response information.

[0018] Furthermore, it is desirable that the above-mentioned workplace-customized industrial safety and health metacognitive evaluation and training system further includes a training processing unit that analyzes vulnerabilities using the metacognitive evaluation information and generates customized training content data based on the analyzed vulnerabilities using a Large Language Model (LLM).

[0019] Furthermore, it is preferable that the above-described workplace-customized industrial safety and health metacognitive evaluation and training system further includes a tracking processing unit that provides the generated customized training content data, tracks the training history of the provided customized training content data, and performs feedback processing of the metacognitive evaluation information. Effects of the invention

[0020] According to the present invention, a workplace-customized industrial safety and health metacognitive evaluation and education system vectorizes workplace characteristic information and matches it semantically with an accident case database accumulated over many years, thereby selecting accident scenarios that users are highly likely to actually face. Consequently, evaluation items directly reflect the risk context for each field process, which substantially improves the accuracy of risk identification compared to the conventional general-purpose checklist method, and thus can enhance workplace customization and risk prediction accuracy.

[0021] In addition, since it presents quantitative risk level indicators through relevance scores of similar cases, it has the advantage of enabling the subject of evaluation to systematically recognize and predict the likelihood of accidents.

[0022] In addition, a large-scale language model generates field-specific questions in real time and automatically scores and classifies responses, offering the advantage of significantly reducing the time previously spent by experts manually writing and scoring scenarios.

[0023] Furthermore, it automatically extracts vulnerable areas based on user response patterns and immediately generates personalized and domain-specific educational content by re-searching relevant cases, laws, and control methods within the same database. Since evaluation results directly lead to educational prescriptions, it enables immediate learning supplementation within the same platform and offers the advantage of continuously tracking the degree of improvement through a re-evaluation loop.

[0024] In addition, since the workplace information structure, accident case DB, and legal DB have been modularized and schematized, the system can be applied not only to the construction industry but also to other industries such as manufacturing, logistics, and energy by simply replacing the data sets. Furthermore, because it is designed with a cloud and web-based architecture, it can improve scalability applicable across industries by lowering implementation and operation costs and facilitating deployment regardless of company size.

[0025] In particular, by storing and managing all responses, confidence levels, and score correction records generated during the evaluation and education process in a database, and utilizing them for item quality verification and algorithm retraining, it has the advantage of possessing a self-reinforcing structure in which item difficulty adjustment and recommendation accuracy automatically improve with repeated use.

[0026] In addition, supervisors and executives acting as evaluators rather than those being evaluated can comprehensively check metacognitive indicators and training history by job group and process through the dashboard. Based on this, they can make data-driven decisions regarding the reallocation of personnel for high-risk processes, investment in additional training, and safety budget priorities, which has the advantage of objectifying and scientifying existing subjective decision-making. Brief explanation of the drawing

[0027] FIG. 1 is a diagram illustrating an example configuration of a workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention. FIGS. 2 and FIGS. 3 are timing diagrams for a workplace-customized industrial safety and health metacognitive evaluation and education system according to an embodiment of the present invention. Specific details for implementing the invention

[0028] Hereinafter, a workplace-customized industrial safety and health metacognition evaluation and education system according to the present invention, having the configuration as described above, will be explained in detail with reference to the attached drawings. The drawings presented below are provided as examples to ensure that the concept of the present invention is sufficiently conveyed to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Furthermore, throughout the specification, the same reference numerals indicate the same components.

[0029] Unless otherwise defined, technical and scientific terms used herein have the meaning commonly understood by those skilled in the art to which this invention pertains, and descriptions of known functions and configurations that could unnecessarily obscure the essence of the invention are omitted in the following description and accompanying drawings.

[0030] Furthermore, a system refers to a set of components, including devices, mechanisms, and means, that are organized and interact regularly to perform necessary functions.

[0031] The workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention recognizes that the combination of a Large Language Model (LLM) and big data on accident cases can provide a clue to resolving the bottleneck mentioned in the background technology, and aims to provide a system for this purpose.

[0032] More specifically, by structuring accident cases, risk assessments, and legal provisions accumulated across various industries and inputting them into a large-scale language model, scenarios and questions specialized for the characteristics of each workplace are dynamically generated. By contextually interpreting the resulting free responses and analyzing not only correct and incorrect answers but also confidence levels and error patterns, the aforementioned problems can be resolved.

[0033] In particular, by utilizing vector similarity search technology, it has the advantage of enabling high-fidelity evaluations that reflect the on-site context by immediately matching accidents that occurred at workplaces most similar to the equipment, process, and environmental information entered by the user.

[0034] Through this, it is possible to provide an integrated metafactor evaluation and education system across all industries that searches for accident cases at similar workplaces based on multidimensional workplace information to construct risk scenarios, uses a large-scale language model to generate and score metafactor evaluation items based on these scenarios, and quantifies individual weaknesses based on the evaluation results to enable customized training.

[0035] The field of industrial safety and health possesses characteristics that are fundamentally different from general educational domains. While general problems are explained by simple causal relationships, such as result (B) caused by cause (A), industrial safety involves the complex interaction of multiple factors, including organizational, workplace, individual, and immediate factors. Furthermore, since problems involving these factors must occur simultaneously to lead to an accident, it is impossible to definitively identify the cause of an accident based on the presence of only one factor.

[0036] What makes the situation even more complex is the discrepancy between legal responsibilities and the reality on the ground. For instance, while regulations stipulate that a safety manager must be appointed, there are many instances in the field where people mistakenly believe that having a safety manager is sufficient for safety. In reality, site safety is only ensured when appointment, dedication, empowerment, substantive activities, site patrols, and training are all in place.

[0037] Furthermore, the biggest technical challenge in the field of industrial safety and health is that even the same safety measures or equipment can have completely different answers depending on the work situation (context).

[0038] For example, in the case of high-rise work, the priority order is safety belts, safety nets, and supervisors, whereas in the case of work in confined spaces, the priority order is ventilation systems, gas concentration measurements, and safety belts, showing a completely different priority order.

[0039] Because such context dependency cannot be predefined for all possible combinations of situations, it is impossible to implement this with a simple rule-based system.

[0040] Furthermore, a skilled safety manager can sense an intuition that something feels dangerous as soon as they enter a work site. This is tacit knowledge derived from decades of accumulated experience. However, attempting to express this as explicit rules presents a problem: it breaks down into fragmentary elements such as "the scaffolding is shaky," "the workers are rushing," or "it is disorganized." A combination of these fragmentary elements alone cannot replicate an expert's intuition.

[0041] Due to these characteristics of the industrial safety and health field, general educational system technologies cannot be used to generate industrial safety metacognitive assessment items, and a new, domain-specific technological approach tailored to the field is currently required.

[0042] FIG. 1 is an exemplary configuration diagram showing a workplace-customized industrial safety and health metacognitive evaluation and education system according to an embodiment of the present invention, and FIG. 2 and FIG. 3 are timing diagrams showing the operation process of a workplace-customized industrial safety and health metacognitive evaluation and education system according to an embodiment of the present invention. As shown in FIG. 1, the workplace-customized industrial safety and health metacognitive evaluation and education system according to an embodiment of the present invention includes an information processing unit (100), a matching processing unit (200), a question processing unit (300), a scoring processing unit (400), and a metacognitive processing unit (500). It is preferable that each component be integrated into a plurality of computational processing means including a CPU or a single computational processing means to perform operations. In addition, the operation process according to FIG. 2 and FIG. 3 is to be described as described below for each component.

[0043] It is preferable that the above information processing unit (100) receives workplace characteristic information corresponding to a preset item from the outside, processes it, and converts it into a semantic vector.

[0044] In detail, the information processing unit (100) receives workplace characteristic information corresponding to pre-set items through a terminal means possessed by a user requiring metacognitive evaluation and education. Pre-set items include the industry (e.g., construction, manufacturing, etc.), scale, ongoing processes (e.g., scaffolding installation, press work, etc.), work environment (e.g., high altitude, confined space, etc.), major equipment used (e.g., mobile crane, welding machine, etc.), number of workers, and contract type. At this time, the workplace characteristic information input is in the form of text data, and it is preferable for the information processing unit (100) to convert it into a high-dimensional semantic vector using a text embedding model (e.g., a BERT-based model, etc.). Since normalizing text data and converting it into a semantic vector in this manner is a standard technique, a detailed explanation is omitted.

[0045] It is preferable that the above-mentioned matching processing unit (200) uses a first DB that stores and manages accident-related data that occurred in the past industrial site to extract accident-related data that matches the vectorized workplace characteristic information through the above-mentioned information processing unit (100).

[0046] At this time, the first DB stores and manages accident-related data, including the circumstances, causes, and results of numerous accidents that occurred in past industrial sites, in the form of text data, and it is desirable to store and manage each accident-related data along with a unique semantic vector.

[0047] The matching processing unit (200) preferably extracts N top accident-related data with high relevance scores and N bottom accident-related data with low relevance scores from the first DB by applying the vectorized workplace characteristic information and cosine similarity, etc. (wherein N is a natural number greater than or equal to 1). Of course, extracting N top accident-related data and N bottom accident-related data in this way is merely one embodiment of the present invention and is not necessarily limited thereto.

[0048] Therefore, it is desirable for the matching processing unit (200) to perform a relevance evaluation on all accident-related data, not just each extracted accident-related data, and to calculate a relevance score. The relevance evaluation comprehensively evaluates the degree of consistency of industry, scale, and work environment, and calculates a relevance score based on the results. The relevance score calculated in this way is subsequently used as basic data for generating questions.

[0049] It is preferable that the above-mentioned question processing unit (300) generates question information for metacognitive evaluation by using at least one of the workplace characteristic information vectorized through the above-mentioned information processing unit (100) and the accident-related data extracted through the above-mentioned matching processing unit (200). The above-mentioned question information is generated as workplace-customized questions for three areas: understanding of laws and regulations, ability to identify risk factors, and ability to derive risk control methods.

[0050] To this end, the above-mentioned question processing unit (300) includes a statute processing unit (310), a first question generation unit (320), a basic generation unit (330), a final generation unit (340), a second question generation unit (350), a control processing unit (360), and a third question generation unit (370), as shown in FIG. 1.

[0051] It is preferable that the above-mentioned law processing unit (310) extracts industrial safety and health law provisions and related rules that match the workplace characteristic information vectorized through the above-mentioned information processing unit (100) by using a second DB that stores and manages all laws related to workplace safety and health, including the Industrial Safety and Health Act.

[0052] In this case, the second DB stores and manages all relevant laws in the form of text data, and it is desirable to store and manage each data along with a unique semantic vector.

[0053] It is preferable for the above-mentioned law processing unit (310) to extract the top N industrial safety and health law provisions and related rules having a high relevance score by applying the vectorized workplace characteristic information and cosine similarity, etc., from the above-mentioned second DB.

[0054] In detail, the above-mentioned law processing unit (310) searches for industrial safety and health law provisions corresponding to the vectorized workplace characteristic information in the above-mentioned second DB, classifies the searched provisions by topic such as safety management system, education and risk assessment, and extracts application conditions and obligations according to each topic.

[0055] Afterwards, it is desirable for the first question generation unit (320) to generate question information for evaluating legal comprehension using a Large Language Model (LLM) based on the provisions and rules extracted through the legal processing unit (310).

[0056] In the present invention, since the question information generated by the first question generation unit (320) is question information for evaluating the understanding of laws and regulations, it is limited to generating O / X type questions; however, this is merely one embodiment and is not necessarily limited thereto. Furthermore, it is desirable to generate question information so that the user can input a level of confidence in the answer for each question, rather than generating only O / X type questions. For example, question information can be generated as 'The employer must take fall prevention measures when working in a place with a height of 2m or more. O / X / Level of confidence in the selected answer (1, 2, 3, 4, 5)' (wherein the 5-point scale is merely one embodiment).

[0057] To give an example of the operation of the above-mentioned law processing unit (310) and the first question generation unit (320), if the text data of a legal provision extracted through the above-mentioned law processing unit (310) is 'The employer shall take measures to prevent falling when working in a place with a height of 2 meters or more,' the above-mentioned first question generation unit (320) inputs the text data of the legal provision as an input value to a large-scale language model. Through this, the large-scale language model sets the input value as a core obligation and generates a question in the form of a proposition that can determine whether it is true (O) or false (X). Through this question information for evaluating legal comprehension, the user's possession of legal knowledge can be directly evaluated.

[0058] As such, it is desirable for the first question generation unit (320) to generate true / false quizzes or short-answer questions that can directly verify whether the user knows the relevant laws accurately through the large-scale language model, thereby aiming to evaluate the user's basic level of legal knowledge.

[0059] An example of the question information generated through the first question generation unit (320) is "In accordance with Article 38 of the Industrial Safety and Health Act, an employer must take fall prevention measures when working in a place with a height of 2 meters or more. (O / X)".

[0060] It is preferable that the above basic generation unit (330) generates a basic scenario including the core risk factors of an accident using the accident-related data with the highest correlation score based on the correlation score of each accident-related data by the above matching processing unit (200).

[0061] In other words, a basic scenario is generated by utilizing the accident-related data with the highest relevance score among the extracted data, setting it as the core risk factor of the accident.

[0062] It is preferable that the final generation unit (340) generates confusion risk factors based on the remaining accident-related data, excluding the accident-related data with the highest correlation score, based on the correlation score of each accident-related data obtained by the matching processing unit (200), and generates a final scenario by including the confusion risk factors in the generated basic scenario. The process of generating the confusion risk factors involves generating a final scenario using core risk factors and appropriate confusion risk factors using a cognitive trap pattern extraction algorithm, and generating item information using a large-scale language model.

[0063] In other words, the cognitive trap pattern extraction algorithm generates confusion risk factors within the accident context that are not core or indirect risk factors of the accident, yet can be mistaken for risk factors by the user (evaluator).

[0064] To explain the cognitive trap pattern extraction algorithm described above, a multi-layered association matrix is ​​constructed in the first step. That is, the cognitive trap pattern extraction algorithm quantifies the 'multi-dimensional associations' between risk factors in an accident context. Specifically, accident-related data with high association scores is used as a reference point, and a number of accident-related data excluding this are compared to perform quantification on major category similarity (a value converted to 0-1 indicating how similar two risk factors are within the major risk factor classification system), accident context coexistence ratio (a value converted to 0-1 indicating the frequency with which two risk factors appear together in the accident circumstances of the same accident case among all accident cases), causal misconception probability (a value converted to 0-1 indicating the probability that the evaluated party will mistake a candidate risk factor for the main cause even though it is not the actual cause (direct or indirect risk factor)), and work association (a value converted to 0-1 indicating the practical probability that two risk factors will be observed simultaneously in the relevant work type). For example, the similarity of major categories is quantified as follows: pairs belonging to the same major category, such as 'temporary passageway ↔ temporary railing', are 0.8; pairs with high semantic similarity between major categories, such as 'temporary passageway ↔ system scaffolding', are 0.6; and pairs with low semantic similarity, such as 'temporary passageway ↔ tower crane', are 0.3. Additionally, the accident context coexistence ratio is quantified as follows: if accident cases in which 'temporary railing' and 'vertical safety net' are mentioned simultaneously account for 70 out of 100 total accident cases, it is 0.7; and if accident cases in which 'temporary railing' and 'tower crane' are mentioned together account for 10 out of 100 total accident cases, it is 0.1.

[0065] Furthermore, to quantify the potential for causal illusion, the aforementioned cognitive trap pattern extraction algorithm calculates a value by synthesizing causal clues combined with candidate risk factors in the accident description text (e.g., 'not installed,' 'defective,' 'damaged,' and 'disassembling,' etc.), the proximity within the sentence between candidate risk factors and accident result expressions (e.g., fall, entrapment, etc.), and the degree of repetition or emphasis of candidate risk factors. The score tends to increase as the association with causal clues is strong, the distance to accident result expressions is close, and the frequency of mention increases. Conversely, the score tends to decrease when clues appear only as background explanations (e.g., location, material name, topographical markers, etc.) or when there are no inaccurate descriptive (expression) clues. Based on this, for example, if the accident circumstances are described as "a worker stepped on a temporary railing and fell 7 m down while dismantling a vertical safety net," the candidate risk factor "vertical safety net" can be assessed as 0.6-0.7 because it is close to the causal clue "during dismantling work." In other words, although it is not the actual cause, there is a possibility of misidentification. If quantification is performed on the "reinforced earth retaining wall," the possibility of misidentification of causality is assessed as 0.0 because it is not mentioned in the description of the accident circumstances.

[0066] In addition, as described above, work relevance is a value that scores the practical probability of a candidate risk factor for confusion appearing together with a work process designated at the site in the same accident record, ranging from 0 to 1. To this end, the cognitive trap pattern extraction algorithm first collects all accident circumstances tagged identically to the work process received as field input. Then, the proportion of accident circumstances in which a candidate risk factor for confusion appears in the collected set of accident circumstances is calculated, and this proportion is used as the work relevance score. For example, if 75 accident circumstances records mention 'vertical safety net' out of 100 accident circumstances classified as 'scaffolding installation', the work relevance score becomes 0.75. In the same context, if there is 1 accident circumstances record mentioning 'excavator', the work relevance score becomes 0.01.

[0067] Of course, this quantification process is merely one embodiment of the present invention and is not necessarily limited thereto.

[0068] In the second step, a misconception score is calculated. The misconception score is an indicator that quantifies the 'probability that the subject will mistake the item for the actual cause' for each candidate risk factor. In the present invention, the 'misconception score' is defined as '(frequency of occurrence in context * 0.4) + (major category similarity * 0.3) + (likelihood of causal misconception * 0.3)', but this is merely one embodiment of the present invention and is not necessarily limited thereto.

[0069] Applying the example described above based on this, in an accident where the actual core risk factor is a 'temporary handrail,' if the candidate for confusion is a 'vertical safety net' (major category similarity 0.80, accident context coexistence ratio 0.70, possibility of mistaking causality 0.65, work association 0.75), the confusion inducement score is calculated to be approximately 0.72. Under the same conditions, if the candidate for confusion is a 'system scaffolding' (major category similarity 0.60, accident context coexistence ratio 0.50, possibility of mistaking causality 0.35, work association 0.60), it is calculated to be approximately 0.50, and if it is an 'elliptical crane' (major category similarity 0.30, accident context coexistence ratio 0.10, possibility of mistaking causality 0.10, work association 0.05), it is calculated to be approximately 0.14.

[0070] In the third step, the educationally optimal range is selected. This involves determining the educational value of the item based on the distribution of illusion-inducing scores.

[0071] In the present invention, values ​​less than 0.60 are excluded as "too obvious incorrect answers," values ​​in the range of 0.60 to 0.80 are prioritized as "appropriate confusion factors," and values ​​greater than 0.80 are excluded as "excessively difficult incorrect answers" to prevent learning confusion. Accordingly, applying the example described above, "vertical protection net" is selected as an appropriate confusion factor, while "system scaffolding" and "tower crane" are excluded due to reasons of under- and over-recognition of age, respectively. At this time, the threshold values ​​for adoption and exclusion can be adjusted according to the target learning group, item difficulty policy, and data characteristics, and are not necessarily limited to the embodiment described above.

[0072] In the fourth step, actual illusion patterns are verified. The algorithm is continuously improved by collecting illusions made by workers from accident-related data. For example, the algorithm is continuously improved by collecting illusion data (such as mentions of incorrect risk factors like "at first I thought it was a safety belt problem") using interview records from after an accident, analyzing problem patterns frequently made by workers during safety training, or converting recurring illusion cases discovered during on-site safety inspections into data.

[0073] It is preferable that the second item generation unit (350) generates item information for evaluating risk factor identification ability using the large-scale language model based on the final scenario by the final generation unit (340).

[0074] In detail, the second item generation unit (350) constructs a vivid work scenario based on similar workplace accident cases, that is, the circumstances of an accident that occurred at a workplace with similar workplace characteristics information, through the large-scale language model, and utilizes both core risk factors and low-relevance confusion risk factors to maximize the discriminability and realism of the items.

[0075] For example, regarding the question information generated through the second question generation unit (350), a question is generated such as, "External panel attachment work is being carried out on a scaffolding 4 stories high. A safety railing is installed at the end of the work platform, and the worker is wearing a safety helmet. At this time, a colleague briefly disconnected the vertical lifeline to which the worker had attached a safety hook in order to move materials. In this situation, what is the most immediate risk factor causing a serious accident?" thereby evaluating the user's ability to identify the real core risk among various information.

[0076] The most dangerous thing in the field of industrial safety and health is the conviction that an incorrect risk factor is the correct answer. For example, in the event of an accident where a worker falls from a height of three stories and sustains serious injuries while installing scaffolding at a new apartment construction site because he was not wearing a safety harness, the actual risk factor is "temporary structure: scaffolding," that is, the unsafe structure where the accident occurred. However, incorrect risk factors may be mentioned, such as "construction tool: safety harness," which is easily mistaken for the cause because it is mentioned in the accident report, or "construction machinery: excavator," which is a meaningless error completely unrelated to the accident.

[0077] Accordingly, in order to maximize the discriminability and realism of the item information generated through the second item generation unit (350) in the present invention, the basic scenario generated by the basic generation unit (330) is updated by utilizing a 'Distractor'—which is extracted through a cognitive trap pattern extraction algorithm and includes factors that a user might mistake for risk factors among various factors that are not core or indirect risk factors of the accident—to generate the final scenario.

[0078] In addition, a final scenario containing both core risk factors (risk factors appearing in the accident-related data with the highest relevance score) and confusion risk factors is input into the large-scale language model through the final generation unit (340) to generate a virtual work situation scenario. Afterward, question information such as "What is the risk factor that caused the most direct accident in the corresponding work situation (situation according to the virtual work situation scenario)" is finally generated to evaluate the user's ability to distinguish between superficial risks and direct core risks. Furthermore, it is desirable that the question information generated through the second question generation unit (350), just like the question information generated through the first question generation unit (320), be generated in such a way that the level of confidence in the input answer can also be input.

[0079] It is preferable for the control processing unit (360) to use a third DB that stores and manages control information regarding various risk factors to extract control information (appropriate control information and inappropriate control information) corresponding to the risk factor based on the accident-related data with the highest relevance score. Here, appropriate control information is processed as a correct answer choice, and inappropriate control information (control information that is related to the risk factor but is ineffective or inappropriate) is processed as an attractive incorrect answer choice.

[0080] It is preferable that the third question generation unit (370) generates question information for evaluating risk control derivation ability using the large-scale language model based on the control information extracted from the control processing unit (360). That is, by combining correct answer choices and incorrect answer choices using the control information through the large-scale language model, a question requiring in-depth understanding, such as "Select the safety measures necessary to prevent a specific risk," can be generated.

[0081] That is, the third item generation unit (370) mixes effective control information and inappropriate control information extracted through the large-scale language model for a given risk situation and presents them as options, and in particular, it is desirable to generate items that require selecting multiple correct answers.

[0082] The reason why the question information generated through the first question generation unit (320) and the second question generation unit (350) has only one correct answer, whereas the question information generated through the third question generation unit (370) requires multiple correct answers to be selected, is that due to the nature of risk control, no single specific control method can be the correct answer. In other words, since various control methods are required to control a single risk, generating questions in a multiple-choice and multiple-correct-answer format is practically helpful for improving competence in terms of safety and health; therefore, it is desirable to generate risk control methods as multiple-choice or multiple-correct-answer questions as well.

[0083] For example, regarding the question information generated through the third question generation unit (370), "options such as (1) immediately stopping work, (2) resuming work after reinstalling the lifeline, (3) continuing work after a verbal warning, (4) temporarily attaching the safety hook to the scaffolding pipe are presented," thereby evaluating the user's ability to comprehensively select effective countermeasures. In addition, it is desirable to generate question information so that the level of confidence in the input answer can be entered together.

[0084] In this way, through the above-mentioned question processing unit (300), beyond simple knowledge verification, it is possible to simulate complex judgment situations that may occur in actual field situations and generate a customized set of evaluation questions that can precisely measure the user's metacognitive ability (the ability to distinguish between what is known and what is not known) from various angles.

[0085] To explain in more detail, when applying conventional item generation technology during the process of generating item information in the item processing unit (300), there are various limitations that will be described later.

[0086] For example, if a simple random selection method is used, there is a high possibility of randomly selecting risk factors from the database (the second DB in this invention) or selecting meaningless incorrect answers that are too obvious, such as 'excavator' or 'concrete pump', that is, completely unrelated to accidents, resulting in no educational effect at all.

[0087] Furthermore, when using the keyword matching method, risk factors related to words in the accident description can be selected; that is, if 'safety belt' is mentioned, selecting 'safety belt' unconditionally can lead to the correct answer. This results in the generation of inappropriate incorrect answers due to superficial matching that ignores context.

[0088] Furthermore, if a general large-scale language model is used, it requests the generation of incorrect answers through the general-purpose model; in the case of high-altitude environments, it naturally generates incorrect answers related to 'height,' leading to unrealistic errors due to a lack of industrial safety domain knowledge, which also renders the training completely ineffective.

[0089] In addition, to overcome the limitations of existing RAG (Retrieval-Augmented Generation) / LLM systems, the present invention newly established a dual search RAG and a vessel design LLM.

[0090] Existing RAG / LLM systems are optimized for finding correct information and perform the processes of question / query input, document search, and answer generation. For example, if a user asks, "What are the control methods to prevent fall accidents?" (question / query input), documents related to keywords such as "fall," "control methods," and "prevention" are searched in the DB. As a result of the search, correct answer information such as "installation of safety railings," "installation of fall prevention nets," and "installation of safety harnesses" is retrieved (document search). Based on this, the correct answer information retrieved in the LLM is synthesized to generate an answer in the form of a sentence such as "To prevent fall accidents, installing safety railings, fall prevention nets, and wearing safety harnesses are effective" (answer generation).

[0091] While this is effective for verifying simple knowledge, it has a fatal limitation in evaluating industrial safety metacognition. In other words, although it can include plausible but incorrect answers—that is, completely irrelevant ones—it is limited in that it fails to capture the cognitive pitfalls that workers actually experience in the field.

[0092] Accordingly, to overcome the limitations of existing RAG / LLG systems, the present invention redesigned RAG into a dual-retrieval process that simultaneously searches for 'correct answers' and 'traps' rather than a single information search, and also transitioned LLM to design 'cognitive traps' based on two types of information rather than information summarization.

[0093] For example, when generating question information to evaluate the ability to derive risk controls for "worker falls from roofs," questions / queries such as "roof fall control methods" and "safety rules for working at heights" are entered, similar to existing RAGs. Based on the search results, the correct answer sets are identified as "installation of safety railings (removal / replacement)," "installation of fall prevention nets (engineering control)," and "wearing and fastening of safety harnesses (personal protective equipment)." Subsequently, to search for information on cognitive pitfalls, measures that workers commonly misunderstand or use inappropriately are searched while simultaneously finding the correct answers. Questions / queries such as "examples of inappropriate fall accident measures," "accidents caused by safety signs alone," and "accidents that failed to be prevented by safety training alone" are entered. Based on the search results, the pitfall sets identified are "conducting safety training only (administrative control)," "installing hazard warning tapes (administrative control)," and "stationing only safety supervisors (administrative control)."

[0094] Subsequently, LLM receives both sets of search results (correct answer set and trap set) and designs educational traps rather than simply summarizing the information to generate sentences.

[0095] The accident context, 'falling accident while working on a roof,' and the correct answer set and trap set are input into the LLM, and the item information is generated by setting design instructions such as 'combining 2 from the correct answer set and 3 from the trap set to create a multiple-choice question that is easily confused in actual industrial sites,' and 'configuring the trap options so that they can be solved by recognizing that they are insufficient on their own rather than being incorrect.'

[0096] An example of the generated question information is as follows: 'Question: Select all of the most effective control methods for fall accidents during work on a roof. ① Installation of safety railings (Correct Answer), ② Wearing and fastening of safety harnesses (Correct Answer), ③ Conducting special safety training before work (Correct Answer), ④ Attaching warning tape to an area spaced away from the edge of the roof (Incorrect Answer - insufficient physical protection), ⑤ Requiring workers to submit a "Personal Health Status Confirmation Pledge" before work (Incorrect Answer - not direct fall prevention).' When examining the generated question information in this way, a worker with only basic knowledge might consider options ④ and ⑤ to be plausible measures and select incorrect answers, whereas only a worker who understands the clear risk control measures—such as the priority order of risk control (Elimination -> Replacement -> Engineering Control -> Administrative Control -> Personal Protective Equipment)—would be able to get the correct answer.

[0097] In addition, LLM tends to express the same content using different vocabulary and sentence structures. For example, when generating item information through LLM such as '1. A dedicated safety manager must be appointed at construction sites with a construction cost of 12 billion KRW or more,' '2. Safety managers at construction sites with a construction cost of 12 billion KRW or more cannot hold other concurrent positions,' and '3. Safety managers must devote themselves solely to the relevant site if the construction cost is 12 billion KRW or more,' the corresponding item information is all legally identical, but existing text similarity or keyword matching cannot detect duplication.

[0098] Accordingly, the present invention utilized a three-stage semantic equivalence determination algorithm to resolve semantic redundancy.

[0099] Instead of comparing sentences as they are in natural language, legal element abstraction was performed by converting them into structured tuples of subject, condition, obligation, and exception, which are the core elements constituting legal meaning. To continue from the example above, it can be converted into '("Employer", "12 billion or more", "Appointment of dedicated safety manager", none)', '("Employer", "12 billion or more", "Prohibition of concurrent employment for safety manager", none), ("Employer", "12 billion or more", "Dedication to safety manager", none).

[0100] Subsequently, semantic normalization is performed to unify the abstracted elements into a standard form using the previously established industrial safety synonym and antonym dictionary.

[0101] Through this, semantic normalization can be performed on 'dedicated placement', 'prohibition of concurrent employment', and 'dedication' to "concurrent employment prohibited", and on 'over 12 billion won' and 'over 12 billion won' to "≥12 billion won".

[0102] As a result, all sentence structures are normalized to ("business owner", "≥12 billion", "concurrent employment not allowed", none). Afterwards, the normalized tuples are compared, and if the structure and content match completely, they are finally determined to be semantically identical items, and by removing duplicates, semantic redundancy in the generated item information can be prevented.

[0103] It is preferable that the scoring processing unit (400) receives response information corresponding to the question information generated through the question processing unit (300) and generates scoring result information.

[0104] The scoring processing unit (400) receives, as the response information, answer information for each question (answer information selected by the user) and confidence level information for the answer information (a subjective judgment value regarding how confident the user is that the answer information selected by the user is correct) on a scale of 1 to 5 points, and generates the scoring result information using these.

[0105] More specifically, the question information generated through the question processing unit (300) is transmitted to the user's terminal means to receive the response information.

[0106] Afterward, the scoring processing unit (400) compares the previously stored correct answer data with the response information to determine whether the response information is correct. In addition, the scoring result information is generated by including the confidence level information based on this.

[0107] In this case, the present invention established a probabilistic answer modeling and an evaluator-AI collaboration system to resolve the ambiguity of the correct answer based on the generated question information.

[0108] In the field of industrial safety and health, the correct answer is not always clear-cut. For example, in item information for evaluating the ability to derive risk controls, the weight of the correct answer differs between "installation of safety railings (mandatory)" and "placement of safety supervisors (correct answer depending on the situation)." Conventional binary (True / False) scoring methods fail to handle such contextual correct answers, leading to problems that undermine the realism and fairness of the evaluation.

[0109] Accordingly, the correctness of the response information is analyzed through a model that calculates the probability of being correct (0.0 - 1.0) for each option included in the above question information, rather than an absolute correct answer (1) or incorrect answer (0). The probability of being correct is calculated by combining the following weights, which can be defined as 'Probability of Correct Answer = P(Mandatory) * 0.6 + P(Recommended) * 0.3 + P(Context) * 0.1'. P(Mandatory) is whether it is a mandatory measure specified in regulations, and P(Recommended) is whether it is a measure recommended by industry standards or guidelines. P(Context) refers to the relevance with the input workplace information.

[0110] In this way, after performing a preliminary score based on the probability model, ambiguous items with a correct answer probability between 0.3 and 0.7 are automatically flagged as 'boundary cases'. Subsequently, the expert evaluator focuses on reviewing only the flagged items and inputs a final judgment based on field experience. As the probability model processes the input final judgment and reasoning as feedback, the probability model consequently comes to operate closer to the expert evaluator's judgment over time.

[0111] Through this, the limitations of mechanical scoring are overcome, and scoring result information is generated through fair evaluation that reflects the complexity and ambiguity of the industrial field.

[0112] It is preferable that the metacognitive processing unit (500) analyzes the input response information and the generated scoring result information to generate user metacognitive evaluation information.

[0113] It is desirable to go beyond simply verifying correct or incorrect answers through the scoring results information above, and to analyze the 'level of confidence information' contained in the responses to identify dangerous cognitive biases such as over-confidence using the data.

[0114] More specifically, the confidence level information is compared with a preset threshold to determine whether the confidence level is high or low. Subsequently, the correct answer status and the confidence level can be combined to establish four metacognitive states, and it is desirable to classify the response information into the corresponding metacognitive state among the four states and generate this as the metacognitive evaluation information.

[0115] It is desirable to have four metacognitive states: High-Confidence Hit (HH) (a state where one accurately judges what is known), Low-Confidence Hit (LH) (a state where one believes the answer is correct but lacks confidence), High-Confidence Miss (HM) (a state where one strongly mistakenly believes something is correct despite being wrong), and Low-Confidence Miss (LM) (a state where one is aware of what is unknown).

[0116] Afterwards, the present invention further includes an education processing unit (600) as illustrated in FIG. 1 to generate customized educational content with a clear goal of correcting identified cognitive biases.

[0117] It is desirable for the above education processing unit (600) to analyze the user's vulnerabilities (vulnerable areas) using the metacognitive evaluation information and to generate customized education content data based on the analyzed vulnerabilities using a large-scale language model.

[0118] In the present invention, among the metacognitive states included in the metacognitive evaluation information, the 'wrong answer-high confidence state,' which is a state of 'overconfidence error' where one believes incorrect knowledge to be correct rather than simply lacking knowledge, is identified as the most dangerous 'cognitive vulnerability area.' When a user is identified as being in an overconfidence error state, it is desirable to set a clear educational goal for the user to 'correct overconfidence bias regarding specific risk factors (topics of questions)' and to generate customized educational content data.

[0119] That is, the above education processing unit (600) transmits structured commands (Promts) such as Table 1 below to a large-scale language model to achieve the set education goal, and this has a clear purpose of ‘correcting cognitive bias’ rather than simple information summarization.

[0120] (1) Context Information: "The following is information regarding the accident case and relevant laws evaluated by the user." - Original Accident Case Text: Provide the text of the 'Circumstances of the Accident Case' that served as the basis for the evaluation questions. - Relevant Laws: Provide the text of the 'Articles of the Occupational Safety and Health Act' related to the accident case. (2) Error Diagnosis Information: "In this situation, the user responded with 'high confidence' that '[The incorrect answer selected by the user]' was the correct measure. This corresponds to a 'High-confidence Miss' error." (3) Instruction & Rules: "Create educational materials aimed at correcting the user's overconfidence bias mentioned above. The educational materials to be created must follow the following rules." - Rule 1 (Direct Error Pointing): Clearly and directly point out why the answer selected by the user is incorrect and how dangerous the behavior is in the actual field. - Rule 2 (Emphasis on Result): Explicitly mention the fatal consequences (e.g., death, serious injury, etc.) of the actual accident case that occurred due to the erroneous judgment to emphasize the severity of the risk. Highlights: - Rule 3 (Presentation of Correct Procedures): Clearly state what the correct safety measures were based on relevant laws. - Rule 4 (Contrastive Explanation): Specifically compare the incorrect measures chosen by the user with the correct measures and explain what differs and how. - Rule 5 (Setting the Tone): Use a firm and objective tone that can raise awareness of the risks without unnecessarily provoking the user's pride.

[0121] Through these commands, it is desirable for the aforementioned large-scale language model to generate unique and persuasive 'corrective feedback' content in real time, tailored exclusively to the user.

[0122] Tables 2 to 5 below are examples of the customized educational content data generated for the same accident (falling accident while working on scaffolding at a 4-story building at a construction site) according to each metacognitive state classification result generated by the education processing unit (600).

[0123] Cases classified as incorrect & low confidence - Creation Goal: Recognizing the lack of knowledge, establish a foundation of correct knowledge by providing clear, step-by-step education starting from the most basic concepts. - LLM Command Rules: (1) Use a kind and encouraging tone. (2) Present the most core and basic safety principles first. (3) Instead of complex regulations or advanced content, clearly explain essential equipment and its roles. - Generated Training Material: - "Let's start safely from the very basics. When working at heights of 2 meters or more, such as on scaffolding, a 'safety helmet' protects the head from falling objects, but it cannot prevent the 'fall' itself. [Step 1: The 3 Elements of Fall Protection] - To prevent falls, a 'Personal Fall Arrest System' is absolutely necessary. This equipment consists of three parts: ① a 'Harness' that wraps around the body, ② a 'Lanyard' that connects the harness to the lifeline, and ③ an 'Anchor Point,' a sturdy anchor point to attach the lanyard. Please check the names and roles of each piece of equipment first through the basic video below."

[0124] Cases classified as incorrect & high confidence - Creation Goal: To directly correct the user's dangerous 'overconfidence' and make them clearly realize, through shock therapy, that incorrect knowledge can lead to actual fatal accidents. - LLM Command Rules: (1) Explain in a firm and objective tone why the user's incorrect answer is dangerous. (2) Induce cognitive dissonance by specifically presenting relevant actual or hypothetical 'cases of fatal accidents'. (3) Clearly contrast the incorrect method with the correct method. - Generated Educational Material - "You were convinced that attaching the safety hook to the 'scaffolding pipe' was safe, but this action is extremely dangerous as it threatens the lives of both you and your colleagues in the event of a fall. [Analysis of Fatal Accident Cases] - In 2023, at the OO new construction site, there was an accident in which a worker fell while working with a safety hook attached to a scaffolding steel pipe, just like you. The scaffolding pipe, unable to withstand the impact load, collapsed along with the worker, resulting in death. Scaffolding pipes are designed only to withstand working loads and cannot withstand the impact load of hundreds of kilograms generated during a fall. You must thoroughly read the material explaining why the 'safety harness attachment equipment' must be secured to a separately installed structure, and clearly understand the difference between the two methods."

[0125] Cases classified as correct & low confidence - Creation Goal: To instill confidence by praising the user's accurate judgment, and to solidify knowledge by presenting the legal and technical basis for it. - LLM Command Rules: (1) First, acknowledge and praise the user for being correct. (2) Explicitly present the legal basis (relevant statutory provisions) explaining why the measure is essential. (3) Include a message of encouragement to help the user have confidence in their knowledge. - Generated Training Material - "That is an accurate judgment! Regardless of incorrect practices in the field, installing a 'separate safety harness attachment system' is the most correct answer and the only lifeline. You may be confident in your own accurate judgment. [Knowledge Reinforcement: Verify Legal Basis] - Please verify that your choice is a legal obligation specified in 'Article 44 (Safety Harness Attachment System, etc.) of the Rules on Occupational Safety and Health.' The regulations specify that the safety harness attachment system must be 'securely installed' and 'capable of withstanding impact loads generated in the event of a fall.' Clearly understanding the legal basis allows you to explain and act more confidently on why principles must be followed in the field."

[0126] Cases classified as correct & high certain: - Creation Goal: To acknowledge the user's expertise and present applied / advanced tasks that go beyond simple knowledge verification to evaluate and strengthen problem-solving abilities for complex issues that may arise in the field. - LLM Command Rules: (1) Convey a message acknowledging the user's expertise. (2) Present advanced situations or tasks from a managerial perspective that are related to the current topic but take it a step further. (3) Stimulate application skills through descriptive or planning-oriented questions with no fixed correct answer. - Generated Training Material - "Perfect. You clearly understand the core principles of fall prevention measures during scaffolding work. [Advanced Assignment: Work Plan Review] - Now, assume you are a supervisor and handle the following situation. The fixing point of the vertical lifeline installed by the workers is not directly above, but is installed diagonally to account for the workers' radius of activity. In this case, explain the risk of the 'Pendulum Effect' that may occur in the event of a fall, and briefly describe how the lifeline installation plan should be modified to secure a safe working radius. Your in-depth knowledge can prevent potential dangers to your colleagues as well."

[0127] In addition, the present invention is not limited to generating customized educational content data and providing it to a user's terminal device, but also, as shown in FIG. 1, it is desirable to track the educational history through a tracking processing unit (700) to check the degree of improvement.

[0128] More specifically, it is preferable that the tracking processing unit (700) provides the customized educational content data generated by the education processing unit (600) to a user's terminal means, tracks the education history of the provided customized educational content data, and performs feedback processing of the corresponding user's metacognitive evaluation information.

[0129] The workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention has the effect of enabling the execution of an industrial safety and health education and training platform business. Specifically, the workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention can be utilized to replace or supplement industrial safety and health education courses based on an online platform. By automatically generating customized evaluation questions using actual accident cases and risk scenarios specific to each workplace, it is possible to provide educational content with high suitability for the field. Through this, companies and institutions can systematically manage training completion records required by law while reducing the time and costs invested in in-house instructors or outsourced training. Furthermore, since evaluation results are stored in real-time, there is the advantage of enabling record management that can objectively prove the quality of education and learning effectiveness.

[0130] In addition, the workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention can be utilized as an integrated corporate EHS (Environment, Health, and Safety) management solution.

[0131] More specifically, the calculated metacognitive indicators and risk prediction indices can be directly linked to corporate safety management indicators. By connecting to ERP or smart factory platforms via API, work process data is automatically reflected, enabling immediate matching of accident cases and the prescription of training based on this data. Furthermore, management can check metacognitive levels by process and job function through a dashboard and take measures to prioritize improvements in vulnerable areas.

[0132] In addition, the workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention can be utilized as a smart site management module for construction and plants.

[0133] By directly linking site data collected from BIM, IoT sensors, and drone footage at smart construction sites with site-specific information, accident cases can be re-searched and new evaluation questions generated based on the latest vector data when changes in work processes or the environment are detected. This offers managers the advantage of disseminating changed risks to on-site workers in real time and immediately verifying training completion.

[0134] In addition, the workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention can be utilized as a vocational training or university safety education curriculum.

[0135] By introducing this into practical training for industrial safety courses at vocational high schools and engineering colleges, individual students' risk awareness levels can be quantitatively assessed. Students can input their confidence levels while solving free-response tasks and improve their metacognitive abilities through immediate feedback. Furthermore, educational institutions can accumulate learning data and utilize it as a basis for improving curriculum design and evaluation systems.

[0136] The domestic industrial safety education market has a low digital transformation rate, leading to an increasing demand for on-site customized online solutions. In line with this, the workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention can secure a competitive advantage differentiated from existing general e-learning services or standalone VR solutions by providing accident case search, quantitative metacognitive evaluation, and AI-based education prescriptions on a single platform.

[0137] Furthermore, since the large-scale language model supports multilingual prompts, the same system can be used in other countries by additionally building databases of accident cases and regulations from overseas sites. In other words, when the workplace-customized industrial safety and health metacognitive assessment and training system according to one embodiment of the present invention is applied to international market entry and multilingual services, it has the advantage of maintaining a consistent safety training system across processes in various countries at global EPC companies or multinational manufacturers, and standardizing training quality through customized questions automatically generated in local languages.

[0138] Furthermore, when the workplace-customized industrial safety and health metacognition evaluation and education system according to one embodiment of the present invention is utilized in a revenue model and commercialization roadmap, the main revenue sources may consist of subscription-based SaaS usage fees, provision of metacognition indicator analysis reports, API licenses for integration with external systems, and customized data construction services. There is an advantage in that phased commercialization is possible, initially by operating a pilot service targeting large construction and manufacturing companies to verify performance, and subsequently expanding to small and medium-sized workplaces and educational institutions.

[0139] Meanwhile, a workplace-customized industrial safety and health metacognitive evaluation and education system according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various means of electronically processing information and recorded on a storage medium. The storage medium may include program instructions, data files, data structures, etc., either individually or in combination.

[0140] Program instructions recorded on a storage medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a device that processes information electronically using an interpreter, such as a computer.

[0141] As described above, the present invention has been explained with specific details such as specific constituent elements and limited exemplary drawings; however, this is provided merely to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above-mentioned exemplary embodiment. Those skilled in the art can make various modifications and variations from this description.

[0142] Accordingly, the scope of the present invention should not be limited to the described embodiments, and all things equivalent to or having equivalent variations to the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention. Explanation of the symbols

[0143] 100 : Information processing unit 200 : Matching processing unit 300 : Item processing unit 310: Legislation Processing Unit 320: 1st Question Generation Unit 330 : Basic generation section 340 : Final generation section 350: Second question generation unit 360: Control processing unit 370 : 3rd Item Generation Section 400 : Scoring Processing Unit 500 : Metacognitive processing unit 600 : Education Processing Department 700 : Trace processing unit

Claims

Claim 1 An information processing unit that receives workplace characteristic information, including at least one of industry, scale, ongoing process, work environment, major equipment, and number of workers, through a terminal means possessed by a user requiring metacognitive evaluation and education, and processes and converts it into a semantic vector; a matching processing unit that uses a first DB storing and managing accident-related data that occurred in past industrial sites to extract accident-related data that matches the vectorized workplace characteristic information based on the similarity between the stored accident-related data and the vectorized workplace characteristic information; a question processing unit that generates corresponding workplace-customized question information for metacognitive evaluation using at least one of the vectorized workplace characteristic information and the extracted accident-related data; and a scoring processing unit that receives response information, including answer information for each question information and confidence level information for the corresponding answer information, corresponding to the generated question information, and generates scoring result information using the above. A workplace-customized industrial safety and health metacognitive evaluation and education system comprising: a metacognitive processing unit that generates user metacognitive evaluation information for a corresponding workplace by analyzing the correct answer information and the degree of confidence information, and classifying the correct answer information and the degree of confidence level into a metacognitive state that cross-combines the correct answer information and the confidence level. Claim 2 In claim 1, the matching processing unit extracts a plurality of accident-related data matched in the first DB based on the similarity between stored accident-related data and vectorized workplace characteristic information, and performs a relevance evaluation for each extracted data to calculate a relevance score, thereby creating a workplace-customized industrial safety and health metacognitive evaluation and education system. Claim 3 In claim 2, the above-mentioned item processing unit utilizes a second DB storing and managing the Industrial Safety and Health Act and related rules to extract industrial safety and health statute provisions and related rules that match vectorized workplace characteristic information based on the similarity between the stored Industrial Safety and Health Act and related rules and vectorized workplace characteristic information; and a first item generation unit generates item information for evaluating statute comprehension using a Large Language Model (LLM) based on the extracted provisions and rules; comprising a workplace-customized industrial safety and health metacognitive evaluation and education system. Claim 4 In claim 3, the item processing unit further comprises: a basic generation unit that generates a basic scenario including a core risk factor of an accident using the accident-related data with the highest relevance score; a final generation unit that generates a confusion risk factor based on the remaining accident-related data excluding the accident-related data with the highest relevance score, and generates a final scenario by including the confusion risk factor in the generated basic scenario; and a second item generation unit that generates item information for evaluating risk factor identification ability using the large-scale language model based on the final scenario; a workplace-customized industrial safety and health metacognitive evaluation and education system. Claim 5 In claim 4, the above-mentioned item processing unit further comprises: a control processing unit that extracts control information corresponding to a risk factor based on accident-related data with the highest relevance score using a third DB that stores and manages control information regarding various risk factors; and a third item generation unit that generates item information for evaluating risk control derivation ability using a large-scale language model based on the control information; a workplace-customized industrial safety and health metacognitive evaluation and education system. Claim 6 delete Claim 7 In claim 1, the workplace-customized industrial safety and health metacognitive evaluation and education system further comprises an education processing unit that analyzes vulnerabilities using the metacognitive evaluation information and generates customized education content data according to the analyzed vulnerabilities using a Large Language Model (LLM). Claim 8 In claim 7, the workplace-customized industrial safety and health metacognitive evaluation and education system further comprises a tracking processing unit that provides the generated customized education content data, tracks the education history of the provided customized education content data, and performs feedback processing of the metacognitive evaluation information.

Images