Risk hidden danger diagnosis method and system based on large language model

Abstract

The invention relates to the technical field of natural language processing and risk management, in particular to a risk hidden danger diagnosis method and system based on a large language model, and the method comprises the steps: confirming a cue word structure and a teacher model according to a pre-trained large language model; according to the confirmed cue word structure and the teacher model, utilizing a LoRA method and a step-by-step distillation method to finely adjust the student model to obtain a risk diagnosis model; and obtaining to-be-diagnosed risk information data in the factory, inputting the risk information data into the risk diagnosis model, reasoning an accident result by using the risk diagnosis model, and generating an analysis report. The method can accurately predict accident consequences and automatically generate an analysis report.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and risk management, and specifically relates to a risk and hidden danger diagnosis method and system based on a large language model. Background Art

[0002] Work safety is an important cornerstone for the development of enterprise factories and also a guarantee for economic development and social stability. Currently, factories are paying increasing attention to factory production safety, and the traditional paper-based management method can no longer meet the growing work safety needs of enterprises. Therefore, various "dual prevention and control" management systems have emerged. These systems aim to improve the efficiency and accuracy of risk management and hidden danger investigation through information technology means. Although the "dual prevention and control" management systems have solved the deficiencies of paper-based management to a certain extent, the existing systems still have the following problems: Most systems only implement the function of information entry, lacking in-depth analysis and utilization of the entered information, resulting in the failure to fully exert the value of the information; the chemical accident risk prediction method and device disclosed in the patent application document with the authorization announcement number CN113705074B, although a SVM accident risk prediction model is established, can only provide prediction results and cannot generate corresponding analysis reports at the same time. This makes it difficult for users to understand the basis of the prediction results and the influencing factors of potential risks. In recent years, significant progress has been made in the field of natural language processing (NLP). In particular, large language model technologies represented by ChatGPT have attracted wide attention globally. Large language models have excellent performance in liberating repetitive mental activities, low-cost and high-efficiency migration applications, etc. Although large language models perform well in general domain question-and-answer systems, they still face challenges in the field of risk and hidden danger, such as insufficient professionalism and high requirements for hardware resources. For example, the patent application document with the publication number CN117493513A discloses a question-and-answer system and method based on vectors and large language models. However, current large language models such as Tongyi Qianwen series can provide basic question-and-answer and classification work, but facing the characteristics of strong professionalism and complex situations in the field of risk and hidden danger, they usually cannot provide relatively accurate answers, and require a large amount of hardware resources in the case of local deployment or training. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a risk and hidden danger diagnosis method and system based on a large language model for solving the technical problem of insufficient accuracy of existing risk management systems in view of the above-mentioned deficiencies in the prior art.

[0004] The object of the present invention is achieved by the following technical solutions: In a first aspect, the present invention provides a risk and hidden danger diagnosis method based on a large language model, including: Confirm the prompt structure and the teacher model according to the pre-trained large language model; the teacher model is the first large language model with the highest accuracy among the selected multiple large language models; Fine-tune the student model using the LoRA method and the step-by-step distillation method according to the confirmed prompt structure and the teacher model to obtain a risk diagnosis model; the student model is the second large language model; Obtain the risk information data to be diagnosed in the factory, input the risk information data into the risk diagnosis model, use the risk diagnosis model to infer the accident result, and generate an analysis report.

[0005] As a further improvement of the present invention, confirming the prompt structure and the teacher model according to the pre-trained large language model specifically includes: Obtain the risk hidden danger related data according to the risk control system and perform preprocessing to form a data set; the risk hidden danger related data at least includes risk points, evaluation objects, hazardous and harmful factors, and accident consequences; Construct multiple blank large language models, train each large language model using the data set respectively; test the trained large language models using the zero-shot prompting method and the few-shot prompting method respectively, and select the prompting method with the highest accuracy as the final prompt structure; During the training process, evaluate the accuracy of the accident results output by each large language model, and select the large language model with the highest accuracy as the teacher model.

[0006] As a further improvement of the present invention, the large language model at least includes two of the qwen series models, GLM series models, Doubao series models, GPT series models, Flamingo, and BlenderBot.

[0007] As a further improvement of the present invention, Fine-tune the student model using the LoRA method and the step-by-step distillation method according to the confirmed prompt structure and the teacher model, specifically including: Update the parameters of the student model using the LoRA method. The formula for updating the parameters by the LoRA method is:

[0008] In the formula, is the parameter initialized by the pre-trained model, is the parameter to be updated, B and A are low-rank decomposition matrices, and the rank .

[0009] As a further improvement of the present invention, according to the confirmed prompt structure and teacher model, the teacher model is fine-tuned using the LoRA method and the step-by-step distillation method, specifically including: The step-by-step distillation method includes: prompting the teacher model to generate accident analysis results in the way of Few-shot CoT prompt template for the input risk information data, and using the generated accident analysis results as additional labels to train the risk diagnosis model; the teacher model is used to supervise the student model.

[0010] As a further improvement of the present invention, risk points, evaluation objects and hazardous and harmful factors are embedded in the few-shot prompt template, and the output format is limited to JSON. The prompt template includes example inputs and example outputs.

[0011] As a further improvement of the present invention, the risk diagnosis model is trained through a comprehensive loss function, and the comprehensive loss function is:

[0012]

[0013]

[0014] In the formula, is the comprehensive loss function; is the balance parameter; is the cross-entropy loss of the accident consequence; is the cross-entropy loss of the analysis generation; is the cross-entropy loss between the predicted token and the target token, is the input, is the output result obtained by the training risk diagnosis model according to the input, is the label in the dataset, that is, the corresponding accident consequence; is the corresponding analysis generated by the teacher model.

[0015] In the second aspect, the present invention provides a risk and hidden danger diagnosis system based on a large language model for implementing the above-mentioned risk and hidden danger diagnosis method based on a large language model, including: A data processing module to confirm the prompt structure and teacher model according to the pre-trained large language model; A model fine-tuning module to fine-tune the student model using the LoRA method and the step-by-step distillation method according to the confirmed prompt structure and teacher model to obtain a risk diagnosis model; A risk diagnosis module to diagnose the potential risks of the current factory based on the risk information data to be diagnosed input to the risk diagnosis model, infer the accident result, and generate an analysis report.

[0016] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, the one or more programs including instructions which, when executed by a computing device, cause the computing device to execute the above-described large language model-based risk and hazard diagnosis method.

[0017] In a fourth aspect, the present invention provides a computing device, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the above-described large language model-based risk and hazard diagnosis method.

[0018] The beneficial effects of the present invention are as follows: The present invention provides a large language model-based risk and hazard diagnosis method for diagnosing potential risks in factories. By using a pre-trained large language model and the step-by-step distillation method, the model can learn rich language knowledge and reasoning abilities, thereby improving the accuracy of accident cause analysis. Using the step-by-step distillation method, based on the existing data set, the teacher model is used to generate the cause of the accident consequence inferred from the basic information, that is, the correlation analysis. Adding the correlation analysis to the training set can provide users with a more accurate accident consequence prediction while providing relevant analysis for users, solving the problem that relevant analysis cannot be automatically generated in the risk field, and laying a foundation for downstream tasks such as providing analysis reports later. Secondly, the present invention uses the LoRA method to fine-tune the model using this data set, reducing the hardware resource consumption by 46.80% during training, and solving the problems of high hardware conditions required for local deployment and training of large models. The present invention also automatically generates accident analysis reports, reducing the time and effort of manual analysis and improving work efficiency.

[0019] Furthermore, by comparing the effects of zero-shot and few-shot prompting methods and selecting the prompting method with the highest accuracy, the accuracy of the model in generating accident analysis results can be significantly improved. By using the prompt structure to guide the model to generate detailed accident analysis results, the interpretability of the model can be improved, making the final analysis report more readable and practical.

[0020] Furthermore, by using the high-quality labels generated by the teacher model for training, the risk diagnosis model can learn more accurate and detailed accident analysis results, thereby improving its prediction accuracy. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of the large model training method for the risk field proposed by the present invention.

[0023] Figure 2 It is a flowchart of the risk diagnosis system in the risk field of the present invention for diagnosing risks.

[0024] Figure 3 It is a comparison chart of the accuracy rates of different prompt words.

[0025] Figure 4 It is a comparison chart of the accuracy rates of each large model.

[0026] Figure 5 It is a comparison chart of the accuracy rates of each training method.

[0027] Figure 6 It is a schematic structural diagram of the electronic device of the present invention. Detailed implementation manners

[0028] In order to make the purpose and technical solutions of the present invention clearer and easier to understand. The following will further elaborate on the present invention in detail in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the accompanying drawings and specific embodiments. Among them, the described embodiments are only some embodiments of the present invention, rather than all embodiments.

[0030] Embodiment 1 As Figures 1-5 shown, this embodiment provides a risk and hidden danger diagnosis method based on a large language model. The following are the specific implementation manners.

[0031] First, confirm the prompt word structure and the teacher model according to the pre-trained large language model.

[0032] Among them, the pre-trained large language model in this embodiment is trained and selected according to the dataset of risk and hidden danger related data. To ensure the accuracy of risk and hidden danger diagnosis, this embodiment constructs multiple blank large language models, trains each large language model based on the dataset, and tests the accuracy of each model, and selects the large language model with the best accuracy as the teacher model.

[0033] Specifically, based on the existing risk control system, relevant data on risk hazards is obtained and preprocessed to form a dataset. In this embodiment, the relevant data on risk hazards at least includes risk points, evaluation objects, hazardous and harmful factors, and accident consequences; Construct multiple blank large language models, and use the dataset to train each large language model respectively; Use the zero-shot prompting method and the few-shot prompting method to test the trained large language models respectively, and select the prompting method with the highest accuracy as the final prompting word structure; During the training process, evaluate the accuracy of the accident results output by each large language model, and select the large language model with the highest accuracy as the teacher model.

[0034] As an example in this embodiment, first randomly divide the dataset into a training set and a test set according to a ratio of 7:3. Extract several samples from the test set, and use the zero-shot prompting method and the few-shot prompting method to test respectively to obtain the accident consequence results output by the model. By comparing the accuracies of the output results under the two prompting methods, select the prompting method with better accuracy performance and use it as the prompting method adopted by the subsequent model operation, so as to ensure the reliability and effectiveness of the model in practical applications.

[0035] Then use the entire test set to evaluate the accuracy of the accident consequences output by each Chinese large model, and select the large model with higher accuracy as the teacher model in the distillation idea.

[0036] In this embodiment, the types of large language models include multiple types, including at least two of the qwen series models, GLM series models, Doubao series models, GPT series models, Flamingo, BlenderBot, DistilBERT, BERT series models, and CLIP. The qwen series models include models such as qwen2.5-1.5b, qwen2.5-14b, qwen2.5-32b, qwen2.5-72b, and qwen-plus; The GLM series models include models such as GLM3-130B; The GPT series models include models such as GPT-4 and GPT-3; The Doubao series models include models such as Doubao-pro-128k.

[0037] According to the confirmed prompting word structure and teacher model, use the LoRA method and the step-by-step distillation method to fine-tune the student model to obtain a risk diagnosis model.

[0038] In this embodiment, both the teacher model and the student model are large language models. Among them, the teacher model is the first large language model, which has strong reasoning ability and high accuracy of the output results, and is a model with a large number of parameters. The student model is the second large language model, with poor reasoning ability and small parameters itself. Therefore, the teacher model is used as a supervisor to supervise the student model and assist in the training of the student model. For example, in this embodiment, Doubao-pro-128k is selected as the teacher model, and qwen2.5-1.5b is selected as the student model.

[0039] The parameters of the student model are updated using the LoRA method, and the LoRA method is used to improve the efficiency of model training and reduce the occupation of hardware resources. The formula for updating parameters using the LoRA method is:

[0040] In the formula, are the parameters initialized by the pre-trained model, are the parameters to be updated, B and A are low-rank decomposition matrices, and the rank . If it is a full-scale fine-tuning situation, the number of parameters to be trained is equal to , and in the case of using LoRA fine-tuning, the number of parameters to be trained is equal to . In the case, the number of training parameters can be significantly reduced, thus greatly reducing the occupation of hardware resources.

[0041] The step-by-step distillation method includes: prompting the teacher model to generate accident analysis results in the way of the Few-shot CoT (Chain-of-Thought) prompt template for the input risk information data, and using the generated accident analysis results as additional labels to train the risk diagnosis model. Specifically, in the way of step-by-step distillation, the data is prompted to the teacher model through Few-shot CoT to generate an analysis of why the corresponding accident consequences occur (i.e., the derivation process), and the obtained accident consequence analysis is added to the task labels to train a small downstream task model. The downstream task model in this embodiment refers to the target model, that is, the risk diagnosis model in this embodiment.

[0042] In the training process of this embodiment, the risk diagnosis model is trained through a comprehensive loss function. Among them, the comprehensive loss function is:

[0043] In the formula, is the comprehensive loss function; is the balance parameter; is the cross-entropy loss of the accident consequences; is the cross-entropy loss of the generated analysis.

[0044] Among them, the accident consequences during training and the calculation method of the loss function for analysis are as follows:

[0045]

[0046] In the formula, is the cross-entropy loss between the predicted token and the target token, is the input, is the output result obtained by the training risk diagnosis model based on the input, is the label in the dataset, that is, the corresponding accident consequence; is the corresponding analysis generated by the teacher model.

[0047] Obtain the risk information data to be diagnosed in the factory, input the risk information data into the risk diagnosis model, use the risk diagnosis model to infer the accident result, and generate an analysis report.

[0048] Specifically, based on information such as risk points, evaluation objects, and hazardous and harmful factors input by the user, use the risk diagnosis model to make inferences, judge the possible accident consequences, and give corresponding analyses.

[0049] In this embodiment, risk information data to be diagnosed is collected from various departments of the factory (such as production, safety, environmental protection, etc.), including accident reports, safety inspection records, equipment maintenance records, etc. According to the input risk information data, the risk diagnosis model infers the accident cause, impact assessment, and countermeasure suggestions. By using a pre-trained large language model and the step-by-step distillation method, the model can learn rich language knowledge and reasoning ability, thereby improving the accuracy of accident cause analysis. Using the LoRA method for parameter-efficient fine-tuning reduces the computational resource requirements, enabling the model to be deployed in resource-constrained environments. Automatically generating accident analysis reports reduces the time and effort of manual analysis and improves work efficiency.

[0050] The model trained based on the above DS-LoRA method is used as the inference model of the entire diagnostic system, which can increase the final accuracy by 47.96% compared to before training, by 8.73% compared to the full-scale fine-tuning method, and is 12.25% higher than the accuracy of the teacher model.

[0051] Embodiment 2 As a preferred embodiment of Embodiment 1, this embodiment includes the following steps: Step 1: Clean and organize the production data of the enterprise in the existing risk double prevention and control system. In data cleaning, some data need to be discarded. For example, if the name of a risk point is "test1111", this kind of data may be used for system testing in the initial stage and needs to be excluded.

[0052] Step 2: Construct risk entities for the training samples, which specifically need to include at least 4 basic information such as risk points, evaluation objects, hazardous and harmful factors, and accident consequences. Among them, the accident consequences are composed of 20 accident types such as object strikes, vehicle injuries, fires, etc. proposed in the national standard GB / T 6441-1986 Classification of Enterprise Employee Casualty Accidents, and 9 consequences including personal injury, casualty diseases, property losses, work stoppages, violations of laws, impacts on business reputation, damage to the working environment, and environmental pollution. The specific generated entity categories are shown in Table 1: Table 1 Set of entity categories

[0053] Step 3: Sample the dataset and randomly divide it into a training set and a test set according to a ratio of 7:3. Use the test set and the constructed risk entities to take 50 data in the test set to test the accuracy of several accident consequences obtained by zero-shot prompting and few-shot prompting. When designing the prompt words, the categories in the entity need to be replaced at the corresponding positions in the template according to the category symbols. For example, fill the risk point into the {riskPointName} position. The specific prompt word design is as follows: The zero-shot prompt is: You are an expert in the risk field and can infer all possible accident consequences based on the hazardous and harmful factors. The result is given in JSON format. In {riskPointName}, after evaluating {evaluationObject}, there are the following hazardous and harmful factors: {harmful}, what possible accident consequences may occur? Options: {A total of 29 accident consequences}, output in the following JSON format: {{"labels": [The most likely several consequences, not exceeding {actual accident number}]}".

[0054] The few-shot prompt is: You are an expert in the risk field and can infer all possible accident consequences based on the hazardous and harmful factors. The result is given in JSON format.

[0055] There are a total of 29 accident consequences as follows: {List of 29 accident consequences}.

[0056] In a special inspection, after evaluating the equipment management, the following hazardous and harmful factors exist: ["Implementation not in place"], which may lead to what accident consequences? Give no more than 1 possible accident consequence. The possible accident consequence is: {{"labels": ["Violation of the law"]}} In the methanol production plant - integrated tank farm, after evaluating the methanol storage tanks, the following hazardous and harmful factors exist: ["Leakage at storage tanks, pipelines, valves and flange connections, in case of ignition source", "Failure to effectively eliminate static electricity accumulation in a timely manner", "Leakage at storage tanks, pipelines, valves and flange connections"], which may lead to what accident consequences? Give no more than 2 possible accident consequences. The possible accident consequences are: {{"labels": ["Poisoning and asphyxiation", "Fire"]}}.

[0057] In {riskPointName}, after evaluating {evaluationObject}, the following hazardous and harmful factors exist: {harmful}, which may lead to what accident consequences? Give no more than {actual accident quantity} possible accident consequences. The possible accident consequences are: Taking the following input as an example: In the crude benzene section of the chemical production workshop, after evaluating the installation of a steam cleaning valve on the rich and lean oil heat exchanger, the following hazardous and harmful factors exist: ['Unlicensed operation, wash oil leakage, using tools to close the inlet and outlet valves, installing valves'], which may lead to what accident consequences? Give no more than 6 possible accident consequences.

[0058] The actual accident consequence data and the result examples obtained from the above different prompt words are as follows: Actual results: [["Mechanical injury", "Vessel explosion", "Poisoning and asphyxiation", "Fire"]] Zero - sample prompt: [["Fire", "Scalding", "Poisoning and asphyxiation", "Environmental pollution", "Property loss", "Production suspension"]] Few - sample prompt: [["Mechanical injury", "Poisoning and asphyxiation", "Fire", "Other injuries", "Environmental pollution", "Occupational disease"]] The number of accident consequences obtained according to the above prompt words will be the same as the original label quantity, so choose to refer to the algorithm, and propose the accuracy calculation formula for this scenario as:

[0059] Among them, is the number of correct accident consequences predicted, is the total number of accident consequences in the prediction.

[0060] After calculation, in the above example results, the zero-shot prompt score is 0.5, and the few-shot prompt score is 0.75. 50 pieces of data in the test set are selected for testing, and the average accuracy of each model is shown in Table 2 as follows: Table 2 Comparison of accuracies of different prompts

[0061] From Figure 3 the data in Table 2, it can be seen that for most models, the accuracy of few-shot prompts will increase. Only the accuracy of qwen2.5-1.5b will decrease instead. It is speculated that because few-shot learning has more context, for models with smaller parameters that have not been trained, the ability to understand context is poor. Figure 3 In it, blue represents the accuracy corresponding to zero-shot; green represents the accuracy corresponding to few-shot. And the abscissa is each large language model shown in Table 2.

[0062] Step 4: As Figure 4 , Figure 5 shown, because the accuracy of few-shot prompts is higher, the few-shot prompt in the above prompt template is used as the final prompt template, and all the data in the test set is used for filling. Then, the large language model is used to generate all possibilities of accident consequences, and combined with entity information for judgment, and the most likely accident consequence is selected from them. The accuracy of generating all possibilities of accident consequences adopts the in Step 3 as the standard, and for the accuracy of selecting the most likely accident consequence, if the judgment is correct, it is 1, and if the judgment is wrong, it is 0. The average accuracy of each model for the test set is shown in Table 3 as follows: Table 3 Comparison of accuracies of different models

[0063] Select the two models with the highest accuracy, and use the idea of step-by-step distillation. Use CoT to generate the analysis of accident consequences and the reasons for the most likely accident consequences respectively. The specific CoT is as follows: In the Methanol Branch - Comprehensive Tank Farm, after evaluating the methanol storage tanks, the following hazardous and harmful factors exist: ["Leakage at storage tanks, pipelines, valves, and flange connections. Explain comprehensively within 200 words why it can cause ["poisoning and asphyxiation", "fire"]: "Methanol is a flammable and toxic liquid, and its vapor can form an explosive mixture with air. Leakage at storage tanks, pipelines, valves, and flange connections will cause methanol and its vapor to escape into the environment. When encountering ignition sources (such as open flames, hot surfaces, etc.), it is extremely easy to trigger a fire or even an explosion. In addition, the accumulation of static electricity that is not eliminated in time may also become a potential ignition source. Inhaling excessive methanol vapor will inhibit the human central nervous system, leading to poisoning and asphyxiation. Therefore, the existence of the above hazardous and harmful factors will directly cause poisoning and asphyxiation as well as fire accidents."

[0064] In {riskPointName}, after evaluating {evaluationObject}, the following hazardous and harmful factors exist: {harmful}. Explain comprehensively within 200 words why it can cause {accident}: Use the results generated to construct a new dataset and entity categories. The newly added categories are shown in Table 4:

[0065] In addition, this embodiment also selects qwen2.5 - 1.5b as the baseline model, uses the above dataset to train the small - parameter model and conducts ablation experiments according to different training methods. The hardware environment during training is NVIDIA H800 PCIe 80GB. The specific steps are as follows: First, use the full - scale fine - tuning training method to test the improvement effect of fine - tuning on the accuracy. The specific formula for the training parameters is:

[0066] Among them are all the parameters of the baseline model, and fine - tuning is to train to make the overall parameters more adaptable to the risk - hidden area. At this time has the same number of parameters as the model.

[0067] Use the LoRA fine - tuning training method to test the impact of LoRA on the accuracy and the impact on hardware resources. The specific formula for the training parameters is:

[0068] Among them, A and B are two low - rank matrices, where , and in the case of using LoRA fine - tuning, the parameters that need to be trained are , it can significantly reduce the number of training parameters, thus greatly reducing the occupation of hardware resources.

[0069] During specific training, the full-scale fine-tuning uses 38288MiB of video memory, while LoRA fine-tuning only requires 20118MiB. The occupation of hardware resources is reduced by 46.80%, and the accuracy rates are respectively improved to 0.8134 and 0.7652.

[0070] Figure 4 As shown, the blue rectangular graph represents the possible accident consequences, and the green rectangular graph represents the most likely accident consequences. The abscissas are the respective models shown in Table 3.

[0071] Using the full-scale - step-by-step distillation training method to test the impact of step-by-step distillation on the accuracy rate. Step-by-step distillation requires modifying the calculation method of the loss function during training and requires additional training of the student model for the analysis of the generated accident consequences. The specific calculation formula is:

[0072]

[0073]

[0074] Among them is the input, is the result generated according to the input, is the actual accident consequence, is the reason generated by the teacher model, is the cross-entropy loss between the predicted token and the target token, is the input, is the balance index, generally set to 0.5, is the loss value of the accident consequence label, is the loss value of the generated analysis, is the final loss value.

[0075] After training, the accuracy rates are respectively improved to 0.8113 and 0.753.

[0076] Finally, using the training method combining LoRA and step-by-step distillation, look at the improvement effect of the accuracy rate and the occupation of hardware resources. The final results and the comparison of the accuracy rates of each method are shown in Table 5, Figure 5 as shown: Table 5 Comparison of accuracy rates of different training methods

[0077] Through comparison, it can be seen that the method of using LoRA-stepwise distillation can effectively improve the performance of small-parameter models in the risk field, and the stepwise distillation method can train the analysis results of the model output at the same time. When using LoRA standard fine-tuning, hallucinations and repetitions may occur, and the output may be as follows: "Object strikes are the most likely accident consequences in the explosive workshop of the cutting cartridge production line. This is because the cutting cartridge is a small gunpowder product. During the production process, various metal materials and gunpowder are used to make the gunpowder shell, and the gunpowder in the cutting cartridge shell is detonated by gunpowder ignition. In this process, if gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder gunpowder...".

[0078] Under the same circumstances, the output of stepwise distillation is more in line with the requirements of this system: "In the process of pressing the ammunition liner in the 201 explosive workshop, the combined action of multiple dangerous and harmful factors increases the risk of object strikes. The specific reasons include: 1) Equipment failures or improper operations may cause tools and parts to fly out; 2) Poor production site management, such as unstable stacking, may cause items to fall; 3) Lack of necessary safety training for operators or inadequate protective measures may also cause misoperations leading to object strikes. In addition, insufficient anti-static facilities and poor lighting conditions will increase the probability of misoperations, thereby indirectly causing object strike accidents. These factors combined make object strikes the most likely accident consequence." Figure 5 As shown, the blue bar chart represents the accident consequence accuracy rate, and the green bar chart represents the most likely accident consequence accuracy rate.

[0079] This analysis result can be synchronously fed back to the user to help the user make a judgment. For the specific flowchart, refer to Figure 1 .

[0080] Based on the trained model, an intelligent diagnosis system for risk hazards is constructed. An AI-assisted generation module is added to the existing system, and the trained model is used as the subsequent inference model. When the user selects AI-assisted generation, corresponding accident consequences can be generated according to the basic information input by the user, and corresponding analyses can be given. After the user selects to use the generated accident consequences, the data will be entered into the database. Finally, by installing the training method described in the present invention at regular intervals using the new data, the performance of the model can be further improved. For the specific flowchart, refer to Figure 2 .

[0081] Example 3 This embodiment provides a risk and hazard diagnosis system based on a large language model, which is used to implement the risk and hazard diagnosis method based on the large language model in the above-mentioned Embodiment 1 and Embodiment 2, including: A data processing module, which confirms the prompt word structure and the teacher model according to the pre-trained large language model; A model fine-tuning module, which fine-tunes the student model by using the LoRA method and the step-by-step distillation method according to the confirmed prompt word structure and the teacher model to obtain a risk diagnosis model; A risk diagnosis module, which diagnoses the potential risks of the current factory based on the risk information data to be diagnosed input into the risk diagnosis model, infers the accident results, and generates an analysis report.

[0082] Embodiment 4 In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in a terminal device and is used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and, of course, the extended storage medium supported by the terminal device. It can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The computer-readable storage medium provides a storage space, and this storage space stores the operating system of the terminal. And, one or more instructions suitable for being loaded and executed by a processor are also stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0083] The computer-readable storage medium also includes data signals propagated in the baseband or as part of a carrier wave, which carry the readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or component. The program code contained on the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0084] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0085] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the risk hazard diagnosis method based on a large language model in the above embodiments; the one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps: Confirm the prompt structure and the teacher model according to the pre-trained large language model; According to the confirmed prompt structure and the teacher model, fine-tune the student model using the LoRA method and the step-by-step distillation method to obtain a risk diagnosis model; Obtain the risk information data to be diagnosed in the factory, input the risk information data into the risk diagnosis model, use the risk diagnosis model to infer the accident result, and generate an analysis report.

[0086] Embodiment 5 Please refer to Figure 6 , the terminal device is a computer device. The computer device 60 in this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, it implements the risk hazard diagnosis method based on a large language model in the embodiment. To avoid repetition, it will not be elaborated here one by one. Alternatively, when the computer program 63 is executed by the processor 61, it implements the functions of each model / unit in the computing system composed of Embodiment 1. To avoid repetition, it will not be elaborated here one by one.

[0087] The computer device 60 can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art can understand that Figure 6Only an example of the computer device 60 is provided, which does not constitute a limitation on the computer device 60. It may include more or fewer components than those shown in the figure, or combine certain components, or have different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0088] The so-called processor 61 may be a central processing unit (CPU), or other general-purpose processors, central processors, graphics processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, data processing logic units based on quantum computing, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0089] The memory 62 may be an internal storage unit of the computer device 60, such as the hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0090] Furthermore, the memory 62 may also include both the internal storage unit and the external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or will be output.

[0091] In each of the embodiments provided by the present application, any reference to a memory, a database, or other media may include at least one of non-volatile and volatile memories. The non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, and the like. The volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0092] The database involved in each of the embodiments provided by the present application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., without limitation. The processor involved in each of the embodiments provided by the present application may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without limitation.

Claims

1. A risk hazard diagnosis method based on a large language model, characterized in that: include: Confirm the prompt word structure and teacher model based on the pre-trained large language model; The teacher model is the first language model with the highest accuracy among the selected multiple language models; According to the confirmed prompt word structure and teacher model, the student model is fine-tuned using the LoRA method and stepwise distillation method to obtain a risk diagnosis model; the student model is the second largest language model; Obtain risk information data to be diagnosed in the factory, input the risk information data into a risk diagnosis model, use the risk diagnosis model to infer accident results, and generate an analysis report.

2. The risk hidden danger diagnosis method based on large language model according to claim 1 is characterized in that: Confirm the prompt word structure and teacher model based on the pre-trained large language model, including: Acquire and pre-process risk hidden danger related data according to the risk management and control system to form a data set; the risk hidden danger related data at least includes risk points, assessment objects, dangerous and harmful factors and accident consequences; Construct multiple blank large language models, and use the data set to train each of the large language models. Use the zero-sample prompt method and the few-sample prompt method to test the trained large language models, and select the prompt method with the highest accuracy as the final prompt word structure. During the training process, the accuracy of the accident results output by each language model is evaluated, and the large language model with the highest accuracy is selected as the teacher model.

3. The risk hidden danger diagnosis method based on large language model according to claim 2 is characterized in that: The large language model includes at least two of the qwen series models, the GLM series models, the Doubao series models, the GPT series models, Flamingo, and BlenderBot.

4. The risk hidden danger diagnosis method based on large language model according to claim 1 is characterized in that: According to the confirmed prompt word structure and teacher model, the student model is fine-tuned using the LoRA method and stepwise distillation method, including: The LoRA method is used to update the parameters of the student model. The formula for updating the parameters of the LoRA method is: In the formula, are the parameters for initializing the pre-trained model, is the parameter that needs to be updated, B and A are low-rank decomposition matrices, and the rank .

5. The risk hidden danger diagnosis method based on large language model according to claim 4 is characterized in that: According to the confirmed prompt word structure and teacher model, the student model is fine-tuned using the LoRA method and the step-by-step distillation method, specifically including: the step-by-step distillation method includes: prompting the teacher model to generate accident analysis results through the Few-shot CoT prompt template for the input risk information data, and using the generated accident analysis results as additional labels to train the risk diagnosis model; the teacher model is used to supervise the student model.

6. The risk hidden danger diagnosis method based on large language model according to claim 2 is characterized in that: The risk points, assessment objects and dangerous and harmful factors are embedded in a small sample prompt template, and the output format is limited to JSON. The prompt template includes sample input and sample output.

7. The risk hidden danger diagnosis method based on large language model according to claim 2 is characterized in that: The risk diagnosis model is trained by a comprehensive loss function, which is: In the formula, is the comprehensive loss function; is the balance parameter; is the cross entropy loss of accident consequences; Cross entropy loss generated for the analysis; is the cross entropy loss between the predicted token and the target token, is the input, It is the output result of the training risk diagnosis model based on the input. is the label in the data set, that is, the corresponding accident consequence; It is the corresponding analysis generated by the teacher model.

8. A risk hidden danger diagnosis system based on a large language model, used to implement the risk hidden danger diagnosis method based on a large language model according to any one of claims 1 to 7, characterized in that: include: The data processing module confirms the prompt word structure and the teacher model based on the pre-trained large language model; The model fine-tuning module uses the LoRA method and stepwise distillation method to fine-tune the student model according to the confirmed prompt word structure and teacher model to obtain the risk diagnosis model; The risk diagnosis module diagnoses the potential risks of the current factory, infers the accident results, and generates an analysis report based on the risk information data to be diagnosed input into the risk diagnosis model.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to execute the risk hazard diagnosis method based on a large language model as described in any one of claims 1 to 7.

10. A computing device, characterized in that: include: One or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the risk hazard diagnosis method based on a large language model as described in any one of claims 1 to 7.

Images