An industrial hidden danger investigation and decision-making method based on multi-agent cooperation

Abstract

The application discloses an industrial hidden danger investigation decision-making method and system based on multi-agent cooperation, and belongs to the technical field of industrial safety management. The method comprises the following steps: receiving a request of a user containing an industrial scene image and investigation requirements; determining a task target through a task planning step; analyzing the image and instructions by using a multi-modal large model to generate a structured hidden danger description; classifying the hidden danger by using a retrieval enhancement generation technology combined with a vectorized knowledge base; determining a risk level by using a quantitative risk assessment model; generating a targeted rectification suggestion; structuring and integrating the whole process data, and feeding back and optimizing the knowledge base through a data closed loop. The application realizes the automation, intelligentization and whole life cycle management of the industrial hidden danger investigation through multi-agent cooperation, significantly improves the investigation efficiency and accuracy, and has the ability of continuous self-optimization.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial safety management, and specifically relates to an industrial hidden danger investigation and decision-making method and system based on multi-agent collaboration. BACKGROUND

[0002] Industrial production is a key link of economic development, but safety hazards and sudden accidents in the production process pose a serious threat to personnel life and enterprise property. Therefore, efficiently and accurately investigating and preventing industrial safety hazards is the core task and long-term challenge in the field of industrial safety management.

[0003] Currently, there are mainly two methods for investigating industrial safety hazards. The first is the traditional method relying on manual inspection and simple sensor monitoring. This method has obvious drawbacks: first, it is inefficient, requiring a large investment in human resources and time, resulting in high inspection costs; second, the inspection results are heavily dependent on the personal experience and responsibility of the inspectors, and are highly subjective, prone to missed detection and misjudgment, and the recorded data formats are not uniform, making it difficult to conduct subsequent quantitative analysis and management; finally, for complex industrial scenes, manual inspection cannot achieve full coverage without dead angles.

[0004] The second is to use single-function artificial intelligence technology for auxiliary investigation. For example, some visual detection systems based on deep learning (such as the YOLO algorithm) can identify pre-trained specific targets such as equipment failure and personnel violations. However, such systems can only identify known categories of hazards, and are powerless against hazards that have not been trained, are atypical, or are more subtle, and lack the ability to classify hazards, assess risks, and provide rectification suggestions. In addition, some analysis tools based on large language models can process text data such as safety manuals and accident reports, but cannot process multi-modal information such as images and videos in industrial sites, and cannot meet the actual on-site investigation needs.

[0005] In summary, the existing technology lacks an intelligent and systematic solution that can integrate multi-modal information input, achieve full-process automation from hazard discovery, analysis, evaluation to rectification suggestion generation, and achieve self-optimization through data closed-loop. SUMMARY

[0006] The present application aims to address the shortcomings of the prior art by providing an industrial hidden danger investigation and decision-making method and system based on multi-agent collaboration to achieve automation, intelligence, and closed-loop management of the entire process of industrial hidden danger investigation.

[0007] Technical solution: The industrial hidden danger investigation and decision-making method based on multi-agent collaboration according to the present application comprises the following steps:

[0008] S1: obtaining an initial request input by a user, the initial request containing at least one industrial scene image and an investigation requirement;

[0009] S2: analyzing the investigation requirement to clarify the investigation task;

[0010] S3: comprehensively analyzing the industrial scene image and the investigation task by using a multi-modal large model to identify and generate at least one piece of structured hazard description information;

[0011] S4: based on a first knowledge base, performing semantic matching on the structured hazard description information to determine its belonging hazard category;

[0012] S5: performing risk level assessment on the classified hazard to determine whether it is a general, major or large accident hazard;

[0013] S6: based on a second knowledge base, generating specific rectification suggestions for the risk level and hazard category;

[0014] S7: integrating the hazard description information, hazard category, risk level and rectification suggestions to generate a structured investigation report.

[0015] Further improve the above technical solutions, at least one of S2 to S7 is executed by a software agent endowed with a specified role definition and a behavior instruction set, the instruction set guides the large model to execute the specified function through the prompt injection technology.

[0016] Further, S2 adopts a hybrid architecture combining a rule engine and a machine learning model for intent understanding, including: the rule engine matches structured and deterministic investigation requirements based on a pre-defined keyword library and regular expressions; the machine learning model processes unstructured and ambiguous investigation requirements based on a fine-tuned BERT model, and can perform entity completion and intent candidate sorting on requirements with missing information or ambiguity

[0017] Further, S2 further includes: when detecting that the user's investigation requirement is insufficient, calling a preset business problem template library to ask the user guiding questions to supplement and improve the investigation task.

[0018] Further, S3 includes: generating the structured hazard description information and target position by using Qwen2.5-VL model; and using Python code interpreter to analyze the structured hazard description information, decomposing multiple hazard descriptions into single hazard description for subsequent steps for cyclic processing.

[0019] Further, the S4 comprises: inputting the hidden danger description information as a query by using a retrieval enhancement generation technology, retrieving a related knowledge item from the first knowledge base, and generating the hidden danger category based on the item; wherein the first knowledge base is constructed by using a BGE-M3 vector model.

[0020] Further, the S5 comprises:

[0021] The occurrence possibility P, the influence degree I and the rectification difficulty D of the hidden danger are quantitatively scored by using a thinking chain prompt to guide a large model;

[0022] The comprehensive risk value R is calculated by using a weighted summation formula , wherein, , , are weights of the occurrence possibility, the influence degree and the rectification difficulty respectively;

[0023] According to a preset numerical interval in which the comprehensive risk value R is located, the risk level is determined.

[0024] Further, the S7 comprises: storing full-process data in a hidden danger basic information table, a classification evaluation table and a rectification tracking table respectively, and realizing association of the three tables through a unique hidden danger ID, and the tables are realized by using an open platform API to realize automatic writing and state updating of data.

[0025] Further, the method further comprises a step S8: after the user confirms the investigation report, extracting a hidden danger category, a risk evaluation logic and a rectification suggestion generated in this task, and using the hidden danger category, the risk evaluation logic and the rectification suggestion to update the first knowledge base and / or the second knowledge base.

[0026] The application also provides an industrial hidden danger investigation decision system for realizing the above-mentioned method, comprising:

[0027] A receiving module is configured to receive an initial request input by a user, the initial request comprising at least one industrial scene image and an investigation demand;

[0028] A task planning module is connected with the receiving module and is configured to use a hybrid architecture combining a rule engine and a machine learning model to understand the intention of the investigation demand to determine an investigation task;

[0029] A multi-modal analysis module is connected with the task planning module and is configured to use a multi-modal large model to comprehensively analyze the industrial scene image and the investigation task to identify and generate at least one piece of structured hidden danger description information;

[0030] The hazard classification module, connected to the multimodal analysis module, is configured to perform semantic matching on the structured hazard description information based on a first knowledge base to determine its hazard category.

[0031] The risk assessment module, connected to the hazard classification module, is configured to use a mind chain prompt-guided large model to assess the risk level of classified hazards in order to determine whether they are general, major, or relatively large accident hazards.

[0032] The rectification suggestion module, connected to the risk assessment module, is configured to generate specific rectification suggestions based on a second preset knowledge base, targeting the risk level and hazard category; and

[0033] The data integration module, connected to the hazard classification module, risk assessment module, and rectification suggestion module, is used to integrate the hazard description information, hazard category, risk level, and rectification suggestions to generate a structured investigation report.

[0034] Beneficial Effects: Compared with existing technologies, the advantages of this invention are as follows: By leveraging multi-agent collaboration and full-process automation, it connects the discovery, analysis, assessment, and reporting stages that originally required significant manual intervention, greatly shortening the investigation cycle and reducing labor costs. Utilizing advanced multimodal large-scale models for deep semantic analysis, it can identify subtle hidden dangers that are easily overlooked by traditional methods; combining Retrieval Enhanced Generation (RAG) technology with a professional knowledge base ensures the professionalism and accuracy of hazard classification, risk assessment, and rectification recommendations; through data integration and data closure steps, the data from each investigation task is structured and stored, and fed back to optimize the knowledge base and investigation rules, enabling the system to have self-learning and continuous evolution capabilities, becoming increasingly intelligent with use; through a quantitative, multi-dimensional risk assessment model, it provides managers with objective and scientific risk ratings, facilitating prioritization and resource allocation, making safety management decisions more evidence-based. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall system architecture provided in an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of the workflow of the risk assessment module in an embodiment of the present invention.

[0037] Figure 3 This is a schematic diagram of the workflow of the data administrator module in an embodiment of the present invention.

[0038] Figure 4 This is a schematic diagram illustrating the deduction process of a specific application scenario of the present invention. Detailed Implementation

[0039] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0040] Example 1: As Figure 1 As shown, this invention provides an industrial hazard investigation and decision-making system based on multi-agent collaboration. It is divided into multiple collaborative modules or agents. The specific flow of the system's execution method is as follows:

[0041] S1: The user uploads images containing industrial scenes and related requirements for hazard investigation. The system receives this input information as the starting data for the hazard investigation process.

[0042] S2: The task planning agent processes the user's input requirements and analyzes the user's investigation intent through the intent understanding module. If the intent understanding is insufficient, it uses the business problem template enhancement function to supplement relevant information and improve the understanding of the user's intent.

[0043] S3: The multimodal analyst uses large model technology to perform multimodal analysis on the uploaded images and user commands, generating structured hazard description information. Simultaneously, hazard target detection is performed to identify various hazard targets present in the images. After S3 completes, a loop is initiated, containing S4, S5, and S6.

[0044] S4: Hazard classifiers classify detected hazards by combining the industrial hazard classification knowledge base, clarifying the category to which the hazard belongs, so as to carry out targeted handling in the future.

[0045] S5: The risk level classifier conducts a risk assessment on the classified hazards based on the received hazard description and hazard classification information, and divides them into different risk levels, namely general accident hazards, major accident hazards, and relatively large accident hazards.

[0046] S6: The rectification suggestion provider provides specific and feasible rectification measures and suggestions based on the hazard description information from the multimodal analyst, targeting hazards of different risk levels, to guide users in eliminating hazards.

[0047] S7: The data engineer receives all the data output from the loop, integrates and cleans it to form a multidimensional data table for data storage, analysis and display, providing data support for system optimization and subsequent hidden danger investigation.

[0048] S8: The inquiry staff will provide the preliminary investigation results and rectification suggestions to the user, and the user will confirm the hazard items. After the user confirms the hazard items, the inquiry staff will provide the final investigation results, rectification suggestions and other information to the data engineer for data update and storage. The hazard investigation task is completed, and the relevant knowledge base will be updated based on this task to optimize the rules for investigating similar hazards.

[0049] Specifically, in step S2, the task planning intelligent agent includes:

[0050] S21: Adopt prompt injection technology:

[0051] {

[0052] Role definition injection: "You are a professional hidden danger analysis system expert, responsible for understanding the user's hidden danger investigation needs, what is the specific goal of the instruction?";

[0053] Example injection: Provide typical investigation instructions and sample pairs of corresponding intentions.

[0054] }

[0055] S22: Use a hybrid architecture of "rule engine + machine learning model" for intent understanding of user input, taking into account accuracy (rules) and flexibility (models).

[0056] Use the rule engine scheme to process structured and highly deterministic user intentions. The rule engine is based on pre-defined business rules (keyword library + regular expressions) to quickly match explicit intentions. For example:

[0057] Keyword library: "investigate, check, hidden danger, problem" as core trigger words; "electrical, fire, mechanical" as hidden danger type keywords; "Workshop A, Warehouse No. 3" as location keywords. Regular expression: used to match structured instructions such as "investigate [location] [type] hidden danger" (such as "investigate the fire hidden danger in Workshop B"), directly extracting location and type entities.

[0058] Use machine learning models to handle unstructured and ambiguous intentions. For complex natural language (such as colloquial expressions, omitted sentences, ambiguous sentences), use a fine-tuned BERT model for processing. In this scheme, a "intent classification model" and an "entity recognition model" based on BERT fine-tuning are used. For example, the user input "recently there is a strange smell in the warehouse, want to see if there is a problem somewhere", the model can identify the core intention as "hidden danger investigation", and the entity as "warehouse (location), odor related (potential hidden danger type)".

[0059] When the user input is ambiguous (such as "check here") or information is missing (such as "check the hidden danger in the workshop" without specifying the type), the module is processed through "entity completion + intent candidate sorting". First, known entities are extracted by the model (such as "workshop" -> location), and then missing items are identified based on the pre-set intent structure of the business, such as the hidden danger checking intent = {action: checking, range: location, type: hidden danger type, standard: basis}. For ambiguous instructions, Top-N possible intents are generated, such as when the user says "the device has a problem", the candidates are "mechanical equipment hidden danger checking" and "electrical equipment hidden danger checking", and the confidence is sorted based on the keyword matching degree.

[0060] S23: The subtask items that need to be checked for hidden dangers are supplemented through a pre-defined historical template knowledge base, as shown in Table 1.

[0061] Table 1: Predefined design and storage table for missing hidden danger entity type based on business scenarios

[0062]

[0063] Taking the user input "check the hidden danger in the workshop" as an example, the process is as follows:

[0064] (1) The incomplete intent output by the intent understanding module:

[0065] Extract known information: action = checking, range = workshop; missing information: hidden danger type, checking standard.

[0066] (2) Template engine matches missing items and templates:

[0067] The template engine reads the "missing entity type" (hidden danger type, standard) in the intent and matches the corresponding follow-up templates from the template library:

[0068] For "hidden danger type": call the template "Do you need to check electrical, fire, or mechanical hidden dangers?"

[0069] For "checking standard": call the template "Do you need to perform national standards or enterprise standards?"

[0070] (3) Generate natural language follow-up questions and interact:

[0071] The task planning agent asks the user: "Do you need to check electrical, fire, or mechanical hidden dangers in the workshop? Do you need to perform national standards or enterprise standards?"

[0072] (4) Integrate feedback and perfect the intent:

[0073] After the user replies "electrical hazards, according to national standards", the template engine fills in the information into the intent structure, forming a complete intent:

[0074] {action: investigation, scope: workshop, type: electrical hazards, standard: national standard}, and passes it to the task planning module.

[0075] In step S3, in the multi-modal analyst agent,

[0076] S31: Use prompt injection technology

[0077] {

[0078] Role definition injection: "You are a senior industrial operation hazard investigation master, proficient in national industrial detection standard legal knowledge, good at analyzing hazards in detail and professionally according to user-provided images, and framing the target position of each hazard, and returning in JSON format."

[0079] Example injection: Provide a typical hazard analysis entry.

[0080] }.

[0081] S32: Structured parameter extraction guides the big model to output formatted hazard entries through structured output design; adopts JSON response format constraints to ensure that the extraction results can be directly processed by the system.

[0082] S33: Accept user instructions as input, and use MLLM to analyze according to user instructions. MLLM uses Qwen2.5-VL. Qwen2.5-VL has high multi-modal fusion accuracy, deep industrial semantic understanding, and strong task analysis capability, and has the ability of target detection, which can return target frame according to text requirements. It is highly matched with the whole process requirements of "image input-hazard detection-task planning-agent cooperation-result generation" in the scene of the invention, and can significantly improve the automation level and decision accuracy of industrial hazard investigation, which is the ideal MLLM selection for this system.

[0083] S34: Use python code interpreter to structure the output of S32, and decompose multiple hazard description contents into single hazard description to form a loop body for subsequent automatic process.

[0084] Specifically, in step S4, in the hazard classifier agent,

[0085] S41: Use hazard classification standard knowledge base, which mainly stores hazard classification standards. These are supplemented by national standard documents (safety technology specifications, safety production specifications, testing processes, etc.), internal specifications collected by different enterprises during deployment services, and structured classification standard entries uploaded by knowledge base maintenance personnel.

[0086] The vector model of the hidden danger classification standard knowledge base mainly uses bge-m3 (multi-functional text embedding model). The vector model candidates are generally bge-m3 and bm25. In the AI inspection scene of industrial hidden danger investigation, bge-m3 has the ability of semantic modeling of high-dimensional dense vectors, and is more suitable for processing complex queries that require deep semantic understanding and cross-terminology association, while bm25 is more suitable for keyword-oriented shallow retrieval tasks. Therefore, selecting BGE-M3 as the vector model of the knowledge base can significantly improve the analysis accuracy and response ability of the system to the semantic requirements of the industrial scene.

[0087] S42: Use prompt injection technology:

[0088] {

[0089] Role definition injection: You are a senior industrial operation hidden danger classification master. You have fully mastered the knowledge base of the hidden danger classification subject, and can give the corresponding hidden danger classification according to the hidden danger description provided by the user.

[0090] Example injection: Provide typical hidden danger classification entries.

[0091] }

[0092] S43: Receive the output of the multi-modal analyst, and output the standard hidden danger classification entry according to the context information in S41 through MLLM.

[0093] In step S5, the overall process of the risk level divisionist is as shown in Figure 2

[0094] S51: The core of the historical risk knowledge base is "recording historical hidden danger cases, occurrence conditions, rectification schemes and consequences", and its initial data is mainly enterprise internal historical data, supplemented by external public cases. The acquisition method is to connect with the enterprise internal system equipment management system through API interface, and to synchronize structured historical data regularly. For unstructured data (such as paper hidden danger reports, handwritten rectification records), upload to the knowledge base after manual arrangement according to the format.

[0095] S52: The risk level divisionist receives the output of S3 and S4, combines the information of the historical risk knowledge base, and analyzes through MLLM to guide MLLM to output the risk score value containing occurrence possibility, impact degree and rectification difficulty three core dimensions in josn format. Use prompt injection technology:

[0096] {

[0097] ​Role Definition Injection: You are a risk level classifier. You have a comprehensive knowledge base about the conditions under which risks occur, the degree of impact, and the difficulty of rectification. Based on the {Hidden Danger Type} and {Hidden Danger Description} in the S3 / S4 input, refer to the top 5 records in the historical knowledge base, and give the risk score value, the occurrence probability P (probability value, 0-10 points), the impact degree I (consequence severity, 0-10 points), and the rectification difficulty D (resource demand, 0-10 points). Please output in JSON format strictly, ensure the fields are consistent with the example, and include the scoring basis and data reference.

[0098] CoT Thought Chain: Based on the input combined with historical data, first determine the basic score of 3-5 points (basic occurrence probability, basic impact, and basic technical rectification complexity). If the trigger condition exists, give additional points. Such as condition bonus (+2 points), causing a chain reaction bonus (+3 points), and resource delay bonus (+2 points)

[0099] Example Injection: JSON Structure and Field Description

[0100] }

[0101] S53: Use the analytic hierarchy process to assign weights to each evaluation index. Through expert scoring and historical data analysis, determine the impact of each index on the risk level. Establish a risk assessment matrix, divide the occurrence probability, impact degree, and rectification difficulty into three levels, respectively, as general accident hazards, major accident hazards, and larger accident hazards. Each level corresponds to a numerical interval. Calculate the comprehensive risk value of the hidden danger by weighted summation. The calculation formula is:

[0102] (1)

[0103] Where R is the comprehensive risk value; 、 、 are the weights of occurrence probability, impact degree, and detectability, respectively. The weights come from prior knowledge obtained using the analytic hierarchy process, and are analyzed in this system based on current historical records 、 、 are 25.34%, 42.35%, and 32.31%, respectively; P, I, and D are the score values of the corresponding dimensions output in S51.

[0104] S54: According to the calculated comprehensive risk value, classify the hidden danger into different levels:

[0105] When , it is classified as a general accident hazard, indicating that the hidden danger will not cause serious consequences.

[0106] When When classified as a major accident hazard, it means that the hazard may lead to partial production interruption or minor personal injury;

[0107] when When a hazard is identified as a major accident hazard, it indicates that if the hazard occurs, it will cause significant casualties, property damage, or prolonged production stoppage.

[0108] in, The settings are generally adjusted according to the industrial site conditions. The default values ​​for α and β in this system are 2 and 6, respectively.

[0109] In step S6, rectification suggestions are provided to the engineers.

[0110] S61: Using hint injection techniques:

[0111] {

[0112] Role definition: You are a senior industrial site operation rectification advisor. You have a thorough understanding of rectification suggestions and can provide corresponding rectification recommendations based on the hazard descriptions provided by users.

[0113] Example injection: Provides typical rectification suggestion entries.

[0114] }

[0115] S62: Receives the output of S3, analyzes it through MLLM, combines it with the historical rectification suggestion knowledge base, and guides MLLM to output rectification suggestions in a formatted manner, including the personnel responsible for implementation and the implementation time.

[0116] In step S7, the data engineer,

[0117] S71: First, raw data is collected from each link of the system, including user-input images, hazard detection data output by multimodal analysts, classification results from hazard classifiers, assessment data from risk level classifiers, and rectification plans from rectification suggestion providers, etc., and then structured information is linked through a Python code interpreter.

[0118] S72: Then, after the loop ends, the data analyst performs data cleaning processing on the collected data:

[0119] Remove duplicate data to ensure data uniqueness;

[0120] To handle missing values, for a small number of missing data, knowledge base matching is used, and empirical values ​​are used to fill in the missing values.

[0121] S73: This solution selects Lark Multidimensional Tables for data management, storage, and retrieval. Based on the multidimensional data model of Lark Multidimensional Tables, core data tables are created, as shown in Tables 2, 3, and 4.

[0122] Table 2: Hidden danger basic information table

[0123]

[0124] Table 3: Classification evaluation table

[0125]

[0126] Table 4: Rectification tracking table

[0127]

[0128] Cross-table association: through the association field function, three tables are associated with the hidden danger ID as the key value, forming a complete hidden danger life cycle data chain.

[0129] S74: Use the multi-dimensional table API of the Feishu open platform to automatically and real-time write the data obtained by S72 into the table through the "add multiple records" module of the dify platform. For unstructured data (such as oral feedback of on-site inspection personnel), manually enter through the Feishu web page, and support attachment upload.

[0130] S75: The data master has table management permissions, can configure fields, create tables, and perform data insertion, deletion and query. When the hidden danger state changes, automatically notify the responsible department head through the Feishu message, realize the rectification closed-loop management. Display the hidden danger state in the form of a card (highlight with a red label for larger risk hidden dangers), support filtering by area and type. Through the built-in "formula field" and "filter condition" of the table, realize complex query logic, for example: filter "larger risk hidden dangers that have not been processed in the past 30 days"; statistics "the number of cases in the mechanical hidden danger type whose rectification time exceeds 72 hours".

[0131] The system of the application can automatically generate a hidden danger investigation report of a specific format compatible with different industrial scenarios. By combining a large model with a RAG knowledge base, the system can automatically and intelligently provide risk level classification and correct repair suggestions for the current hidden danger item.

[0132] An important feature of the application is its cross-industry adaptation capability, which requires "lightweight data fine-tuning + knowledge base expansion + rule parameter update". For example, if the system needs to be applied from the chemical industry to the pharmaceutical industry, the following steps are performed:

[0133] 1. Multimodal large model fine-tuning

[0134] Adopting the LoRA fine-tuning technology, about 1000-2000 pairs of "image-hidden danger text description" data specific to the pharmaceutical industry are collected to fine-tune the Qwen2.5-VL model, so that the model can quickly learn and adapt to the image features (such as clean room environment) and professional terms (such as "microbial over-standard") of the pharmaceutical industry.

[0135] 2. Knowledge base and rule extension: Extend the first and second knowledge bases, import the pharmaceutical industry-specific knowledge base, including hidden danger classification knowledge base, historical risk knowledge base, and rectification suggestion knowledge base. The knowledge base has general industry standards, and if there are specific implementation standards in new industries, such as "Good Manufacturing Practice for Drugs" and "Design Standard for Pharmaceutical Industry Clean Rooms", they need to be supplemented and extended.

[0136] Rule parameter update: Adjust the risk level calculation weights in the configuration file (such as the risk weight of "sterile environment destruction" in pharmaceuticals is higher than that of "general environment leakage" in chemical industry), and add specific judgment rules for pharmaceuticals (such as "microbial over-standard on the surface of equipment directly contacting products → directly judged as high risk").

[0137] As shown in one specific deduction scenario: Figure 4 The safety supervisor of a chemical enterprise uses human language to generate an investigation task.

[0138] Step 1: User demand input

[0139] The safety supervisor of a chemical enterprise uploads the panoramic photo of the storage tank area and the instruction: "Investigate whether there is a leakage risk and safety protection defect in the storage tank area". The system receives the image and demand, and starts the investigation process.

[0140] Step 2: Task planning and intent analysis

[0141] The hidden danger investigation task planning module uses the large model to analyze the instruction and clearly identifies the core intent as "leakage risk detection" and "protection facility inspection", and supplements the chemical storage tank area investigation standards (such as pressure monitoring equipment integrity, anticorrosion coating aging detection, etc.) through the historical template knowledge base to refine the task objectives.

[0142] Step 3: Multi-agent collaborative work

[0143] Multi-modal analyst: Call the industrial multi-modal large model to analyze the image and identify hidden dangers such as paint peeling on the surface of storage tank A and abnormal display of pressure monitoring instruments on storage tank B, generate structured descriptions (including location, type, image coordinates). The automated program parses multiple structured data and decomposes each hidden danger into an independent task to build an automated processing loop.

[0144] Hidden danger classifier: According to the RAG knowledge base, "paint peeling" is classified as "equipment protection defect", and "instrument abnormality" is classified as "monitoring system failure".

[0145] Risk level classifier: According to the historical risk knowledge base, combined with the duration, diffusion potential and other indicators, through risk registration analysis, the hidden danger of storage tank B is classified as a major accident hidden danger, and the hidden danger of storage tank A is classified as a general accident hidden danger.

[0146] Rectification suggestion provider: Through large model retrieval RAG knowledge base, generate targeted solutions:

[0147] Storage tank B (major hidden danger): "Immediately stop using the storage tank, contact professional team to repair the pressure instrument, and simultaneously evacuate the personnel in the surrounding area";

[0148] Storage tank A (general hidden danger): "Complete the anticorrosive coating repair within 72 hours, and conduct weekly inspection in the future".

[0149] Step 4: report generation and data closed loop

[0150] Data scientist: integrate hidden danger information, classification results, risk level and rectification suggestions into a report in the format of the chemical industry standard, and store it in the multi-dimensional table of Feishu.

[0151] Inquirer: push the report to the user, and after the user confirms, the system will archive the data, and update the RAG knowledge base based on this case, and optimize the rules for similar hidden danger investigation.

[0152] The application discloses an industrial hidden danger investigation and decision system based on multi-agent cooperation, which realizes the full-process automatic investigation of industrial scene hidden dangers through multi-agent cooperation and large model technology. The user uploads industrial images and investigation requirements, and the system analyzes the task intention through the intention understanding module, supplements the information, generates a task, and analyzes the image and instruction through a large model to generate a structured hidden danger description. The structured data is parsed through code to form a loop body. The hidden danger classifier, risk level classifier and rectification suggestion provider complete hidden danger classification, risk assessment (classified as general, larger and major accident hidden dangers) and rectification suggestion generation in the loop body. The data scientist integrates the output data of each agent in the loop body, cleanses and forms a multi-dimensional table for storage and analysis; the inquirer feeds back the result to the user for confirmation, and finally the data is closed loop stored to complete the investigation task.

[0153] The present application has the following advantages: (1) The multi-agent cooperative efficiency improvement troubleshooting system realizes the whole process automation of "image analysis-data analysis-classification evaluation-reform suggestion" through the division of labor of multi-modal analysts, hidden danger classifiers, data analysts and other roles, and avoids the efficiency loss caused by manual intervention; (2) The large model technology guarantees the troubleshooting accuracy, the multi-modal analyst performs semantic-level analysis on the industrial image based on the large model technology, can identify the subtle hidden dangers that are easily ignored by traditional manual work and the special hidden dangers that are not known by manual work, realizes the scientificity of hidden danger risk grading in combination with the historical hidden danger knowledge base, and improves the hidden danger identification accuracy of complex industrial scenes; (3) The data closed loop drives the system to be continuously optimized, the data analyst integrates the whole process data into a multi-dimensional table, supports functions such as historical hidden danger trend analysis and reform effect evaluation, and feeds back to the large model training and business rule optimization.

[0154] As described above, although the present application has been shown and described with reference to specific preferred embodiments, it is to be understood that the present application is not limited to the particular preferred embodiments. Various changes in form and detail can be made without departing from the spirit and scope of the present application as defined by the appended claims.

Claims

1. An industrial hidden trouble investigation and decision method based on multi-agent cooperation, characterized in that: The method comprises the following steps: S1: obtaining an initial request input by a user, the initial request comprising at least one industrial scene image and an investigation requirement; S2: analyzing the investigation requirement to determine an investigation task; S3: comprehensively analyzing the industrial scene image and the investigation task by using a multi-modal large model to identify and generate at least one piece of structured hidden danger description information; S4: based on a first knowledge base, performing semantic matching on the structured hidden danger description information to determine a hidden danger category to which the structured hidden danger description information belongs; S5: performing risk level evaluation on the classified hidden danger to determine a risk level; S6: based on a second knowledge base, generating rectification suggestions for the risk level and the hidden danger category; S7: integrating the hidden danger description information, the hidden danger category, the risk level and the rectification suggestions to generate a structured investigation report; At least one of S2 to S7 is executed by a software agent endowed with a specified role definition and a behavior instruction set, and the instruction set guides the large model to execute a specified function through a prompt injection technology; S5 comprises: adopting a thinking chain prompt to guide the large model to quantitatively score three dimensions of the occurrence possibility P, the influence degree I and the rectification difficulty D of the hidden danger; The comprehensive risk value R is calculated by a weighted sum formula The comprehensive risk value R is calculated by a weighted sum formula , , are the weights of the occurrence probability, the impact degree, and the rectification difficulty, respectively. determining the risk level according to a preset numerical interval in which the comprehensive risk value R is located.

2. The industrial hazard investigation and decision-making method based on multi-agent collaboration as described in claim 1, characterized in that, S2 adopts a hybrid architecture combining a rule engine and a machine learning model for intent understanding, including: the rule engine matches structured and deterministic investigation requirements based on a pre-defined keyword library and regular expressions; the machine learning model processes unstructured and ambiguous investigation requirements based on a fine-tuned BERT model, and can perform entity completion and intent candidate sorting on requirements with missing information or ambiguity.

3. The industrial hazard investigation and decision-making method based on multi-agent collaboration as described in claim 2, characterized in that, S2 further comprises: when it is detected that the user's investigation requirement is insufficient, a preset business problem template library is called to ask the user guiding questions to supplement and improve the investigation task.

4. The industrial hazard investigation and decision-making method based on multi-agent collaboration as described in claim 1, characterized in that, S3 comprises: generating the structured hidden danger description information and the target position by using a Qwen2.5-VL model; and parsing the structured hidden danger description information by using a Python code interpreter to decompose multiple hidden danger descriptions into a single hidden danger description for subsequent steps for cyclic processing.

5. The industrial hazard investigation and decision-making method based on multi-agent collaboration as described in claim 1, characterized in that, S4 comprises: adopting retrieval enhancement generation technology, inputting the hidden danger description information as a query, retrieving related knowledge items from the first knowledge base, and generating the hidden danger category based on the items; wherein the first knowledge base is constructed by using a BGE-M3 vector model.

6. The industrial hazard investigation and decision-making method based on multi-agent collaboration according to claim 1, characterized in that, S7 comprises: storing the full-process data in a hidden danger basic information table, a classification evaluation table and a rectification tracking table respectively, and realizing the association of the three tables through a unique hidden danger ID, and the tables are automatically written and updated through an open platform API.

7. The industrial hazard investigation and decision-making method based on multi-agent collaboration according to claim 1, characterized in that, The method further comprises a step S8: after the investigation report is confirmed by the user, the hidden danger category, risk evaluation logic and rectification suggestions generated in this task are extracted to update the first knowledge base and / or the second knowledge base.

8. An industrial hidden trouble checking and decision system for implementing the industrial hidden trouble checking and decision method based on multi-agent cooperation according to any one of claims 1-7, characterized in that, Comprise: The receiving module is configured to receive an initial request input by a user, the initial request comprising at least one industrial scene image and an investigation requirement; The task planning module is connected to the receiving module and is configured to perform intent understanding on the investigation requirement by using a hybrid architecture combining a rule engine and a machine learning model to determine an investigation task; The multi-modal analysis module is connected to the task planning module and is configured to perform comprehensive analysis on the industrial scene image and the investigation task by using a multi-modal large model to identify and generate at least one piece of structured hidden danger description information; The hidden danger classification module is connected to the multi-modal analysis module and is configured to perform semantic matching on the structured hidden danger description information based on a first knowledge base to determine a hidden danger category to which the hidden danger description information belongs; The risk assessment module is connected to the hidden danger classification module and is configured to perform risk level assessment on the classified hidden danger by using a thinking chain prompt guided large model to determine whether the hidden danger is a general, major or relatively major accident hidden danger; The rectification suggestion module is connected to the risk assessment module and is configured to generate specific rectification suggestions based on the risk level and the hidden danger category by using a second knowledge base; and The data integration module is connected to the hidden danger classification module, the risk assessment module and the rectification suggestion module and is configured to integrate the hidden danger description information, the hidden danger category, the risk level and the rectification suggestions to generate a structured investigation report.

Images