Safety production intelligent monitoring method and device

Abstract

The invention discloses a safety production intelligent monitoring method and device, and the method comprises the steps: employing a safety production large model, combining a video and operation type definition of a safety production operation process, and recognizing the operation type of safety production; obtaining a safety rule corresponding to the identified operation type, wherein the safety rule is obtained by using a safety production large model; obtaining environmental data by using a large safety production model and combining the operation type and the corresponding safety rules and environmental parameters; obtaining a risk behavior analysis result by using a multi-modal large model in combination with the operation type, the corresponding safety rule, the environment data and the video of the safety production operation process; and obtaining risk behavior suggestion measures by using a safety production large model and combining the risk behavior analysis result. According to the invention, the risk behavior can be identified efficiently and accurately, and suggested measures can be given.

Description

Technical Field

[0001] The present invention relates to the technical field of safety production data processing and monitoring, and particularly relates to a safety production intelligent monitoring method and device. Background Art

[0002] This section aims to provide background or context for the embodiments of the present invention stated in the claims. The descriptions herein are not admitted to be prior art merely because they are included in this section.

[0003] Special operations are indispensable in the production and operation process of chemical enterprises. Conducting special operations, especially hot work operations and operations within confined spaces, is one of the main causes of frequent accidents. The frequent accidents in special operation links are mainly due to reasons such as the ineffective implementation of the enterprise's special operation management system, unclear risk identification before operations, ineffective risk control during operations, and insufficient emergency response capabilities of guardians.

[0004] To strengthen the safety risk control in special operation links and prevent major and particularly serious production safety accidents, especially during hot work operations and operations within confined spaces, there are currently laws and regulations stipulating the safety requirements for special operations such as hot work operations, confined space operations, blind plate blinding and unblinding operations, high-altitude operations, lifting operations, temporary electricity use operations, earthwork operations, and roadblock operations in hazardous chemical enterprises. However, in current safety production, whether the requirements of laws and regulations are met mainly depends on manual judgment, which is prone to judgment errors and has low efficiency. Summary of the Invention

[0005] The embodiments of the present invention provide a safety production intelligent monitoring method, which can analyze the videos during the safety production process, efficiently and accurately identify risk behaviors, and give recommended measures, including:

[0006] Using a safety production large model, combined with the videos of the safety production operation process and the definition of operation types, to identify the operation types of safety production, where the safety production large model is obtained by training a machine learning model based on historical videos, historical safety rule files, historical environmental parameters, and historical recommended measures of the safety production operation process;

[0007] Obtaining the safety rules corresponding to the identified operation types, where the safety rules are obtained using the safety production large model;

[0008] Using the safety production large model, combined with the operation type and the corresponding safety rules and environmental parameters, to obtain environmental data;

[0009] Using a multimodal large model, combining the operation type, corresponding safety rules, environmental data, and videos of the safe production operation process, to obtain risk behavior analysis results. The multimodal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safe production operation process;

[0010] Using a safe production large model, combining the risk behavior analysis results, to obtain risk behavior recommended measures.

[0011] An embodiment of the present invention provides a safe production intelligent monitoring device, which can analyze videos during the safe production process, identify risk behaviors efficiently and accurately, and give recommended measures. The device includes:

[0012] An operation type recognition module, which is used to use a safe production large model, combine videos of the safe production operation process and operation type definitions, to identify the operation type of safe production. The safe production large model is obtained by training a machine learning model based on historical videos of the safe production operation process, historical safety rule files, historical environmental parameters, and historical recommended measures;

[0013] A safety rule acquisition module, which is used to obtain the safety rules corresponding to the identified operation type. The safety rules are obtained using a safe production large model;

[0014] An environmental data acquisition module, which is used to use a safe production large model, combine the operation type, corresponding safety rules, and environmental parameters, to obtain environmental data;

[0015] A risk behavior analysis result acquisition module, which is used to use a multimodal large model, combine the operation type, corresponding safety rules, environmental data, and videos of the safe production operation process, to obtain risk behavior analysis results. The multimodal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safe production operation process;

[0016] A recommended measure acquisition module, which is used to use a safe production large model, combine the risk behavior analysis results, to obtain risk behavior recommended measures.

[0017] An embodiment of the invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned safe production intelligent monitoring method is implemented.

[0018] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned safe production intelligent monitoring method is implemented.

[0019] An embodiment of the present invention also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned intelligent monitoring method for work safety is implemented.

[0020] In an embodiment of the present invention, a large model for work safety is used, combined with videos of the work safety operation process and job type definitions, to identify the job types of work safety. The large model for work safety is obtained by training a machine learning model based on historical videos of the work safety operation process, historical safety rule files, historical environmental parameters, and historical recommended measures; obtaining the safety rules corresponding to the identified job types, where the safety rules are obtained using the large model for work safety; using the large model for work safety, combined with the job types and corresponding safety rules and environmental parameters, to obtain environmental data; using a multi-modal large model, combined with the job types and corresponding safety rules, environmental data, and videos of the work safety operation process, to obtain risk behavior analysis results, where the multi-modal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the work safety operation process; using the large model for work safety, combined with the risk behavior analysis results, to obtain risk behavior recommended measures. Compared with the prior art steps of manually judging the risks in the work safety process, the method proposed in the embodiment of the present invention can automatically and efficiently identify job types and safety rules, and then accurately analyze the risk behavior analysis results and give risk behavior recommended measures. The whole process does not require manual judgment, has higher efficiency, and provides a good user experience. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0022] Figure 1 is a flowchart of the intelligent monitoring method for work safety in an embodiment of the present invention;

[0023] Figure 2 is a flowchart of identifying the job types of work safety in an embodiment of the present invention;

[0024] Figure 3 is a flowchart of safety rule construction in an embodiment of the present invention;

[0025] Figure 4 is an example of safety rule representation in an embodiment of the present invention;

[0026] Figure 5Flowchart for generating security rule updates in the embodiments of the present invention;

[0027] Figure 6 Flowchart for obtaining environmental data in the embodiments of the present invention;

[0028] Figure 7 Flowchart for obtaining the risk behavior analysis result in the embodiments of the present invention;

[0029] Figure 8 Flowchart for obtaining recommended measures for risk behaviors in the embodiments of the present invention;

[0030] Figure 9 Example of the preset recommended measure output structure in the embodiments of the present invention;

[0031] Figure 10 Flowchart for generating a work safety report in the embodiments of the present invention;

[0032] Figure 11 Schematic diagram of the work safety intelligent monitoring device in the embodiments of the present invention;

[0033] Figure 12 Schematic diagram of the work safety intelligent monitoring system in the embodiments of the present invention;

[0034] Figure 13 Schematic diagram of the computer device in the embodiments of the present invention. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer and more explicit, the following further elaborates on the embodiments of the present invention with reference to the accompanying drawings. Herein, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but not to limit the present invention.

[0036] Figure 1 Flowchart of the work safety intelligent monitoring method in the embodiments of the present invention. The method is applied to the first node in an asynchronous distributed system, and the first node is any node in the asynchronous distributed system. The method includes:

[0037] Step 101: Use a work safety large model, in combination with the video of the work safety operation process and the operation type definition, to identify the operation type of work safety. The work safety large model is obtained by training a machine learning model based on the historical videos, historical safety rule files, historical environmental parameters, and historical recommended measures of the work safety operation process;

[0038] Step 102: Obtain the safety rules corresponding to the identified operation type. The safety rules are obtained using the work safety large model;

[0039] Step 103: Use the safety production large model, combine the operation type and the corresponding safety rules and environmental parameters to obtain environmental data;

[0040] Step 104: Use the multimodal large model, combine the operation type, the corresponding safety rules, environmental data, and the video of the safety production operation process to obtain the risk behavior analysis result. The multimodal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safety production operation process;

[0041] Step 105: Use the safety production large model, combine the risk behavior analysis result to obtain risk behavior recommended measures.

[0042] The following is a detailed introduction to each step.

[0043] First of all, it should be noted that in the embodiments of the present invention, the special chemical operation is taken as an example, but it can actually be applied to any industry involving safety production, such as the mining industry, the construction industry, the transportation industry, etc.

[0044] In step 101, use the safety production large model, combine the video of the safety production operation process and the operation type definition to identify the operation type of safety production. The safety production large model is obtained by training a machine learning model based on historical videos of the safety production operation process, historical safety rule files, historical environmental parameters, and historical recommended measures;

[0045] In the embodiments of the present invention, the safety production large model is obtained after training (fine-tuning) a machine learning model. It is an intelligent model facing the industrial safety production field, which realizes functions such as operation safety prediction and early warning, risk analysis, and multimodal intelligent question answering through the integration of large language models + industry knowledge + environmental information.

[0046] Figure 2 It is a flowchart for identifying the operation type of safety production in the embodiments of the present invention. In one embodiment, use the safety production large model, combine the video of the safety production operation process and the operation type definition to identify the operation type of safety production, including:

[0047] Step 201: Analyze the video of the safety production operation process. If a key event is detected, continuously capture multiple frames of images before, during, and after the occurrence of the key event;

[0048] When obtaining images from a surveillance camera, a key event detection algorithm is introduced. By analyzing information such as action features and device status changes in the video stream, key events such as device startup, personnel entering the operation area, and tool use are identified. When these key events are detected, a frame extraction operation is automatically triggered, and multiple frames before, during, and after the occurrence of the key event are continuously intercepted to ensure a complete record of the key operation process of the operation. For example, in a hot work operation, when the ignition action is detected, not only the image at the moment of ignition is intercepted, but also the images of the equipment preparation state before ignition and the flame situation after ignition are intercepted, providing more comprehensive information for the identification of the operation type.

[0049] Subsequently, according to factors such as the lighting conditions and picture complexity of the operation site, the resolution and compression ratio of the image are dynamically adjusted. An image quality assessment algorithm is used to analyze the video picture in real time. When the picture is dimly lit or has complex textures, the image resolution is increased to retain more details; when the picture content is relatively simple, the resolution is appropriately reduced and the compression ratio is optimized to reduce the data storage volume. At the same time, combined with edge computing technology, part of the image processing work is completed on the edge device close to the surveillance camera, reducing the data transmission pressure and improving the image acquisition efficiency.

[0050] Step 202: Determine the role played by the safety production large model and the corresponding operation type identification instructions according to the risk level and complexity of the safety production operation scenario.

[0051] A dynamic generation model of the role played and operation type identification instructions can be established, and according to the risk level of the operation scenario, the role played by the safety production large model and the operation type identification instructions are automatically adjusted. For example, when the safety production operation scenario is a confined space operation with a high risk level, the prompt description corresponding to the role played by the safety production large model is set as "You are a chemical high-risk operation type identification specialist", and the operation type identification instruction is "Please carefully check the key safety elements such as personal protective equipment, ventilation equipment, and gas detection instruments in the image according to the provided materials and operation type definitions, and comprehensively judge the operation type"; when the safety production operation scenario is an earthwork operation scenario with a low risk level, the prompt description corresponding to the role played is "You are an assistant for identifying basic operation types", and the operation type identification instruction is simplified to "Quickly judge the operation type based on the operation tools and operation methods in the image". Through this dynamic adjustment, the safety production large model better meets the identification needs of different operation scenarios. The above dynamic generation model can be obtained through training.

[0052] Step 203: Determine the operation type features that the safety production large model needs to focus on from the text information extracted from the image;

[0053] Using semantic analysis technology in natural language processing, guide the safety production large model to focus on the operation type features in the image related to the text information extracted from the image, such as "there is a large amount of combustible materials piled up on site", enhance the semantic richness of the prompt words, help the large model better understand the operation scenario, and improve the recognition accuracy.

[0054] Step 204, use the role-playing and operation type recognition instructions, operation type features, operation type definitions, and preset output operation type structure of the safety production large model as the prompt words of the safety production large model, and input them into the safety production large model to obtain the current operation type of safety production.

[0055] The preset operation types include live working, action operation, confined space operation, and working at height, etc., which are not restricted here. The prompt words of the safety production large model can be marked as Prompt Word A. The safety production large model is trained based on the role-playing, operation type recognition instructions, operation type features, and defined operation types of a large number of safety production operations. Among them, the preset output operation type structure can adopt the json format, which is not restricted here.

[0056] The embodiment of the present invention can adopt an interactive prompt word mechanism. When the confidence level of the recognition result output by the safety production large model is low, supplementary prompt words are automatically generated. For example, if the safety production large model is uncertain about the recognition of a certain operation type, the supplementary prompt words can be "Please further check the identification of the operation personnel in the image or the nameplate information on the equipment to assist in determining the operation type", and input the supplementary prompt words and the original image into the safety production large model again for secondary recognition. Through this interactive method, gradually guide the safety production large model to obtain more effective information and improve the recognition accuracy.

[0057] In the embodiment of the present invention, multiple safety production large models with different architectures and training data are introduced to construct a collaborative recognition network. Each safety production large model has different advantages. For example, some safety production large models are good at recognizing the equipment form in complex scenarios, and some safety production large models have a high accuracy in recognizing human actions. Input the operation process image and prompt words into multiple safety production large models at the same time. Each safety production large model independently conducts operation type recognition, and then through model fusion algorithms, such as methods based on confidence weighted voting, Bayesian fusion, etc., comprehensively integrate the output results of multiple safety production large models to obtain the final operation type recognition conclusion. This heterogeneous model collaboration method can give full play to the strengths of different safety production large models, reduce the limitations of a single safety production large model, and improve the overall recognition performance.

[0058] In the embodiments of the present invention, a dynamic evolution system for the safety production large model is established, and the safety production large model is automatically optimized according to the recognition result feedback in the actual operation scenario. The differences between the actual operation types recognized each time and the model output results are recorded. When it is found that the recognition error rate of a certain type of operation exceeds the threshold, relevant operation images, prompt words, and correct results are automatically collected to form a new training dataset, and incremental training is performed on the corresponding safety production large model. At the same time, the performance of all models is regularly evaluated, the models with poor performance are eliminated, and new safety production large models are introduced for supplementation to maintain the advancement and adaptability of the models.

[0059] In addition to the above-recognized operation types, the embodiments of the present invention also support users to specify operation types for subsequent safety rules corresponding to the operation types.

[0060] Step 102, obtain the safety rules corresponding to the recognized operation type, where the safety rules are obtained using the safety production large model;

[0061] Specifically, the safety rules are obtained by extracting all safety rule files. Each operation type corresponds to different safety rules. After the operation type is recognized, the corresponding safety rules can be directly obtained;

[0062] Figure 3 This is the flowchart for constructing safety rules in the embodiments of the present invention. In one embodiment, the method further includes:

[0063] Step 301, obtain all safety rule files;

[0064] Step 302, extract the text content from all safety rule files and use it as the safety rule text of the safety production large model;

[0065] In specific implementation, safety production rule files in the relevant industry can be automatically collected, whether in the form of electronic texts or other forms such as image texts. For non-electronic text files, advanced OCR technology and deep learning models are integrated to achieve high-precision text recognition, and the text in the image is accurately converted into editable text content. At the same time, the parsing of multiple file formats is supported. For example, Word and PDF files are processed using regular expression matching and semantic analysis techniques, Excel tables are parsed through data pattern recognition algorithms, and key information in PPT documents is extracted by visual element analysis.

[0066] Step 303, determine the role played by the safety production large model and the corresponding rule extraction instructions;

[0067] Step 304: input the role played and the corresponding rule extraction instruction, safety rule text, safety rule example, and preset output rule template as prompt words of the safety production big model into the safety production big model to obtain the safety rules;

[0068] The preset output rule template can be used as follows Figure 4 The structure shown, Figure 4 It is an example of the safety rule representation in the embodiment of the present invention, including the rule abbreviation, complete rule description, rule category (such as: personnel safety / equipment safety / environmental safety, etc.), severity (low / medium / high / emergency), applicable scenarios, limited to: general / personal protection / hot work / height work, required safety equipment or conditions, prohibited items or behaviors, environmental requirements, alarm content when risks are discovered, content that needs to be notified, measures that need to be taken, etc.

[0069] In a specific implementation example, the extracted safety rules can be manually intervened, including adding rules, deleting rules, modifying rules, matching job types, etc.; the safety rules after manual intervention form a reliable safe production rule library, and the environmental parameters include wind speed ≤5m / s, relative humidity ≤80%, etc.

[0070] Figure 5 This is a flowchart of security rule update generation in an embodiment of the present invention. In one embodiment, after obtaining the security rules, it also includes:

[0071] Step 501, analyzing the context in the text content, extracting key information, and constructing a context feature text;

[0072] When analyzing the context in the text content, you can first clean the text content to remove noise data such as special symbols and invalid blank characters. Use the word segmentation tool to segment the text content and mark the part of speech to prepare for subsequent analysis. Encode the text content after word segmentation to obtain the semantic vector representation of the text content. Use the semantic vector representation to analyze the context of the text content and extract key information, such as the safety field involved, related scene keywords (hot work, high-altitude work, etc.), and the degree words describing the safety regulations (serious, urgent, etc.), to construct the context feature text.

[0073] Step 502, analyzing the obtained security rules to obtain missing fields and / or ambiguous descriptions;

[0074] judge Figure 4 Whether the severity field is completely filled in, whether the requirements in the condition field are clear, etc.

[0075] Step 503, generating a supplementary prompt word based on the context feature text and the obtained security rules;

[0076] For example, if Figure 4 the severity field is missing in the text, according to the risk consequences described in the text content, supplementary prompt words are generated through the obtained safety rules.

[0077] Step 504: Input the supplementary prompt words, as well as the missing fields and / or ambiguous descriptions into the safety production large model to obtain updated safety rules.

[0078] In step 103, use the safety production large model, combine the operation type, the corresponding safety rules, and environmental parameters to obtain environmental data;

[0079] Figure 6 This is a flowchart for obtaining environmental data in an embodiment of the present invention. In one embodiment, the method further includes:

[0080] Step 601: Based on all safety rules, construct a safety rule knowledge graph, which is used to store the relationship between the operation type and the corresponding environmental parameters;

[0081] When constructing the safety rule knowledge graph, named entity recognition (NER) technology is required to label key entities, such as operation types and environmental parameters. For example, it is recognized that the environmental parameters required for hot work operations include wind speed ≤ 5m / s, relative humidity ≤ 80%, etc.

[0082] Using the safety production large model, combine the operation type, the corresponding safety rules, and environmental parameters to obtain environmental data, including:

[0083] Step 602: Determine the role played by the safety production large model;

[0084] For example, the role played by the safety production large model is an environmental perception assistant, and the corresponding prompt description can be "You are an environmental perception assistant".

[0085] Step 603: Based on the operation type, query the safety rule knowledge graph to obtain environmental parameters and use them as environmental parameters;

[0086] In a specific embodiment, based on the parsed operation type, relevant environmental parameters are retrieved from the safety rule knowledge graph. For example, when outdoor construction involves high-altitude operations, environmental parameters such as wind speed, temperature, and rainfall probability are retrieved.

[0087] In step 604, analyze the time characteristics and space characteristics of the operation type, and according to the safety rules corresponding to the operation type, obtain an environmental data acquisition instruction including the time dimension and the space dimension;

[0088] For example, for a 24-hour pipeline welding operation, the environmental data acquisition instruction can be described as "Please, according to the provided materials, call an external environmental perception tool through the mcp protocol to obtain the environmental data for each hour within the next 24 hours in the operation area."

[0089] Step 605: Use the role played by the work safety large model, the environmental data acquisition instruction, the environmental parameters, the environmental data examples, and the preset output environmental data template as prompts for the work safety large model, and then call an external environmental perception tool through the mcp protocol to obtain environmental data.

[0090] Among them, the environmental perception tool can be an existing tool, service, or model. The work safety large model calls the environmental perception tool through the mcp protocol to obtain the desired environmental data.

[0091] In step 104, a multi-modal large model is used, combining the operation type, the corresponding safety rules, environmental data, and videos of the work safety operation process to obtain a risk behavior analysis result. The multi-modal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the work safety operation process.

[0092] Figure 7 This is a flowchart for obtaining the risk behavior analysis result in an embodiment of the present invention. In one embodiment, a multi-modal large model is used, combining the operation type, the corresponding safety rules, environmental data, and videos of the work safety operation process to obtain a risk behavior analysis result, including:

[0093] Step 701: Determine the role played by the work safety large model.

[0094] For example, the prompt description of the role played by the work safety large model can be: "You are a professional safety consultant, focusing on safety issues in industrial scenarios."

[0095] Step 702: Extract the constraint conditions in the safety rules.

[0096] In specific implementation, semantic encoding can be performed on the safety rule text, and then the core intention of the semantic encoding can be identified through an intention classification model to extract the constraint conditions in the safety rules, such as must wear a safety belt, equipment grounding resistance ≤ 4Ω.

[0097] Step 703: Based on the risk level of the work safety operation scenario and the constraint conditions, determine the emotional intensity and the corresponding expression.

[0098] In specific implementation, an emotion mapping function can be designed to map the risk level to the emotional intensity, such as high risk corresponding to a serious / urgent tone, and a tone word library can be constructed, including expressions with different emotional intensities, such as must strictly comply, it is recommended to refer to.

[0099] Step 704: Determine a non - safety - risk analysis instruction based on the emotional intensity and the corresponding expression mode.

[0100] For example, the non - safety - risk analysis instruction can be "Please combine the operation type, safety rules, environmental data, and non - safety - risk examples to determine whether the operation behavior in the image complies with the specifications and output according to the preset output structure."

[0101] Step 705: According to the risk - point distribution map corresponding to the operation type, determine the frame extraction frequency for each risk point of the video of the safe - production operation process, and extract frames from the video of the safe - production operation process according to the frame extraction frequency for each risk point to obtain images.

[0102] The video of the safe - production operation process can be divided into time - series segments in chronological order. At the same time, sort out the standard operation steps of this operation type, clarify the possible risk points in each step, and establish a risk - point distribution map, corresponding each risk point to the time node of the operation process. For example, in the fire - welding operation process, the risk points corresponding to steps such as ignition and welding are fire, explosion, etc.

[0103] Based on time - series analysis, combined with the operation process and risk - point distribution, set the initial frame extraction frequency. Before the operation starts, according to the operation type and risk level, preset different frame extraction frequencies for different stages. For example, in fire - welding operations, in the non - critical operation stage, the frame extraction frequency can be set to 1 - 2 frames per minute; while in the critical operation stages such as ignition and welding, increase the frame extraction frequency to 1 - 2 frames per second to ensure that the key operation details are fully recorded.

[0104] In specific implementation, a target - detection algorithm can be used to perform real - time analysis on the monitoring - video screen. The target - detection algorithm continuously scans elements such as objects and behaviors in the screen, and extracts frames from the video of the safe - production operation process according to the frame extraction frequency for each risk point to obtain images.

[0105] Step 706: Use the role - playing of the safe - production large - model, the non - safety - risk analysis instruction, environmental data, non - safety - risk examples, and the preset output structure as prompt words for the multi - modal large - model, and input them into the safe - production large - model to obtain the risk - behavior analysis result.

[0106] The safe - production large - model is obtained by training a machine - learning model.

[0107] The results of risk behavior analysis include information such as whether key behaviors comply with safety specifications, whether the equipment status is normal, and whether there are potential risks in the scenario. The analysis results are presented in a visual form, such as generating a risk heat map, a key behavior detection report, etc., providing intuitive and accurate decision-making basis for safety management personnel, so as to take corresponding measures in time to ensure operation safety.

[0108] In step 105, using the large model for work safety, combined with the above-mentioned risk behavior analysis results, obtain risk behavior recommended measures;

[0109] When the large model for work safety is trained (fine-tuned), it can be trained based on historical recommended measures at the same time, so that the large model for work safety can intelligently analyze risk behavior recommended measures. Specifically, the risk behavior analysis results can be fused with multi-source data such as historical accident case data and industry best practice data to jointly form the content of the prompt. By analyzing the handling methods of similar risk behaviors in historical accident cases and the safety measures in industry best practices, enrich the connotation of the prompt, guide the large model for work safety to draw on past experience, output more feasible and effective recommended measures, and realize work safety analysis and recommendation generation based on multi-dimensional data.

[0110] Figure 8 This is a flowchart for obtaining risk behavior recommended measures in an embodiment of the present invention. In one embodiment, using the large model for work safety, combined with the above-mentioned risk behavior analysis results, obtain risk behavior recommended measures, including:

[0111] Step 801, determine the role played by the large model for work safety and the recommended measure instruction;

[0112] For example, the role played by the large model for work safety can be "You are a professional production safety inspection expert", and the recommended measure instruction can be "Please analyze the safety rules violated by the risk behavior analysis results, risk description, severity, and recommended measures".

[0113] Step 802, form a prompt for the large model for work safety with historical work safety accident case data, a preset recommended measure output structure, the role played by the large model for work safety, and the recommended measure instruction, and input it into the large model for work safety to obtain recommended measures.

[0114] Figure 9 This is an example of the preset recommended measure output structure in an embodiment of the present invention, including safety rule identification, risk description, severity, and recommended measures.

[0115] Figure 10 This is a flowchart for generating a work safety report in an embodiment of the present invention. In one embodiment, the method further includes:

[0116] Step 1001: Determine the role played by the safety production large model and the safety report generation instruction;

[0117] For example, the role played by the safety production large model can be "You are an expert in making safety production reports", and the safety report generation instruction can be "Please form an inspection report by combining the risk behavior analysis results and recommended measures".

[0118] Step 1002: Form a prompt for the safety production large model with the risk behavior analysis results, recommended measures, preset safety report output structure, role played by the safety production large model, and safety report generation instruction, and input it into the safety production large model to obtain a safety report.

[0119] When the safety production large model is being trained (fine-tuned), it can be trained based on historical safety reports at the same time, enabling the safety production large model to intelligently generate safety reports.

[0120] In specific implementation, the preset safety report output structure can adopt the markdown format, and finally the safety report can include an overview of the overall situation, analysis of major problems, risk levels, risk behavior analysis results and recommended measures, follow-up tracking plans, etc.

[0121] The embodiment of the present invention also proposes a safety production intelligent monitoring device, the principle of which is similar to the safety production intelligent monitoring method and will not be elaborated here.

[0122] Figure 11 It is a schematic diagram of the safety production intelligent monitoring device in the embodiment of the present invention, including:

[0123] An operation type recognition module 1101, which is used to use the safety production large model and combine the video of the safety production operation process and the operation type definition to recognize the operation type of safety production. The safety production large model is obtained by training a machine learning model based on historical videos of the safety production operation process, historical safety rule files, historical environmental parameters, and historical recommended measures;

[0124] A safety rule acquisition module 1102, which is used to acquire the safety rules corresponding to the recognized operation type. The safety rules are obtained by using the safety production large model;

[0125] An environmental data acquisition module 1103, which is used to use the safety production large model and combine the operation type, corresponding safety rules, and environmental parameters to obtain environmental data;

[0126] A risk behavior analysis result acquisition module 1104, configured to use a multi-modal large model, combine the operation type, corresponding safety rules, environmental data, and videos of the safe production operation process to obtain risk behavior analysis results, where the multi-modal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safe production operation process;

[0127] A recommended measure acquisition module 1105, configured to use a safe production large model, combine the risk behavior analysis results, and obtain risk behavior recommended measures.

[0128] In one embodiment, the operation type identification module 1101 is configured to:

[0129] Analyze the video of the safe production operation process. If a key event is detected, continuously capture multiple frames of images before, during, and after the occurrence of the key event;

[0130] Determine the role played by the safe production large model and the corresponding operation type identification instructions according to the risk level and complexity of the safe production operation scenario;

[0131] Determine the text information extracted from the images as the operation type features that the safe production large model needs to focus on;

[0132] Use the role played by the safe production large model, the operation type identification instructions, operation type features, operation type definitions, and preset output operation type structures as prompts for the safe production large model, and input them into the safe production large model to obtain the operation type of the current safe production.

[0133] In one embodiment, the device further includes a safety rule construction module, configured to:

[0134] Obtain all safety rule files;

[0135] Extract the text content from all safety rule files and use it as the safety rule text of the safe production large model;

[0136] Determine the role played by the safe production large model and the corresponding rule extraction instructions;

[0137] Use the role played and the corresponding rule extraction instructions, safety rule text, safety rule examples, and preset output rule templates as prompts for the safe production large model, and input them into the safe production large model to obtain safety rules.

[0138] In one embodiment, the safety rule construction module is further configured to:

[0139] After obtaining the safety rules, use a pre-trained model to analyze the context environment in the text content, extract key information, and construct context environment feature text;

[0140] Analyze the obtained security rules to obtain missing fields and / or ambiguous descriptions;

[0141] Generate supplementary prompt words based on the context environment features text and the obtained security rules;

[0142] Input the supplementary prompt words, as well as the missing fields and / or ambiguous descriptions, into the safety production large model to obtain updated security rules.

[0143] In one embodiment, the security rule construction module is further configured to:

[0144] Construct a security rule knowledge graph based on all security rules, where the security rule knowledge graph is used to store the relationship between job types and corresponding environment parameters;

[0145] The environment data acquisition module 1103 is configured to:

[0146] Determine the role played by the safety production large model;

[0147] Query the security rule knowledge graph based on the job type to obtain environment parameters and use them as environment parameters;

[0148] Analyze the time characteristics and space characteristics of the job type, and obtain an environment data acquisition instruction including time dimension and space dimension according to the security rules corresponding to the job type;

[0149] Use the role played by the safety production large model, the environment data acquisition instruction, the environment parameters, the environment data example, and the preset output environment data template as prompt words for the safety production large model, and input them into the safety production large model to obtain environment data.

[0150] In one embodiment, the risk behavior analysis result acquisition module 1104 is configured to:

[0151] Determine the role played by the safety production large model;

[0152] Extract the constraint conditions in the security rules;

[0153] Determine the emotional intensity and corresponding expression methods based on the risk level of the safety production operation scenario and the constraint conditions;

[0154] Determine a non-safety risk analysis instruction based on the emotional intensity and the corresponding expression method;

[0155] According to the risk point distribution map corresponding to the job type, determine the frame extraction frequency for each risk point of the video of the safety production operation process, and extract frames from the video of the safety production operation process according to the frame extraction frequency for each risk point to obtain images;

[0156] Use the role played by the safety production large model, the non-safety risk analysis instruction, environmental data, non-safety risk examples, and the preset output structure as prompts for the multi-modal large model, and input them into the safety production large model to obtain the risk behavior analysis results.

[0157] In one embodiment, the recommended measure obtaining module 1105 is configured to:

[0158] Determine the role played by the safety production large model and the recommended measure instruction;

[0159] Form the prompts of the safety production large model with the historical safety accident case data, the preset recommended measure output structure, the role played by the safety production large model, and the recommended measure instruction, and input them into the safety production large model to obtain the recommended measures.

[0160] In one embodiment, the device further includes a safety report generation module, which is configured to:

[0161] Determine the role played by the safety production large model and the safety report generation instruction;

[0162] Form the prompts of the safety production large model with the risk behavior analysis results and recommended measures, the preset safety report output structure, the role played by the safety production large model, and the safety report generation instruction, and input them into the safety production large model to obtain the safety report.

[0163] Figure 12 It is a schematic diagram of the safety production intelligent monitoring system in the embodiment of the present invention. The system is used to implement the safety production intelligent monitoring device. The safety production intelligent monitoring system includes an infrastructure layer, a data layer, a model layer, and an application layer.

[0164] The infrastructure layer provides access to monitoring cameras, video files, and safety production rule files for the system.

[0165] The safety rule database and the risk behavior analysis result database in the data layer are respectively used to store the safety rules extracted from the safety production rule files and the safety rules set manually, the operation record-related data, and the risk behavior analysis results.

[0166] The model layer provides the safety production large model, the multi-modal large model, and other models mentioned in the embodiment of the present invention.

[0167] The application layer includes a user interface, an environmental perception service, a job type recognition service, a rule extraction service, a safety detection service, and a video detection service.

[0168] The user interface is the entry for users to operate in the entire system.

[0169] The rule extraction service extracts safety rules from safety production rule files through various models and saves them to the database. The constructed safety rules support manual intervention.

[0170] The job management service is responsible for the management of jobs and related data.

[0171] The video processing service is responsible for pulling the video stream from the surveillance camera, saving the video at intervals, and obtaining the job frames from the video and handing them over to the job detection service for detection.

[0172] The job detection service detects the job frames through a multi-modal large model, and saves the detection results to the database.

[0173] In summary, in the method and device proposed in the embodiments of the present invention, a safety production large model is used, combined with the video of the safety production operation process and the job type definition, to identify the job type of safety production. The safety production large model is obtained by training a machine learning model based on the historical video, historical safety rule file, historical environmental parameters, and historical recommended measures of the safety production operation process; obtain the safety rules corresponding to the identified job type, and the safety rules are obtained using the safety production large model; use the safety production large model, combined with the job type and the corresponding safety rules and environmental parameters, to obtain environmental data; use a multi-modal large model, combined with the job type and the corresponding safety rules, environmental data, and the video of the safety production operation process, to obtain the risk behavior analysis result. The multi-modal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and the historical video of the safety production operation process; use the safety production large model, combined with the risk behavior analysis result, to obtain the risk behavior recommended measure. Compared with the prior art steps of monitoring risks in the safety production process through manual judgment, the method proposed in the embodiments of the present invention can automatically and efficiently identify job types and safety rules, and then accurately analyze the risk behavior analysis result and give the risk behavior recommended measure. The whole process does not require manual judgment, has higher efficiency, and good user experience. In addition, all the models proposed in the embodiments of the present invention can be continuously updated and iterated, so that the models can better understand safety rules and adapt to multi-modal information fusion: integrating text, images, videos, audio, and environmental data to achieve multi-dimensional three-dimensional perception and accurate judgment of complex job scenarios, which is significantly better than traditional image recognition models; relying on the powerful general understanding and reasoning ability of the large model, it can adapt to different enterprises, different job types, and complex scenarios, and is easy to migrate and expand; it can realize a closed-loop operation process from risk detection, alarm triggering, rule comparison to report generation, greatly improving management efficiency and response speed.

[0174] The embodiments of the present invention also provide a computer device, Figure 13Schematic diagram of a computer device in an embodiment of the present invention. The computer device 1300 includes a memory 1310, a processor 1320, and a computer program 1330 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1330, the data visualization method based on the data exchange model described above is implemented.

[0175] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the data visualization method based on the data exchange model described above is implemented.

[0176] An embodiment of the present invention further provides a computer program product including a computer program, and when the computer program is executed by a processor, the data visualization method based on the data exchange model described above is implemented.

[0177] Those skilled in the art should understand that the embodiments of the present invention may be provided as a method, a system, or a computer program product. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0178] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one or more of the processes Figure 1 or a plurality of processes and / or blocks

[0179] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the specified functions in Figure 1 one or more of the processes Figure 1 or a plurality of processes and / or blocks

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or boxes Figure 1 one process or a plurality of processes and / or boxes Figure 1 steps for implementing the functions specified in one box or a plurality of boxes

[0181] The specific embodiments described above further elaborate on the objectives, technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent monitoring method for safe production, characterized in that, Including: Using a large safety production model, combined with videos of the safety production operation process and job type definitions, to identify the job types of safety production. The large safety production model is obtained by training a machine learning model based on historical videos of the safety production operation process, historical safety rule files, historical environmental parameters, and historical recommended measures; Obtaining the safety rules corresponding to the identified job types, where the safety rules are obtained using the large safety production model; Using the large safety production model, combined with the job type and the corresponding safety rules and environmental parameters, to obtain environmental data; Using a multimodal large model, combined with the job type and the corresponding safety rules, environmental data, and videos of the safety production operation process, to obtain risk behavior analysis results. The multimodal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safety production operation process; Using the large safety production model, combined with the risk behavior analysis results, to obtain risk behavior recommended measures.

2. The method according to claim 1, characterized in that, Using a large safety production model, combined with videos of the safety production operation process and job type definitions, to identify the job types of safety production, including: Analyzing the video of the safety production operation process. If a key event is detected, continuously capture multiple frames of images before, during, and after the key event occurs; According to the risk level and complexity of the safety production operation scenario, determine the role played by the large safety production model and the corresponding job type recognition instructions; Determine the job type features that the large safety production model needs to focus on from the text information extracted from the images; Use the role played by the large safety production model, the job type recognition instructions, job type features, job type definitions, and the preset output job type structure as prompts for the large safety production model, and input them into the large safety production model to obtain the current job types of safety production.

3. The method according to claim 1, characterized in that, Also including: Obtaining all safety rule files; Extracting the text content from all safety rule files and using it as the safety rule text for the large safety production model; Determining the role played by the large safety production model and the corresponding rule extraction instructions; Use the role played and the corresponding rule extraction instructions, safety rule text, safety rule examples, and preset output rule templates as prompts for the large safety production model, and input them into the large safety production model to obtain safety rules.

4. The method according to claim 3, characterized in that, After obtaining the safety rules, it also includes: Analyzing the context environment in the text content, extracting key information, and constructing a context environment feature text; Analyzing the obtained safety rules to obtain missing fields and / or ambiguous descriptions; Generating supplementary prompts based on the context environment feature text and the obtained safety rules; Input the supplementary prompts, as well as the missing fields and / or ambiguous descriptions, into the large safety production model to obtain updated safety rules.

5. The method according to claim 1, wherein Also including: Constructing a safety rule knowledge graph based on all safety rules, where the safety rule knowledge graph is used to store the relationship between job types and corresponding environmental parameters; Using the large safety production model, combined with the job type and the corresponding safety rules and environmental parameters, to obtain environmental data, including: Determining the role played by the large safety production model; Query the safety rule knowledge graph based on the job type, obtain the environmental parameters, and use them as the environmental parameters; Analyze the time characteristics and spatial characteristics of the job type, and obtain the environmental data acquisition instruction including the time dimension and the spatial dimension according to the safety rules corresponding to the job type; Use the role played by the safety production large model, the environmental data acquisition instruction, the environmental parameters, the environmental data example, and the preset output environmental data template as the prompt words of the safety production large model, and call the external environmental perception tool through the mcp protocol to obtain the environmental data.

6. The method according to claim 1, characterized in that Use the multimodal large model, combine the job type and the corresponding safety rules, environmental data, and the video of the safety production operation process to obtain the risk behavior analysis results, including: Determine the role played by the safety production large model; Extract the constraint conditions in the safety rules; Based on the risk level of the safety production operation scenario and the constraint conditions, determine the emotional intensity and the corresponding expression; Based on the emotional intensity and the corresponding expression, determine the non-safety risk analysis instruction; According to the risk point distribution map corresponding to the job type, determine the frame extraction frequency of each risk point in the video of the safety production operation process, and extract frames from the video of the safety production operation process according to the frame extraction frequency of each risk point to obtain images; Use the role played by the safety production large model, the non-safety risk analysis instruction, the environmental data, the non-safety risk example, and the preset output structure as the prompt words of the multimodal large model, and input them into the safety production large model to obtain the risk behavior analysis results.

7. The method according to claim 1, wherein Use the safety production large model, combine the risk behavior analysis results, and obtain the risk behavior suggestion measures, including: Determine the role played by the safety production large model and the suggestion measure instruction; Form the prompt words of the safety production large model with the historical safety accident case data, the preset suggestion measure output structure, the role played by the safety production large model, and the suggestion measure instruction, and input them into the safety production large model to obtain the suggestion measures.

8. The method according to claim 1, wherein Also include: Determine the role played by the safety production large model and the safety report generation instruction; Form the prompt words of the safety production large model with the risk behavior analysis results, the suggestion measures, the preset safety report output structure, the role played by the safety production large model, and the safety report generation instruction, and input them into the safety production large model to obtain the safety report.

9. An intelligent safety production monitoring device, characterized in that, Include: The job type recognition module is used to use the safety production large model, combine the video of the safety production operation process and the job type definition to identify the job type of safety production. The safety production large model is obtained by training a machine learning model based on the historical video of the safety production operation process, the historical safety rule file, the historical environmental parameters, and the historical suggestion measures; The safety rule acquisition module is used to obtain the safety rules corresponding to the identified job type. The safety rules are obtained by using the safety production large model; The environmental data acquisition module is used to use the safety production large model, combine the job type and the corresponding safety rules and environmental parameters to obtain the environmental data; A risk behavior analysis result acquisition module, which is used to use a multimodal large model, combine the operation type, corresponding safety rules, environmental data, and videos of the safe production operation process to obtain risk behavior analysis results. The multimodal large model is obtained by training a machine learning model based on historical safety rules, historical environmental data, and historical videos of the safe production operation process; A recommended measure acquisition module, which is used to use a safe production large model, combine the risk behavior analysis results, and obtain risk behavior recommended measures.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 8.

Images