Park safety risk early warning and response plan generation method

The park's risk score is calculated through convolutional neural network, long-term memory network and attention mechanism, and a response plan is generated using a big model, which solves the problem that industrial parks are difficult to conduct advanced security risk warnings and flexible responses to emergencies, and realizes security risks dynamic warning and response.

CN119026905APending Publication Date: 2024-11-26HONGTU INTELLIGENT DIGITAL TECHNOLOGY CO LTD +1
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411077821.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

It is difficult for industrial parks to provide advanced security risk warnings from the trend of changing multimodal sensor data, and it is difficult to recommend the best response plan for responding to emergencies.

Method used

The convolutional neural network, long and short-term memory network, attention mechanism and fully connected network are used to calculate the sub-item risk score and comprehensive risk score of the park. The risk score is mapped into risk description through the Query constructor and input it into the big model. Dynamically generate response plans.

Benefits of technology

It has achieved pre-warning of safety risks in industrial parks and dynamic generation of response plans, breaking through the limitations of traditional systems that can only be detected after the event, and being able to flexibly respond to emergencies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026905A_ABST
    Figure CN119026905A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of safety control, and discloses a park safety risk early warning and response plan generation method, which uses a convolutional neural network, a long and short term memory network, an attention mechanism and a full-connection network to calculate and obtain a subentry risk score and a comprehensive risk score of a park. The problem that in the prior art, it is difficult to carry out advanced early warning on safety risks from the data change trend of a multi-mode sensor is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of security control technology, in particular to a method for generating a park security risk early warning and response plan. Background Art

[0002] Although many industrial enterprise parks have deployed relevant systems such as CMS, CRM, ERP, WMS, and installed a lot of security, fire protection, and environmental monitoring sensor equipment, most of these systems can only detect events after the fact and cannot truly avoid the loss of life and property caused by safety risks.

[0003] At the same time, big models are a new AI computing paradigm that has emerged in recent years. Typical natural language big models in the industry include GPT-4 (US, OpenAI), Wenxin Yiyan (China, Baidu), Tongyi Qianwen (China, Alibaba), etc. Visual big models have also been widely used in fields such as autonomous driving, intelligent security, and medical imaging. Text-to-image technology based on multimodal big models has also developed rapidly, such as Sora (US, OpenAI), etc. Content generation has become the next key area of ​​AI development. For professional decision-making scenarios, China Industrial Internet (China), Tianrang Intelligence (China), and Kezhi Technology (China) have applied big models to energy, construction, and operation and maintenance scenarios to achieve intelligent auxiliary decision-making. In risk prediction scenarios, the Digital Traffic Big Model (China, Zhuoshi Zhitong) and DriveGPT (China, Haomo Zhixing Technology) achieve accurate prediction of traffic risks and driving environments. However, in the field of safety risk management, relatively mature products have not yet appeared.

[0004] Through investigation, it was found that the current industrial park system or platform is difficult to provide advance warning of security risks from the changing trends of multimodal sensor data, and it is difficult to recommend the best response plan for dealing with current emergencies. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a method for generating a park safety risk warning and response plan, which solves the problems existing in the prior art such as difficulty in providing advance warning of safety risks from the changing trend of multimodal sensor data.

[0006] The technical solution adopted by the present invention to solve the above problems is:

[0007] A park safety risk early warning method uses convolutional neural networks, long short-term memory networks, attention mechanisms, and fully connected networks to calculate the park's sub-item risk scores and comprehensive risk scores.

[0008] As a preferred technical solution, the following steps are included:

[0009] A1, image feature extraction: extract the temporal features from the continuous video frames of the live camera and predict the temporal features;

[0010] A2, sensor data feature extraction: extract the historical time series features of field sensors and predict the time series features;

[0011] A3, multimodal attention processing: input the time series features predicted by step A1 and the time series features predicted by step A2 into the attention mechanism for processing;

[0012] A4, output module: output the sub-item risk score and comprehensive risk score of the park;

[0013] The execution order of A1 and A2 is one of the following: execute A1 first and then execute A2, execute A2 first and then execute A1, or execute A1 and A2 simultaneously.

[0014] As a preferred technical solution, in step A1, a convolutional neural network including a convolutional layer and a pooling layer is used. By inputting continuous video frames from a live camera into the convolutional neural network, local features and hierarchical features in the image are extracted through the convolutional layer and the pooling layer in turn, and then the features of the continuous video frames are expanded and input into a long short-term memory network to extract temporal features.

[0015] As a preferred technical solution, in step A2, the historical time series features of the field sensor are input into the long short-term memory network, and the long short-term memory network is used to predict the time series features.

[0016] As a preferred technical solution, in step A3, in a multimodal scenario, the multimodal temporal features predicted by the long short-term memory network are input into the attention mechanism. For visual data and text data, the calculation method of the multimodal attention features is as follows:

[0017] MAttention(V,T)=softmax((WV·V)·(WT·T)T / sqrt(d k ))·V

[0018] Among them, MAttention(V,T) represents the multimodal attention feature matrix, V represents the visual feature matrix, T represents the text feature matrix, WV represents the weight matrix for visual data, WT represents the weight matrix for text data, d k Indicates the dimension size, softmax(·) is the normalized exponential function, and sqrt(·) is the square root operation.

[0019] As a preferred technical solution, in step A4, the multimodal attention feature matrix output by the attention mechanism is input into the fully connected network, and the sub-item risk scores and the comprehensive risk scores are output. The calculation formula is as follows:

[0020]

[0021] Among them, x represents the output value of the previous layer network, and y represents the security risk score.

[0022] A method for generating a park safety risk response plan includes the park safety risk early warning method, mapping the park's comprehensive risk score and sub-item risk score into a risk description, constructing a query prompt word and inputting it into a large model, and using the large model to obtain a response plan.

[0023] As a preferred technical solution, the following steps are included:

[0024] B1, Construction of industrial knowledge base: Utilizing the document parsing capability of the big model, by inputting the relevant standards, specifications and documents of the industrial safety risk response plan, an industrial knowledge base of the park safety risk response plan is constructed;

[0025] B2, Query construction: construct query prompt words based on comprehensive risk score and sub-item risk score;

[0026] B3, Plan Generation: Input the query prompt words into the big model. The big model generates a safety risk response plan based on the industrial knowledge base by parsing and understanding the query prompt words.

[0027] As a preferred technical solution, step B2 includes the following steps:

[0028] B21, mapping the park’s comprehensive risk score and sub-item risk scores into risk descriptions;

[0029] B22, constructing a query prompt word according to the set query prompt word template: replacing the descriptions at corresponding positions of the query prompt word template with the actual comprehensive risk description and the sub-item risk description in turn to obtain a new query prompt word.

[0030] As a preferred technical solution, in step B3, the query prompt words are input into the big model through the API interface of the big model open source programming framework.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) The present invention utilizes a self-developed safety risk prediction method integrating multimodal attention and a dynamic generation method for response plans based on a query builder and an industrial large model to achieve the transformation of industrial park safety risk from post-detection to pre-warning, and the transformation of emergency response plans from static matching to dynamic generation;

[0033] (2) The present invention aims to solve the problem that early warning of park management safety risks is difficult and traditional response plans cannot flexibly deal with emergencies. The present invention uses convolutional neural networks (CNNs), long short-term memory networks (LSTMs), attention mechanisms, and fully connected networks to calculate sub-item risk scores and comprehensive risk scores. The scores are then assembled into query prompts through a query builder and input into a large model. The large model extracts relevant information from the industrial knowledge base by understanding the semantics of the query prompts and dynamically generates a response plan.

[0034] (3) The present invention aims to solve the problem of difficulty in early warning of safety risks in park management by fusing image features with other sensor features to provide early warning of risks;

[0035] (4) In order to address the problem that traditional response plans cannot flexibly deal with emergencies, the present invention designs a dynamic generation method for response plans based on a query builder and an industrial big model. By mapping the comprehensive risk score and the sub-item risk score into risk descriptions, query prompts are constructed, and the understanding and generation capabilities of the big model are used to obtain a response plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A method flow for generating a park safety risk warning and response plan according to an embodiment of the present invention;

[0037] Figure 2 A security risk prediction method integrating multimodal attention according to an embodiment of the present invention;

[0038] Figure 3 This is a flowchart of the operation of the AI ​​early warning system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0040] Example 1

[0041] like Figures 1 to 3 As shown, the present invention discloses a method for generating a safety risk warning and response plan for a park, which utilizes a self-developed safety risk prediction method integrating multimodal attention and a dynamic generation method for response plans based on a query constructor and an industrial big model to realize the transformation of safety risks in industrial parks from post-detection to pre-warning, and the transformation of emergency response plans from static matching to dynamic generation.

[0042] In order to reduce the safety risks in industrial parks, a method for generating a safety risk warning and response plan for industrial parks is designed, involving the following three core invention points:

[0043] 1. Aiming at the problem that early warning of safety risks in park management is difficult and traditional response plans cannot flexibly deal with emergencies, the present invention uses convolutional neural networks (CNN), long short-term memory networks (LSTM), attention mechanisms, and fully connected networks to calculate sub-item risk scores and comprehensive risk scores. Then, the query prompt words are assembled through the query constructor and input into the large model. The large model extracts relevant information from the industrial knowledge base by understanding the semantics of the query prompt words and dynamically generates a response plan. The above process is as follows: Figure 1 shown.

[0044] 2. Safety risk warning method integrating multimodal attention: To solve the problem of difficult safety risk warning in park management, image features are integrated with other sensor features to provide early warning of risks, such as Figure 2 shown.

[0045] (1) By inputting multiple frames of images from the on-site camera into the convolutional neural network, the complex features in the image are extracted through the convolution layer and pooling layer, and then the images are expanded and input into the LSTM network to extract the temporal features.

[0046] (2) The sensor data feature extraction module is used to extract the time series characteristics of field sensors such as combustible gas concentration, tank liquid level, temperature and humidity, and gas pressure.

[0047] (3) The image features are integrated with the multi-sensor features using the attention mechanism, and then the sub-risk scores of each type of sensor are output through a fully connected network. Finally, a comprehensive risk score is obtained through weighted calculation.

[0048] 3. Dynamic generation method of response plans based on industrial knowledge base and query builder: In view of the problem that traditional response plans cannot flexibly deal with emergencies, a dynamic generation method of response plans based on query builder and industrial big model is designed. By mapping the comprehensive risk score and sub-item risk score into risk description, query prompt words are constructed, and the understanding and generation capabilities of the big model are used to obtain the response plan.

[0049] Among them, the mapping relationship between risk score and risk description is shown in Table 1.

[0050] Table 1 Mapping relationship between risk score and risk description

[0051]

[0052] The risk scoring constructor replaces the current risk description with the Query prompt word template. The generated Query prompt words are used to ask questions to the locally deployed industrial large model. After vectorization, knowledge base retrieval and matching, the large model generates a response plan to deal with the current risk. The Query prompt word template, example and generated response plan are as follows:

[0053] (1) Query prompt word template: $Comprehensive risk description$ Enterprise, the following situations exist: $Sub-item risk description$, list some treatment suggestions point by point, each suggestion is controlled within 20 words.

[0054] (2) Query prompt word example: For enterprises with relatively high risks, the following situations exist: the liquid level in the storage tank is too high, the gas pressure is too high, and some treatment suggestions are listed point by point. Each suggestion should be within 20 words.

[0055] (3) Response plan example: 1. Immediately reduce the pressure to a safe range. 2. Adjust the feed to control the liquid level to normal. 3. Check the equipment and find out the cause of the fault. 4. Add a safety valve to prevent overpressure.

[0056] Example 2

[0057] like Figures 1 to 3 As shown, as a further optimization of Example 1, based on Example 1, this embodiment also includes the following technical features:

[0058] Therefore, in order to solve the problem that it is difficult to warn the safety risks in park management and the traditional response plan cannot flexibly deal with emergencies, a method for generating a park safety risk warning and response plan is proposed. Figure 1 As shown in the figure. The sub-item risk scores and comprehensive risk scores are calculated using convolutional neural networks, long short-term memory networks, attention mechanisms, and fully connected networks. Then, the query prompts are assembled into query words through the query builder and input into the big model. The big model dynamically generates response plans based on the pre-established industrial knowledge base by understanding the semantics of the query prompt words.

[0059] 1. Security risk prediction method integrating multimodal attention: including image feature extraction module, sensor data feature extraction module, multimodal attention module and output module, such as Figure 2 As shown. The functions of the four modules are as follows:

[0060] (1) Image feature extraction module: The live camera image is input into the convolutional neural network, and the complex features in the image are extracted through the convolution layer and pooling layer. After expansion, the image is input into the LSTM network to extract the temporal features.

[0061] (2) Sensor data feature extraction module: By inputting the historical time series features of field sensors such as combustible gas concentration, tank liquid level, temperature and humidity, and gas pressure into the multi-layer LSTM network, the multi-layer LSTM layer is used to predict the time series features while avoiding gradient explosion.

[0062] (3) Multimodal attention module: In multimodal scenarios, the attention mechanism needs to consider information from two or more modalities at the same time. For visual and textual data, the calculation method is as follows: MAttention(V,T) = softmax((WV·V)·(WT·T)T / sqrt(d k ))·V

[0063] Among them, MAttention(V,T) represents the multimodal attention feature matrix, V represents the visual feature matrix, T represents the text feature matrix, WV represents the weight matrix for visual data, WT represents the weight matrix for text data, d k Indicates the dimension size, softmax(·) is the normalized exponential function, and sqrt(·) is the square root operation.

[0064] (4) Output module: A fully connected network is used, and the ReLU activation function (i.e., y = max(0, x)) is usually used to help alleviate the gradient vanishing problem. However, in order to ensure that the output sub-item risk scores and comprehensive risk scores are between 0 and 100, the last two layers of the output module (sub-item risk score layer and comprehensive risk score layer) use the improved activation function sReLU, and the calculation formula is as follows:

[0065]

[0066] Among them, x represents the output value of the previous layer network, and y represents the security risk score.

[0067] 2. Dynamic generation method of response plan based on industrial knowledge base and query builder: By mapping the comprehensive risk score and sub-item risk score into risk description, query prompt words are constructed and input into the big model. The response plan is obtained by using the understanding and generation capabilities of the big model. The specific description is as follows:

[0068] (1) Industrial knowledge base: The national, local, and enterprise standard documents for park management are uploaded through the big language model open source programming framework. The powerful semantic understanding and reasoning capabilities of the big model are used to extract information from the documents, thereby building a private industrial knowledge base for the park’s safety risk response plan.

[0069] (2) Query constructor: 1) First, the comprehensive risk score and sub-item risk score are mapped to risk description according to Table 1.

[0070] 2) Construct Query prompts according to the set Query prompt template. Replace the comprehensive risk description and sub-item risk description with the following Query prompt template ($comprehensive risk description$ enterprise, the following conditions exist: $sub-item risk description$, list some treatment suggestions in points, each suggestion is controlled within 20 words.) The obtained Query prompt examples are as follows: Large risk enterprise, the following conditions exist: the tank liquid level is too high, the gas pressure is too high, list some treatment suggestions in points, each suggestion is controlled within 20 words.

[0071] (3) Plan generation: Through the API interface of the big model open source programming framework, the query prompt words are input into the offline deployed big model. Through the big model's understanding and analysis of the query prompt words, the information in the industrial knowledge base is retrieved to generate a safety risk warning response plan. The generated plan example is as follows: 1. Immediately reduce the pressure to a safe range. 2. Adjust the feed and control the liquid level to normal. 3. Check the equipment and find out the cause of the fault. 4. Add a safety valve to prevent overpressure.

[0072] In general, by deploying an AI early warning system in industrial parks, such as Figure 3 As shown. The multimodal sensor data is obtained through the data acquisition layer, and the safety risk prediction method integrating multimodal attention is used to warn the safety risks of the industrial park. Then, the Query constructor is used to convert the calculated risk real-time score into a warning risk description. Finally, the current warning risk description is input into the large model with the industrial knowledge base, and the corresponding response plan is output.

[0073] 1. According to business needs, configure the risk monitoring frequency in the scheduled task tool and regularly obtain multimodal data of the industrial park.

[0074] 2. Calculate the risk score through the risk warning model and insert the calculation result of each time into the database history table.

[0075] 3. If a security risk is predicted, a response plan will be dynamically generated through the large model.

[0076] 4. The system automatically recommends security risk warnings and response plans to the corresponding personnel in the park’s security management department.

[0077] As described above, the present invention can be preferably implemented.

[0078] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0079] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. A park safety risk early warning method, characterized in that: The sub-item risk scores and comprehensive risk scores of the park are calculated using convolutional neural networks, long short-term memory networks, attention mechanisms, and fully connected networks.

2. A park safety risk early warning method according to claim 1, characterized in that: The following steps are involved: A1, image feature extraction: extract the temporal features from the continuous video frames of the live camera and predict the temporal features; A2, sensor data feature extraction: extract the historical time series features of field sensors and predict the time series features; A3, multimodal attention processing: input the time series features predicted by step A1 and the time series features predicted by step A2 into the attention mechanism for processing; A4, output module: output the sub-item risk score and comprehensive risk score of the park; The execution order of A1 and A2 is one of the following: execute A1 first and then execute A2, execute A2 first and then execute A1, or execute A1 and A2 simultaneously.

3. A park safety risk early warning method according to claim 2, characterized in that: In step A1, a convolutional neural network including a convolutional layer and a pooling layer is used. By inputting continuous video frames from a live camera into the convolutional neural network, local features and hierarchical features in the image are extracted through the convolutional layer and the pooling layer in turn. Then, the features of the continuous video frames are expanded and input into a long short-term memory network to extract temporal features.

4. A park safety risk early warning method according to claim 3, characterized in that: In step A2, the historical time series features of the field sensors are input into the long short-term memory network, and the long short-term memory network is used to predict the time series features.

5. A park safety risk early warning method according to claim 4, characterized in that: In step A3, in a multimodal scenario, the multimodal temporal features predicted by the long short-term memory network are input into the attention mechanism. For visual data and text data, the calculation method of the multimodal attention features is as follows: MAttention(V,T)=softmax((WV·V)·(WT·T)T / sqrt(d k ))·V Among them, MAttention(V,T) represents the multimodal attention feature matrix, V represents the visual feature matrix, T represents the text feature matrix, WV represents the weight matrix for visual data, WT represents the weight matrix for text data, d k Indicates the dimension size, softmax(·) is the normalized exponential function, and sqrt(·) is the square root operation.

6. A park safety risk early warning method according to any one of claims 2 to 5, characterized in that: In step A4, the multimodal attention feature matrix output by the attention mechanism is input into the fully connected network to output the sub-item risk score and the comprehensive risk score. The calculation formula is as follows: Among them, x represents the output value of the previous layer network, and y represents the security risk score.

7. A method for generating a park safety risk response plan, characterized in that: It includes a park safety risk early warning method as described in any one of claims 1 to 6, mapping the park's comprehensive risk score and sub-item risk scores into risk descriptions, constructing query prompt words and inputting them into a large model, and using the large model to obtain a response plan.

8. A method for generating a park safety risk response plan according to claim 7, characterized in that: The following steps are involved: B1, Construction of industrial knowledge base: Using the document parsing capability of the big model itself, by inputting the standards, specifications and documents of the industrial safety risk response plan, an industrial knowledge base of the park safety risk response plan is constructed; B2, Query construction: construct query prompt words based on comprehensive risk score and sub-item risk score; B3, Plan Generation: Input the query prompt words into the big model. The big model generates a safety risk response plan based on the industrial knowledge base by parsing and understanding the query prompt words.

9. A method for generating a park safety risk response plan according to claim 8, characterized in that: Step B2 includes the following steps: B21, mapping the park’s comprehensive risk score and sub-item risk scores into risk descriptions; B22, constructing a query prompt word according to the set query prompt word template: replacing the descriptions at corresponding positions of the query prompt word template with the actual comprehensive risk description and the sub-item risk description in turn to obtain a new query prompt word.

10. A method for generating a park safety risk response plan according to claim 8 or 9, characterized in that: In step B3, the query prompt words are input into the big model through the API interface of the big model open source programming framework.

Citation Information

Cited By

  • Safety management method for comprehensive security and protection integrated management system

    CN120070135A

  • Multi-modal park data co-processing method and system based on dynamic weight fusion

    CN121167646A