Multi-modal fusion gas station leakage-fire early warning method and system

Abstract

The invention provides a multi-mode fusion gas station leakage-fire early warning method and system, and belongs to the field of gas system leakage and fire monitoring. The method comprises the following steps: firstly, acquiring infrared image data, pressure and gas concentration time sequence data of a gas station pipeline; secondly, the collected data are preprocessed; thirdly, an abnormal heat source image data set is manufactured, and a yov5 abnormal heat source monitoring model is constructed and trained and used for recognizing an abnormal heat source and calculating the fire symptom abnormal probability; and then, merging the fire symptom anomaly probability with a sensor data set, making a leakage-fire data set, and establishing a multi-mode gas station leakage-fire anomaly identification model for training and tuning. And finally, the trained multi-mode model is used for carrying out leakage-fire abnormity monitoring on the gas station, and corresponding early warning information is sent out according to an output result. According to the invention, gas leakage-fire joint abnormity discrimination can be realized, and the safety and early warning accuracy of the gas station are improved.

Description

Technical Field

[0001] The present invention belongs to the field of gas system leakage and fire monitoring, and relates to a multi-modal fusion gas station leakage-fire early warning method and system. Background Art

[0002] The safety of urban gas systems has always been a matter of great concern to all sectors of society. As the core facility for gas storage management and urban gas transportation, the importance of safety assurance of gas stations is self-evident. If gas leaks are not discovered and handled in time, they can easily cause fires or even explosions. Traditional gas leak monitoring methods mainly rely on single sensor data, such as gas concentration, pressure, flow, etc. These devices have limitations in sensitivity, reliability and accuracy, and it is difficult to meet the high safety requirements of modern urban gas systems. At the same time, fire warning systems also face similar challenges. In traditional fire monitoring, sensors are used to identify several fire parameters, such as smoke, fire scale, initial flame area and air temperature. Traditional technologies expose many problems. For example, it takes a while for smoke to rise to the ceiling, resulting in delayed fire alarms. Because traditional fire systems lack effective identification of fire signs, when a fire is caused by a gas leak, the fire will develop rapidly and cause an explosion. Therefore, the role of traditional fire warning systems is quite limited. In addition, leakage monitoring and fire warning technologies are mostly based on single-modal data processing and cannot fully utilize complementary information from different modalities (such as images, audio, temperature, gas concentration, etc.), limiting the accuracy and reliability of the warning system.

[0003] It will be a more effective choice to replace the gas leak / fire monitoring system with a single sensor with multimodal data fusion technology. The purpose of multimodal fusion is to combine multiple modal data in one model to make more robust decisions. By merging multiple modalities, more information for a given task can be obtained, so that higher and more stable classification accuracy can be obtained. Among them, the fusion of image data and sensor data is a common form. Deep learning technology shows great potential in multimodal fusion. By constructing a deep neural network model to automatically extract the deep features of the data, monitor whether the gas system is leaking and judge whether there is a fire environment and explosion risk based on visual clues, temperature changes and other features, and realize effective fusion and joint learning between different modal data, it is also the method adopted by the present invention when extracting the correlation of different modal information and data fusion.

[0004] At present, there are mostly the following problems in urban gas leakage and fire monitoring based on multimodal fusion technology: (1) Gas system leakage monitoring and regional fire identification are often performed as two independent tasks. However, in the context of gas leakage, once an abnormal heat source or even an open flame appears in the leakage environment, it may cause a fire or even an explosion. Therefore, it is necessary to identify the elements that may cause a fire in the gas station environment. However, there is currently a lack of technology that combines gas leakage monitoring with the identification of elements that may cause fire and explosion in the surrounding environment of the gas pipeline; (2) There is a lack of publicly available data sets for deep learning-based methods to be applied to the above two tasks at the same time; (3) The existing multimodal methods ignore the spatial correlation between monitoring data at different nodes during data processing, which leads to a high false alarm rate in the existing multimodal fusion methods during monitoring. Summary of the invention

[0005] The present invention mainly improves the shortcomings of the multimodal fusion technology for gas station leakage monitoring and fire warning, and proposes a multimodal fusion gas station leakage-fire warning method and system, which takes video images and multi-type sensor data as input, and uses deep learning to identify gas leaks and key elements of fires and explosions caused by gas leaks, thereby realizing gas leak monitoring and early prediction of fire and explosion accidents, and assisting management personnel to locate hidden dangers.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A multimodal fusion gas station leakage-fire warning method, the method first collects infrared image data, pressure and gas concentration time series data of the gas station pipeline. Secondly, the collected data is preprocessed, including normalization, Mosaic data enhancement and adaptive scaling. Thirdly, an abnormal heat source image data set is produced, and a yolov5 abnormal heat source monitoring model is constructed and trained to identify abnormal heat sources and calculate the abnormal probability of fire signs. Then, the abnormal probability of fire signs is merged with the sensor data set to produce a leakage-fire data set, and a multimodal gas station leakage-fire anomaly recognition model is established for training and tuning. Finally, the trained multimodal model is used to monitor the gas station for leakage-fire anomalies, and corresponding warning information is issued according to the output results. The following steps are included:

[0008] Step S1: Collect multimodal data for gas station pipeline leakage monitoring and regional fire warning. The multimodal data of the present invention includes infrared image data G collected by a dual-spectrum thermal imaging camera in each area of ​​the gas station, pressure time series data P collected by a pipeline pressure sensor, and gas concentration time series data C collected by a pipeline gas concentration sensor.

[0009] Step S2: normalize and preprocess the collected pressure time series data and gas concentration time series data to construct the sensor time series data set D, D = [P', C'], perform Mosaic data enhancement and adaptive scaling on the infrared image data, and use the preprocessed infrared image data to create a gas station heat source image data set F, F = [G']. Specifically:

[0010] Step S21: normalize the pressure time series data P and the gas concentration time series data C to form a sensor time series data set D after normalization, where D = [P', C'].

[0011] Step S22: For the infrared image data G of the gas pipeline area collected by the dual-spectrum thermal imaging camera, an infrared image of the area is taken at the same time interval as the sensor data set D. Mosaic data enhancement is performed on the infrared image data G, and then image adaptive scaling is performed on the infrared image data after data enhancement to output an N×N scaled image.

[0012] Step S23: Generate a gas station heat source image data set F from the pre-processed infrared image data, where F = [G'].

[0013] Step S3: Create an abnormal heat source image dataset, build a yolov5 abnormal heat source monitoring model, and perform training, fine-tuning, and testing. Take the gas station heat source image dataset F as input and calculate the abnormal probability P of fire signs in different areas of the gas station. x,t .

[0014] Specific:

[0015] Step S31: Use a dual-spectrum thermal imaging camera to shoot any scene containing flame burning, high-temperature iron blocks, and smoke generated by combustion. Use Labelme to mark the high-temperature objects in the abnormal heat source image with the number "1" to represent the "abnormal heat source" label, and form an abnormal heat source image dataset.

[0016] Step S32: construct a yolov5 abnormal heat source monitoring model, wherein the core structure of the yolov5 abnormal heat source monitoring model consists of a backbone network and a neck network, and a combined loss function is constructed using binary cross entropy and CIoU.

[0017] Step S33: Load the yolov5 abnormal heat source monitoring model with randomly initialized hyperparameters, divide the images in the abnormal heat source image data set into a training set and a test set in a ratio of 7:3, preprocess the image data of the training set and the test set according to the infrared image data processing method in step S22, and input the preprocessed image data into the yolov5 abnormal heat source monitoring model. The model is trained and tested multiple times by fine-tuning the hyperparameters of the yolov5 abnormal heat source monitoring model multiple times, and the model with the highest F1 score in the test results is selected as the final yolov5 abnormal heat source monitoring model for the calculation of the abnormal probability of fire signs. The F1 score refers to an indicator used to measure the accuracy of the classification model, and the value range is between 0 and 1. The closer to 1, the better the effect of the yolov5 abnormal heat source monitoring model in identifying abnormal heat sources.

[0018] Step S34: Load the yolov5 abnormal heat source monitoring model trained in step S33, and use the gas station heat source image dataset F as input. Use weighted non-maximum suppression operation to delete redundant overlapping detection frames for the same target, and retain the detection frame with the highest confidence. If multiple abnormal heat sources are identified in an image, the model will compare the confidence of each abnormal heat source in the image. The confidence refers to the predicted probability that the yolov5 abnormal heat source monitoring model will classify the identified object into the "abnormal heat source" category, and output the maximum confidence value of all abnormal heat sources identified in the current image, and use the maximum confidence as the abnormal probability P of fire signs in the xth area at time t. x,t , set the confidence threshold P T is 0.6. When the confidence of the abnormal heat source in the identification result is ≥ 0.6, then P x,t The confidence calculated by the model is retained and output. When the confidence of the identified abnormal heat source is less than 0.6, "no abnormal heat source" is output. Considering the accidental nature of the accident, when outputting "no abnormal heat source", the smaller accidental probability of the accident is used as P. x,t Output.

[0019] Step S4: Set the fire sign abnormal probability P x,t Merge it with the sensor dataset D to create the leakage-fire dataset Leakage-Fire, establish a multimodal gas station leakage-fire anomaly recognition model, and train and optimize the multimodal gas station leakage-fire anomaly recognition model; specifically:

[0020] Step S41: Select 10,000 time-synchronized continuous pressure time series data P t ', Gas concentration time series data F t ', Fire probability time series data P x,t Data, the time series data at the same time is regarded as s (t) =(P t ', Ct ',P x,t ) Create a Leakage-Fire dataset; based on the actual gas data, (t) Add labels, including "normal", "leakage" and "fire-explosion risk". Divide the leakage-fire dataset into training set s according to the ratio of 7:1:2 train , validation set val and test set s test , construct the adjacency matrix A of the pipe network topology in the form of a directed graph;

[0021] Step S42: construct a multimodal gas station leakage-fire anomaly recognition model, wherein the structure of the multimodal gas station leakage-fire anomaly recognition model includes four parts: sensor information embedding, graph structure learning, graph attention-based prediction, and graph deviation scoring.

[0022] Specific:

[0023] The sensor information embedding part calculates the embedding vector of sensor i and the candidate relationship j∈C i The normalized dot product e between ji , and then select the first k normalized dot products, using TopK to represent the index of the first k values ​​of its input. The graph structure learning part uses the graph structure in the adjacency matrix A to fuse the node information with its neighboring node information. Based on the prediction part of the graph attention, the attention coefficient α is calculated i,j , then each coefficient α i,j The corresponding time series embedding values ​​are multiplied element by element, and the resulting values ​​of all nodes in the adjacency matrix A are used as the input of the fully connected layer. In the graph deviation scoring part, the loss function is set to the mean square error between the predicted value and the true value, and then the individual anomaly score of each sensor is calculated and normalized. The normalized anomaly score a of each node is i (t) Screening is performed and the anomaly score of each sensor is aggregated to generate a smooth score A through a simple moving average method. i (t). If A i (t) exceeds the predetermined threshold A T,i , time t of the i-th node will be marked as abnormal.

[0024] Furthermore, the loss function of the graph deviation scoring part is the mean square error between the predicted value and the true value, and the predicted value refers to the multimodal gas station leakage-fire anomaly recognition model based on the data s in the input leakage-fire data set (t) Predicted data (t+1) ', the true value refers to the data s corresponding to the leak-fire dataset at time t+1 (t+1) .

[0025] Step S43: Normalize the training set, verification set, and test set data divided in step S41. The normalized training set is used to train the multimodal gas station leakage-fire anomaly recognition model, and then the verification set is input into the trained model. The A of each node calculated in the verification set model is selected. s,i (t) The maximum value is taken as the abnormal threshold A T,i Using the F1 score as the model performance evaluation indicator, the model is trained, verified and tested multiple times by adjusting the model hyperparameters. The trained multimodal gas station leakage-fire anomaly recognition model corresponding to the highest F1 score in all test results is selected for gas leakage-fire anomaly recognition and positioning in actual gas station scenarios.

[0026] Step S5: Use the trained multimodal gas station leakage-fire anomaly recognition model, and combine it with the yolov5 abnormal heat source monitoring model to perform leakage-fire anomaly recognition on the recognition results of abnormal heat sources in gas station scenes. According to the output results of the model, when the output result of the multimodal gas station leakage-fire anomaly recognition model is "normal", the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, and sends abnormal heat source warning information; when the output result of the multimodal gas station leakage-fire anomaly recognition model is "leakage", a leakage warning information is sent; the output result of the multimodal gas station leakage-fire anomaly recognition model is "there is a fire-explosion risk", and the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, and sends a fire and explosion warning information. Specifically:

[0027] Step S51: Set the model result output condition. When the i-th node has 10 consecutive time series data s i (t) is marked as abnormal, and the time series data s (t) There are more than 3 P x,t ≥0.6, the anomaly is output as “there is a risk of fire and explosion”; otherwise, the model anomaly recognition result is output as “leakage”.

[0028] Step S52: When the multimodal gas station leakage-fire anomaly recognition model recognizes that the result is "normal", but the yolov5 abnormal heat source monitoring model recognizes that there is an abnormal heat source in the area, the abnormal heat source warning sent includes the abnormal heat source location information and the infrared image data of the abnormal period of the area to which the abnormal heat source belongs.

[0029] Step S53: When the multimodal gas station leakage-fire anomaly recognition model identifies a “leakage”, the anomaly scores of the model output corresponding to the continuous anomaly period data are accumulated and summed to obtain A sum,i (t)'. Calculate the cumulative anomaly scores for all nodes in the adjacency matrix A and select A from the calculation results. sum,i(t)'The pipe sections connected to the largest and second largest abnormal nodes are marked as abnormal pipe sections, and the leakage warning information sent includes the leakage monitoring results output by the multi-modal gas station leakage-fire anomaly recognition model and A sum,i (t)'The largest and second largest abnormal node numbers.

[0030] Step S54: If the multi-modal gas station leakage-fire anomaly identification model identifies that the fire-explosion risk exists, the fire and explosion warning information sent includes the fire-explosion risk monitoring result output by the multi-modal gas station leakage-fire anomaly identification model, the A obtained in the same manner as step S53, and the sum,i (t)'The largest and second largest abnormal node numbers and the infrared image data of the area to which the abnormal pipe section belongs during the abnormal period output by the yolov5 abnormal heat source monitoring model.

[0031] The present application also provides a multi-modal fusion gas station leakage-fire warning system, and a multi-modal fusion gas station leakage-fire warning method, including the following modules:

[0032] Data acquisition and storage module: Collects multimodal data required for gas station leakage and fire warning. The multimodal data includes infrared image data of each pipeline area of ​​the gas station collected by a dual-spectrum thermal imaging camera, pressure time series data collected by a pipeline pressure sensor, and gas concentration time series data collected by a gas concentration sensor. The data sampling interval is 5 seconds / time, and the collected data is stored in the data acquisition and storage module.

[0033] Data preprocessing module: normalizes the collected pressure time series data and gas concentration time series data, performs mosaic data enhancement and image adaptive scaling preprocessing on the infrared images collected by the dual-spectrum thermal imaging camera, and outputs the sensor time series data set D, D = [P', C'] and the gas station heat source image data set F, F = [G'].

[0034] Abnormal heat source identification model training and evaluation module: load and initialize the yolov5 abnormal heat source monitoring model, use the abnormal heat source image data set to train and test the yolov5 abnormal heat source monitoring model, and store the model parameter file with the best test results as the "best_fd.pt" file in the current module.

[0035] Abnormal heat source identification module: load the optimal model parameters in the "best.pt" file in the fire probability model training module described in this embodiment into the yolov5 abnormal heat source monitoring model, take the gas station heat source image dataset F as input, and output the infrared image of the area where the abnormal heat source is located marked by the box and the abnormal probability P of fire signs in the xth area at time t x,t .

[0036] Multimodal model training and evaluation module: load and initialize the multimodal gas station leakage-fire anomaly recognition model, and convert s train The data and the adjacency matrix A constructed based on the pipeline network topology structure are used as input to train the multimodal gas station leakage-fire anomaly recognition model. val Calculate the anomaly threshold A in the trained multimodal gas station leak-fire anomaly recognition model T,i , using s test Data testing: The trained model with the determined anomaly threshold is trained, and the training, validation, and testing processes are repeated multiple times. The model with the highest F1 score in the test results is selected, and the model parameters are adjusted based on the anomaly threshold A. T,i Save it as "best_lfd.pt" file.

[0037] Gas leak-fire identification module: Load the optimal model parameters in the "best_lfd.pt" file into the multimodal gas station leak-fire anomaly identification model, and align the pressure time series data, gas concentration time series data, and fire probability time series data to form a time series data s (t) Input the multimodal gas station leak-fire anomaly recognition model. The model output results include: (1) "normal"; (2) when the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data. T,i , time series data (t) Medium x,t If the amount of data with a value greater than 0.6 is less than 3, the output result is “leakage”; (3) When the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data T,i And the time series data s (t) There are more than 3 P x,t ≥0.6, the output result is "there is a risk of fire and explosion".

[0038] Leakage-fire warning module: Send warning information according to the output results of the abnormal heat source identification module and the gas leakage-fire identification module described in this embodiment. When the monitoring result of the gas leakage-fire identification module is "normal", but the abnormal heat source identification module identifies the presence of an abnormal heat source in the area, this module sends abnormal heat source warning information and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel; when the monitoring result of the gas leakage-fire identification module is "leakage", but the abnormal heat source identification module does not identify the abnormal heat source, this module sends leakage warning information to the management personnel, and attaches the leakage pipe section number; when the monitoring result of the gas leakage-fire identification module is "there is a fire-explosion risk", this module sends fire-explosion warning information, leakage pipe section number and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention is different from the existing gas station abnormality monitoring method in which leakage monitoring and fire warning are treated as two independent tasks. The present invention innovatively monitors pipeline leakage while also paying close attention to fire sign factors in the environment, and closely integrates gas leakage monitoring and fire warning into one related task, aiming to significantly reduce the risk of further fire or explosion caused by sudden gas leakage accidents;

[0041] (2) Most fire warning methods used in the present invention rely on sensors to identify parameters such as smoke and temperature, which often leads to a high delay in fire alarms. In contrast, the present invention uses a method that combines computer vision technology with deep learning to quickly identify abnormal temperature areas in the environment and calculate the probability of abnormal fire signs in the area, thereby effectively assisting safety managers in determining potential fire hazards;

[0042] (3) The multimodal gas station leakage-fire anomaly recognition model designed by the present invention, compared with most anomaly monitoring models, integrates multimodal data such as fire sign anomaly probability, pressure time series data and gas concentration time series data as node features, and can learn more comprehensive pipeline network features, thereby achieving more accurate anomaly recognition, effectively advancing the intervention timing of firefighting operations, and reducing the losses caused by gas leaks and possible fires and explosions. In addition, the model also introduces the pipeline network topology as input, so that the model can capture the spatial correlation of input features between monitoring nodes, further improving the classification accuracy of the model. After identifying the anomaly, the model can immediately provide safety management personnel with abnormal location information, assist in quickly locating abnormal pipe sections, and provide strong support for safety management decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the process of the present invention.

[0044] Figure 2 It is a structural schematic diagram of the yolov5 abnormal heat source monitoring model designed by the present invention.

[0045] Figure 3 It is a structural schematic diagram of the multi-modal gas station leakage-fire anomaly recognition model designed by the present invention.

[0046] Figure 4 It is a structural schematic diagram of the multi-modal fusion gas station leakage-fire warning system designed by the present invention. Specific implementation methods

[0048] The technical solution of the present invention will be further described below in conjunction with specific embodiments and drawings.

[0049] Example 1 A multi-modal fusion gas station leakage-fire warning method, the steps are as follows:

[0050] Step S1: Collect multimodal data for gas station pipeline leakage monitoring and regional fire warning. The multimodal data of the present invention includes infrared image data G collected by a dual-spectrum thermal imaging camera in each area of ​​the gas station, pressure time series data P collected by a pipeline pressure sensor, and gas concentration time series data C collected by a pipeline gas concentration sensor.

[0051] Step S2: normalize and preprocess the collected pressure time series data and gas concentration time series data to construct the sensor time series data set D, D = [P', C'], perform Mosaic data enhancement and adaptive scaling on the infrared image data, and use the preprocessed infrared image data to create a gas station heat source image data set F, F = [G']. Specifically:

[0052] Step S21: normalize the pressure time series data P and the gas concentration time series data C to form a sensor time series data set D after processing, where D = [P', C']. The normalization calculation formula is:

[0053] P'=(P-min(P)) / (max(P)-min(P))

[0054] C'=(C-min(C)) / (max(C)-min(C))

[0055] Wherein, P' represents the normalized pressure time series data, C' represents the normalized gas concentration time series data, min(·) represents the minimum value operation, and max(·) represents the maximum value operation.

[0056] Step S22: For the infrared image data G of the gas pipeline area collected by the dual-spectrum thermal imaging camera, the regional infrared image is taken at the same time interval as the sensor data set D. First, the infrared image data G is enhanced by Mosaic data, and the four training images are randomly scaled, randomly cropped, and randomly arranged to form one image, so that the label data of the four images can be calculated at one time during the calculation process to improve the model training efficiency. Image adaptive scaling is adopted for the infrared image data after data enhancement, with 416×416 as the input size, the original size of the input image is 800×600, and the scaling coefficients a1=416 / 800=0.52 and a2=416 / 600=0.69 are calculated, and the smaller value of a1 and a2 is selected as the scaling coefficient. In this embodiment, a1=0.52 is used as the scaling coefficient, and the image size obtained by the first scaling is 416×(600×0.52)=416×312, and then the pixel value of the area to be filled at both ends of the height is calculated to be 0.5×(416-312)=52, that is, 52 black edges with a pixel value of 0 are filled at the top and bottom of the image with a size of 416×312 after the first scaling, respectively, and the image size after filling is 416×416, and a scaled image of 416×416 is output.

[0057] Step S23: Generate a gas station heat source image data set F from the pre-processed infrared image data, where F = [G'].

[0058] Step S3: Create an abnormal heat source image dataset and construct Figure 3 The yolov5 abnormal heat source monitoring model shown in the figure is trained, fine-tuned and tested, and the gas station heat source image dataset F is used as input to calculate the abnormal probability P of fire signs in different areas of the gas station. x,t Specifically:

[0059] Step S31: Use a dual-spectrum thermal imaging camera to shoot image materials including flame burning, high-temperature iron blocks, etc. in any scene, and use Labelme to mark the high-temperature objects in the abnormal heat source image with the number "1" to represent the "abnormal heat source" label, thereby forming an abnormal heat source image dataset.

[0060] Step S32: Construct the yolov5 abnormal heat source monitoring model. The core structure of the yolov5 abnormal heat source monitoring model consists of a backbone network and a neck network. The backbone network consists of a Focus layer, Conv, CSP1, SPP, and CSP2, and the neck network consists of four CSP2s. Among them, CSP1 consists of Conv, a residual unit layer, a convolution layer, a BN layer, and a LeakyRelu layer, CSP2 consists of Conv, a convolution layer, a BN layer, and a LeakyRelu layer, and Conv consists of a convolution layer, a BN layer, and a LeakyRelu layer.

[0061] The combined loss function is constructed using binary cross entropy and CIoU. The loss function calculation formula is:

[0062] Loss = λ1Loss CLS +λ2LOSS OBJ +λ3LOSS CIoU

[0063]

[0064]

[0065] Among them, λ i (i=1,2,3) represents the weights of different loss functions. In this example, they are set to λ1=0.2, λ2=0.4, λ3=0.4. N represents the total number of pixels in the image. i represents the true label of the i-th pixel (where 1 represents “abnormal heat source” and 0 represents “no target”), represents the probability that the i-th pixel predicted by the model belongs to the “abnormal heat source” category, M represents the total number of predicted bounding boxes, and C j Indicates whether the jth bounding box contains the target (where 1 is included if the target is included and 0 is not included if the target is not included). represents the predicted probability that the jth bounding box contains the target, log(·) is the logarithmic calculation operation, b and b gt Represents the predicted box B and the real box B respectively gt The center point of the 2 represents the square of the diagonal length of the minimum bounding box C, p 2 (·) represents the calculation operation of the squared Euclidean distance between two center points, w and h represent the width and height of the prediction box respectively, w gt and h gt They represent the width and height of the real box respectively, v represents the similarity of the aspect ratio, and α is a weight parameter that is automatically adjusted based on the IoU and v of each prediction result of the model.

[0066] Step S33: Load the yolov5 abnormal heat source monitoring model, divide it into a training set and a test set in a ratio of 7:3 after preprocessing, preprocess the image data of the training set and the test set according to the infrared image data processing method in step S22, and input the yolov5 abnormal heat source monitoring model after preprocessing. The model is trained and tested multiple times by fine-tuning the learning rate lr, batch size batch_size, training cycle epcoh and other hyperparameters of the yolov5 abnormal heat source monitoring model multiple times, and the model with the highest F1 score in the test results is selected for the calculation of the abnormal probability of fire signs. The F1 score calculation formula is:

[0067]

[0068] Among them, P represents the precision of the model, R represents the recall rate of the model, TP represents the number of pictures correctly predicted by the model as having abnormal heat sources, FP represents the number of pictures incorrectly predicted by the model as having abnormal heat sources, and FN represents the number of pictures incorrectly predicted by the model as not having abnormal heat sources.

[0069] Step S34: Load the trained yolov5 abnormal heat source monitoring model and use the gas station heat source image dataset F as input. Use the weighted non-maximum suppression operation to delete redundant overlapping detection frames for the same target and retain the detection frame with the highest confidence. The model outputs the maximum confidence of all abnormal heat sources detected in the current image coverage area, and uses this confidence as the abnormal probability P of fire signs in the xth area at time t. x,t , the calculation formula is:

[0070] P x,t =max(Coff)=max(P o ×P c )

[0071] Among them, Coff represents the confidence level, P o represents the probability that the bounding box contains any object, P c represents the probability that the bounding box contains a specific category, and max(·) represents the maximum value operation. In this embodiment, the confidence threshold P is set T is 0.6. When the confidence of the abnormal heat source in the identification result is ≥ 0.6, then P x,t The confidence calculated by the model is retained and output. When the confidence of the identified abnormal heat source is less than 0.6, "no abnormal heat source" is output. Considering the accidental nature of the accident, when outputting "no abnormal heat source", the smaller accidental probability of the accident is used as P. x,t Output, in this embodiment, is set to 10%.

[0072] Step S4: Set the fire sign abnormal probability P x,t Merge it with the sensor dataset D to create the leakage-fire dataset Leakage-Fire, establish a multimodal gas station leakage-fire anomaly recognition model, and train and optimize the model; specifically:

[0073] Step S41: Select 10,000 time-synchronized continuous pressure time series data P t ', Gas concentration time series data F t ', Fire probability time series data P x,t Data, the time series data at the same time is regarded as s (t) =(P t ', C t',P x,t ) Create a Leakage-Fire dataset. According to the actual gas data, (t) Add labels, including "normal", "leakage" and "fire-explosion risk". Divide into training sets s according to the ratio of 7:1:2 train ={s train (1) ,s train (2) ,...,s train (Ttrain)}、Validation set s val ={s val (1) ,s val (2) ,...,s val (Tval)} and test set s test ={s test (1) ,s test (2) ,...,s test (Ttest)}, construct the adjacency matrix A of the pipeline network topology in the form of a directed graph.

[0074] Step S42: Construct a multimodal gas station leakage-fire anomaly recognition model, which includes four parts: sensor information embedding, graph structure learning, graph attention-based prediction, and graph deviation scoring. The sensor information embedding part calculates the embedding vector of sensor i and the candidate relationship j∈C i The normalized dot product e between ji , and then select the first k normalized dot products, using TopK to represent the index of the first k values ​​of its input. The graph structure learning part uses the graph structure in the adjacency matrix A to fuse the node information with its neighboring node information. The aggregate representation of node i (using z i Represents) as:

[0075]

[0076] in, is the input feature of node i, N(i)={j|A ij >0} is the adjacency set obtained by node i from the learned adjacency matrix A, A ij indicates that there is a directed edge from node i to node j, W is the trainable weight matrix that applies a shared linear transformation to each node, and ReLU(·) represents the calculation of the nonlinear activation function ReLU.

[0077] The prediction part based on graph attention, the attention coefficient α i,j The calculation is as follows:

[0078]

[0079] Among them, ⊕ represents feature concatenation, a is the learning coefficient vector of the attention mechanism, LeakyReLU(·) represents the calculation of the nonlinear activation function LeakyReLU, and softmax(·) represents the calculation of the normalized exponential function. The attention coefficient is calculated using the LeakyReLU function, and then normalized using the softmax function. Each coefficient is element-wise multiplied by the corresponding time series embedding value, and the resulting values ​​of all nodes in the adjacency matrix A are used as the input of the fully connected layer.

[0080] In the graph deviation scoring section, the loss function is set to the mean square error between the predicted value and the true value, and then a separate anomaly score is calculated for each sensor, which can be used to locate the abnormal sensor. At the same time, these scores are combined to generate a summary anomaly score for each sensor at each time node. When calculating the anomaly score, the error value of each sensor is normalized, and the calculation process is:

[0081]

[0082] Among them, Error i (t) is the anomaly score between the true value and the predicted value, and They are Error i (t) median and interquartile range on the time scale, a i (t) is the normalized anomaly score of the ith node at time t. The normalized anomaly scores of all nodes are filtered using the max function. The anomaly scores of each sensor are aggregated to generate a smooth score A using the simple moving average method. i (t). If A i (t) exceeds the predetermined threshold A T,i , time t of the i-th node will be marked as abnormal;

[0083] Step S43: Normalize the training set, verification set and test set data. The normalization process is as follows:

[0084] s (t) '=(s (t) -min(s (t) )) / (max(s (t) )-min(s (t) ))

[0085] Among them, s (t)’The normalized training set is used to train the multimodal gas station leakage-fire anomaly recognition model, and then the validation set is input into the trained model. The A calculated in the validation set model is selected. i (t) The maximum value is taken as the abnormal threshold A T,i Using the F1 score as the model performance evaluation indicator, the model was trained, verified and tested multiple times by adjusting the model hyperparameters. The trained multimodal gas station leakage and fire anomaly recognition model corresponding to the largest F1 score in all test results was selected and applied to the gas leakage and fire anomaly recognition and positioning in the actual scene of the gas station.

[0086] Step S5: Use the trained multi-modal gas station leakage-fire anomaly recognition model, and combine the yolov5 abnormal heat source monitoring model to perform leakage-fire anomaly recognition on the recognition results of abnormal heat sources in the gas station scene. According to the output results of the model, when the output result of the multi-modal gas station leakage-fire anomaly recognition model is "normal", the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, and sends abnormal heat source warning information; when the output result of the multi-modal gas station leakage-fire anomaly recognition model is "leakage", a leakage warning information is sent; the output result of the multi-modal gas station leakage-fire anomaly recognition model is "there is a fire-explosion risk", and the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, and sends a fire and explosion warning information.

[0087] Step S51: Setting the output conditions of the multimodal gas station leakage-fire anomaly recognition model result. When the i-th node has 10 consecutive time series data s i (t) is marked as abnormal, and the time series data s (t) There are more than 3 P x,t ≥0.6, the anomaly is output as “there is a risk of fire and explosion”; otherwise, the model anomaly recognition result is output as “leakage”.

[0088] Step S52: When the recognition result of the multimodal gas station leakage-fire anomaly recognition model is "normal", but the yolov5 abnormal heat source monitoring model recognizes the existence of an abnormal heat source in the area, the abnormal heat source warning sent includes the abnormal heat source location information and the infrared image data of the area to which the abnormal pipe section belongs during the abnormal period output by the yolov5 abnormal heat source monitoring model.

[0089] Step S53: When the multi-modal gas station leakage-fire anomaly recognition model identifies the result as “leakage”, the 10 consecutive time series data s i (t) The corresponding model output anomaly scores are summed up to get A sum,i(t)'. Calculate the cumulative anomaly scores for all nodes in the adjacency matrix A and select A from the calculation results. sum,i (t)'The pipe sections connected to the largest and second largest abnormal nodes are marked as abnormal pipe sections, and the leakage warning information sent includes the leakage monitoring results output by the multi-modal gas station leakage-fire anomaly recognition model and A sum,i (t)'The largest and second largest abnormal node numbers.

[0090] Step S54: If the multi-modal gas station leakage-fire anomaly identification model identifies that the fire-explosion risk exists, the fire and explosion warning information sent includes the fire-explosion risk monitoring result output by the multi-modal gas station leakage-fire anomaly identification model, the A obtained in the same manner as step S53, and the sum,i (t)'The largest and second largest abnormal node numbers and the infrared image data of the area to which the abnormal pipe section belongs during the abnormal period output by the yolov5 abnormal heat source monitoring model.

[0091] Embodiment 2 A multi-modal fusion gas station leakage-fire warning system, which implements a multi-modal fusion gas station leakage-fire warning method, includes the following modules:

[0092] Data acquisition and storage module: Collects multimodal data required for gas station leakage and fire warning. Multimodal data includes infrared image data of each pipeline area of ​​the gas station collected by a dual-spectrum thermal imaging camera, pressure time series data collected by a pipeline pressure sensor, and gas concentration time series data collected by a gas concentration sensor. The data sampling interval is 5 seconds / time, and the collected data is stored in this module.

[0093] Data preprocessing module: normalizes the collected pressure time series data and gas concentration time series data, performs mosaic data enhancement and image adaptive scaling preprocessing on the infrared images collected by the dual-spectrum thermal imaging camera, and outputs the sensor time series data set D, D = [P', C'] and the gas station heat source image data set F, F = [G'].

[0094] Abnormal heat source identification model training and evaluation module: load and initialize the yolov5 abnormal heat source monitoring model, use the abnormal heat source image data set to train and test the yolov5 abnormal heat source monitoring model, and store the model parameter file with the best test results as the "best_fd.pt" file in the current module.

[0095] Abnormal heat source identification module: load the optimal model parameters in the "best.pt" file in the fire probability model training module described in this embodiment into the yolov5 abnormal heat source monitoring model, take the gas station heat source image dataset F as input, and output the infrared image of the area where the abnormal heat source is located marked by the box and the abnormal probability P of fire signs in the xth area at time t x,t .

[0096] Multimodal model training and evaluation module: load and initialize the multimodal gas station leakage-fire anomaly recognition model, and convert s train The data and the adjacency matrix A constructed based on the pipeline network topology structure are used as input to train the multimodal gas station leakage-fire anomaly recognition model. val Calculate the anomaly threshold A in the trained multimodal gas station leak-fire anomaly recognition model T,i , using s test Data testing: The trained model with the determined anomaly threshold is trained, and the training, validation, and testing processes are repeated multiple times. The model with the highest F1 score in the test results is selected, and the model parameters are adjusted based on the anomaly threshold A. T,i Save it as "best_lfd.pt" file.

[0097] Gas leak-fire identification module: Load the optimal model parameters in the "best_lfd.pt" file into the multimodal gas station leak-fire anomaly identification model, and align the pressure time series data, gas concentration time series data, and fire probability time series data to form a time series data s (t) Input the multimodal gas station leak-fire anomaly recognition model. The model output results include (1) normal; (2) when the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data T,i , time series data (t) Medium x,t If the amount of data with a value greater than 0.6 is less than 3, the output result is “leakage”; (3) When the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data T,i And the time series data s (t) There are more than 3 P x,t ≥0.6, the output result is "there is a risk of fire and explosion".

[0098] Leakage-fire warning module: Send warning information according to the output results of the abnormal heat source identification module and the gas leakage-fire identification module described in this embodiment. When the monitoring result of the gas leakage-fire identification module is "normal", but when the abnormal heat source identification module identifies the presence of an abnormal heat source in the area, this module sends abnormal heat source warning information and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel; when the monitoring result of the gas leakage-fire identification module is "leakage", but the abnormal heat source identification module does not identify the abnormal heat source, this module sends leakage warning information to the management personnel, and attaches the leakage pipe section number; when the monitoring result of the gas leakage-fire identification module is "there is a fire-explosion risk", this module sends fire-explosion warning information, leakage pipe section number and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel.

[0099] The above are only preferred embodiments of the present invention, but are not intended to limit the scope of protection of the present invention. Based on the technical solution of the present invention, all equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A multi-modal fusion gas station leakage-fire early warning method, characterized in that: The gas station leakage-fire early warning method first collects infrared image data, pressure and gas concentration time series data of the gas station pipeline; secondly, the collected data is preprocessed, including normalization, Mosaic data enhancement and adaptive scaling; Secondly, an abnormal heat source image dataset is created, and the yolov5 abnormal heat source monitoring model is constructed and trained to identify abnormal heat sources and calculate the abnormal probability of fire signs. Then, the abnormal probability of fire signs is merged with the sensor dataset to create a leakage-fire dataset, and a multimodal gas station leakage-fire anomaly recognition model is established for training and tuning. Finally, the trained multimodal model is used to monitor the leakage-fire anomaly of the gas station, and corresponding warning information is issued according to the output results.

2. A multi-modal fusion gas station leakage-fire early warning method according to claim 1, characterized in that: The following steps are involved: Step S1: Collect multimodal data for gas station pipeline leakage monitoring and regional fire warning, wherein the multimodal data includes infrared image data G collected by a dual-spectrum thermal imaging camera in each area of ​​the gas station, pressure time series data P collected by a pipeline pressure sensor, and gas concentration time series data C collected by a pipeline gas concentration sensor; Step S2: normalize and preprocess the collected pressure time series data and gas concentration time series data to construct a sensor time series data set D, D = [P', C'], perform Mosaic data enhancement and adaptive scaling on the infrared image data, and use the preprocessed infrared image data to create a gas station heat source image data set F, F = [G']; Step S3: Create an abnormal heat source image dataset, build a yolov5 abnormal heat source monitoring model, and perform training, fine-tuning, and testing. Take the gas station heat source image dataset F as input and calculate the abnormal probability P of fire signs in different areas of the gas station. x,t ; Step S4: Set the fire sign abnormal probability P x,t Merge it with the sensor dataset D to create the leakage-fire dataset Leakage-Fire, establish a multimodal gas station leakage-fire anomaly recognition model, and train and optimize the multimodal gas station leakage-fire anomaly recognition model; Step S5: Use the trained multimodal gas station leakage-fire anomaly recognition model, and combine it with the yolov5 abnormal heat source monitoring model to perform leakage-fire anomaly recognition on the recognition results of the abnormal heat source in the gas station scene; according to the output result of the model, when the output result of the multimodal gas station leakage-fire anomaly recognition model is "normal", the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, then sends abnormal heat source warning information; when the output result of the multimodal gas station leakage-fire anomaly recognition model is "leakage", send leakage warning information; the output result of the multimodal gas station leakage-fire anomaly recognition model is "there is a fire-explosion risk", and the yolov5 abnormal heat source monitoring model recognizes the abnormal heat source, and sends fire and explosion warning information.

3. A multi-modal fusion gas station leakage-fire early warning method according to claim 2, characterized in that: The step S2 is specifically: Step S21: normalizing the pressure time series data P and the gas concentration time series data C to form a sensor time series data set D, where D = [P', C']; Step S22: for the infrared image data G of the gas pipeline area collected by the dual-spectrum thermal imaging camera, an infrared image of the area is taken at the same time interval as the sensor data set D; Mosaic data enhancement is performed on the infrared image data G, and then image adaptive scaling is performed on the infrared image data after data enhancement to output an N×N scaled image; Step S23: Generate a gas station heat source image data set F from the pre-processed infrared image data, where F = [G'].

4. A multi-modal fusion gas station leakage-fire early warning method according to claim 3, characterized in that: The step S3 is specifically: Step S31: Use a dual-spectrum thermal imaging camera to shoot smoke image materials in any scene, use Labelme to mark the high-temperature objects in the abnormal heat source image with the number "1" to represent the "abnormal heat source" label, and form an abnormal heat source image dataset; Step S32: construct a yolov5 abnormal heat source monitoring model, wherein the core structure of the yolov5 abnormal heat source monitoring model consists of a backbone network and a neck network, and a combined loss function is constructed using binary cross entropy and CIoU; Step S33: load the yolov5 abnormal heat source monitoring model with randomly initialized hyperparameters, divide the images in the abnormal heat source image data set into a training set and a test set in proportion, perform image data preprocessing on the training set and the test set respectively according to the infrared image data processing method in step S22, and input the preprocessed image data into the yolov5 abnormal heat source monitoring model; train and test the model multiple times by adjusting the hyperparameters of the yolov5 abnormal heat source monitoring model multiple times, select the model with the highest F1 score in the test results as the final yolov5 abnormal heat source monitoring model, and use it for abnormal probability calculation of fire signs; Step S34: loading the yolov5 abnormal heat source monitoring model trained in step S33, taking the gas station heat source image dataset F as input; The weighted non-maximum suppression operation is used to delete redundant overlapping detection frames for the same target, and the detection frame with the highest confidence is retained. If multiple abnormal heat sources are identified in an image, the model will compare the confidence of each abnormal heat source in the image. The confidence refers to the predicted probability that the yolov5 abnormal heat source monitoring model will classify the identified object into the "abnormal heat source" category, and output the maximum confidence value of all abnormal heat sources identified in the current image. The maximum confidence value is used as the abnormal probability P of fire signs in the xth area at time t. x,t , set the confidence threshold P T , when the confidence of abnormal heat source in the identification result ≥ P T , then P x,t Keep and output the confidence calculated by the model. When the confidence of the identified abnormal heat source is less than P T , then the output is "no abnormal heat source". Considering that accidents are accidental, when outputting "no abnormal heat source", the smaller accidental probability of the accident is used as P x,t Output.

5. A multi-modal fusion gas station leakage-fire early warning method according to claim 4, characterized in that: In step S34, the confidence threshold P T is 0.

6.

6. A multi-modal fusion gas station leakage-fire early warning method according to claim 4, characterized in that: The step S4 is specific as follows: Step S41: Select multiple time-synchronized continuous pressure time series data P t ', Gas concentration time series data F t ', Fire probability time series data P x,t Data, the time series data at the same time is regarded as s (t) =(P t ', C t ',P x,t ) Create a leakage-fire dataset Leakage-Fire; According to the actual gas data, for each s (t) Add labels, including "Normal", "Leakage" and "Fire-Explosion Risk"; Divide the leak-fire dataset into training sets s respectively according to the proportion train , validation set val and test set s test , construct the adjacency matrix A of the pipe network topology in the form of a directed graph; Step S42: constructing a multimodal gas station leakage-fire anomaly recognition model, wherein the structure of the multimodal gas station leakage-fire anomaly recognition model includes four parts: sensor information embedding, graph structure learning, graph attention-based prediction, and graph deviation scoring; Step S43: normalize the training set, verification set and test set data divided in step S41; use the normalized training set to train the multimodal gas station leakage-fire anomaly recognition model, then input the verification set into the trained model, select the A of each node calculated in the verification set model s,i (t) The maximum value is taken as the abnormal threshold A T,i ; Use the F1 score as the model performance evaluation indicator, train, verify and test the model multiple times by adjusting the model hyperparameters, and select the trained multimodal gas station leakage and fire anomaly recognition model corresponding to the highest F1 score in all test results for gas leakage and fire anomaly recognition and positioning in actual gas station scenarios.

7. A multi-modal fusion gas station leakage-fire early warning method according to claim 6, characterized in that: In the step S42: The sensor information embedding part calculates the embedding vector of sensor i and the candidate relationship j∈C i The normalized dot product e between ji , then select the first k normalized dot products and use TopK to represent the index of the first k values ​​of its input; The graph structure learning part uses the graph structure in the adjacency matrix A to fuse the node information with its neighboring node information, and calculates the attention coefficient α based on the prediction part of the graph attention. i,j , then each coefficient α i,j They are all multiplied element-by-element with the corresponding time series embedding values, and the resulting values ​​of all nodes in the adjacency matrix A are used as the input of the fully connected layer; The graph deviation scoring part sets the loss function as the mean square error between the predicted value and the true value, and then calculates the individual anomaly score of each sensor and normalizes it. The normalized anomaly score a of each node is i (t) Screening, aggregation of the anomaly scores of each sensor to generate a smooth score A i (t); if A i (t) exceeds the predetermined threshold A T,i , the time t of the i-th node will be marked as abnormal; the loss function is the mean square error between the predicted value and the true value, and the predicted value refers to the multimodal gas station leakage-fire anomaly recognition model based on the data s in the input leakage-fire dataset (t) Predicted data (t+1) ', the true value refers to the data s corresponding to the leak-fire dataset at time t+1 (t+1) .

8. A multi-modal fusion gas station leakage-fire early warning method according to claim 6, characterized in that: The step S5 is specifically: Step S51: Setting the model result output condition. When the i-th node has multiple consecutive time series data s i (t) is marked as abnormal, and the time series data s (t) There are more than 3 P x,t ≥P T , the anomaly is output as "there is a risk of fire and explosion"; otherwise, the model anomaly recognition result is output as "leakage"; Step S52: When the multimodal gas station leakage-fire anomaly recognition model recognizes that the result is "normal", but the yolov5 abnormal heat source monitoring model recognizes that there is an abnormal heat source in the area, the abnormal heat source warning sent includes the abnormal heat source location information and the infrared image data of the abnormal period of the area to which the abnormal heat source belongs; Step S53: When the multi-modal gas station leakage-fire anomaly recognition model identifies a "leakage", the anomaly scores output by the model corresponding to the continuous anomaly period data are accumulated and summed to obtain A sum,i (t)'; Calculate the cumulative anomaly scores of all nodes in the adjacency matrix A, and select A from the calculation results sum,i (t)'The pipe sections connected to the largest and second largest abnormal nodes are marked as abnormal pipe sections, and the leakage warning information sent includes the leakage monitoring results output by the multi-modal gas station leakage-fire anomaly recognition model and A sum,i (t)'The largest and second largest abnormal node numbers; Step S54: If the multi-modal gas station leakage-fire anomaly recognition model identifies that the fire-explosion risk exists, the fire and explosion warning information sent includes the fire-explosion risk monitoring result output by the multi-modal gas station leakage-fire anomaly recognition model, the A obtained in the same manner as step S53, sum,i (t)'The largest and second largest abnormal node numbers and the infrared image data of the area to which the abnormal pipe section belongs during the abnormal period output by the yolov5 abnormal heat source monitoring model.

9. A multi-modal fusion gas station leakage-fire warning system, characterized in that: The system is used to execute the multi-modal fusion gas station leakage-fire warning method according to any one of claims 1 to 8, and includes the following modules: Data acquisition and storage module: collects multimodal data required for gas station leakage and fire warning. The multimodal data includes infrared image data of each pipe network area of ​​the gas station collected by a dual-spectrum thermal imaging camera, pressure time series data collected by a pipeline pressure sensor, and gas concentration time series data collected by a gas concentration sensor. The collected data is stored in the data acquisition and storage module; Data preprocessing module: normalizes the collected pressure time series data and gas concentration time series data, performs Mosaic data enhancement and image adaptive scaling preprocessing on the infrared images collected by the dual-spectrum thermal imaging camera, and outputs the sensor time series data set D, D = [P', C'] and the gas station heat source image data set F, F = [G']; Abnormal heat source identification model training and evaluation module: load and initialize the yolov5 abnormal heat source monitoring model, use the abnormal heat source image data set to train and test the yolov5 abnormal heat source monitoring model, and take the model with the best test results as the optimal model; Abnormal heat source identification module: load the optimal model parameters into the yolov5 abnormal heat source monitoring model, take the gas station heat source image dataset F as input, and output the infrared image of the area where the abnormal heat source is marked by the box and the abnormal probability P of fire signs in the xth area at time t x,t ; Multimodal model training and evaluation module: load and initialize the multimodal gas station leakage-fire anomaly recognition model, and convert s train The data and the adjacency matrix A constructed based on the pipeline network topology structure are used as input to train the multimodal gas station leakage-fire anomaly recognition model. val Calculate the anomaly threshold A in the trained multimodal gas station leak-fire anomaly recognition model T,i , using s test The data test is to train the model with the determined anomaly threshold, repeat the training, validation and testing process for many times, select the model with the highest F1 score in the test results as the optimal model, and use the optimal model parameters as the anomaly threshold A. T,i save; Gas leak-fire identification module: load the optimal model parameters into the multimodal gas station leak-fire anomaly identification model, align the pressure time series data, gas concentration time series data and fire probability time series data to form a time series data s (t) Input the multimodal gas station leak-fire anomaly recognition model, and the model output results include: (1) "normal"; (2) When the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data T,i , time series data (t) Medium x,t If the amount of data with a value greater than 0.6 is less than 3, the output result is "leakage"; (3) When the anomaly score of any node exceeds the anomaly threshold A for 10 consecutive time series data T,i And the time series data s (t) There are more than 3 P x,t ≥0.6, the output result is "there is a risk of fire and explosion"; Leakage-fire warning module: sends warning information based on the output results of the abnormal heat source identification module and the gas leakage-fire identification module; when the monitoring result of the gas leakage-fire identification module is "normal", but the abnormal heat source identification module identifies the existence of an abnormal heat source in the area, this module sends abnormal heat source warning information and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel; when the monitoring result of the gas leakage-fire identification module is "leakage", but the abnormal heat source identification module does not identify the abnormal heat source, this module sends leakage warning information to the management personnel, together with the leakage pipe section number; when the monitoring result of the gas leakage-fire identification module is "there is a fire-explosion risk", this module sends fire-explosion warning information, leakage pipe section number and infrared images of the abnormal period of the area where the abnormal heat source is marked by the box to the management personnel.

Images