Fire source detection method, system and application based on infrared thermal image analysis in petrochemical area

Abstract

The application provides a petrochemical area fire source detection method and system based on infrared thermal image analysis and application, and belongs to the technical field of image processing. The method comprises the following steps: collecting an indoor and outdoor infrared thermal imaging fire source scene, generating an infrared binary image sequence through gray threshold division, then performing gray threshold division and morphological transformation on a visible light image to generate a binary image sequence similar to the infrared thermal imaging, and comprehensively obtaining a binary fire source detection sequence; increasing an image prediction channel and adding an attention mechanism to obtain the average precision mean predicted by a deep neural network; further analyzing the time domain characteristics of the target predicted by the deep learning network, finally extracting the target meeting the condition to obtain the final fire source detection position of the irregular highlight area in the oil pipeline. The application can effectively identify the fire source target, greatly reduce false positives, and is conducive to wide application in petrochemical monitoring scenes.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically discloses a method, system and application for detecting fire sources in petrochemical areas based on infrared thermal image analysis. Background Technology

[0002] Fire is the most significant threat to safety in the oil and petrochemical industry. Early detection of fires and the implementation of effective emergency response measures are crucial for reducing and preventing major loss of life and property. Infrared thermal imaging is unaffected by electromagnetic interference, has a long operating range, can image in the absence of light, is unaffected by adverse weather conditions such as rain, snow, and fog, enabling 24 / 7 monitoring. Furthermore, it can display the temperature distribution on object surfaces, making it ideal for fire monitoring and alarm systems in critical areas of petrochemical zones.

[0003] Currently, video surveillance systems for fire source detection in petrochemical zones mainly fall into two categories: visible light-based target recognition and infrared thermal imaging temperature measurement. Visible light-based target recognition is easily affected by interference from red or yellow clothing, flags, helmets, and other objects, leading to false alarms and impacting system functionality. Infrared thermal imaging temperature measurement typically uses set over-temperature thresholds for warnings of high-temperature targets. However, due to the numerous high-temperature pipelines in petrochemical zones, threshold setting is extremely difficult, resulting in numerous false alarms for high-temperature pipelines and a poor user experience. With the development of deep learning technology, using convolutional neural networks combined with infrared imaging to analyze infrared images and directly identify fire sources is an effective way to solve the high false alarm rate in petrochemical zone fire source detection systems.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0005] (1) Existing technologies cannot effectively identify fire source targets, have a high false alarm rate, and have low accuracy of data information monitored in petrochemical monitoring scenarios.

[0006] (2) The reliability of existing fire source detection methods is poor. Summary of the Invention

[0007] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide a method, system and application for detecting fire sources in petrochemical areas based on infrared thermal image analysis.

[0008] According to a first aspect of this application, the present invention provides a method for detecting fire sources in petrochemical areas based on infrared thermal image analysis. This method utilizes a deep neural network to obtain suspected targets, performs temporal feature analysis on the obtained suspected targets and determines the location of the fire source, and issues an alarm for dangerous fire sources in the petrochemical area based on the determination result. Specifically, it includes the following steps:

[0009] S1, by performing grayscale thresholding and morphological transformation on visible light images, a binary image sequence similar to infrared thermal imaging is generated, and combined with the original binary image sequence extracted from infrared thermal imaging to generate a binary fire source image training dataset.

[0010] S2, by increasing the image prediction channel and adding an attention mechanism, obtains the average accuracy of deep neural network prediction;

[0011] S3. Further analysis of the temporal characteristics of the target predicted by the deep learning network, and finally extraction of targets that meet the criteria of static target recognition and temporal dynamic feature determination, to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

[0012] In step S1, generating a binary image sequence of infrared thermal imaging by performing grayscale thresholding and morphological transformation on the visible light image specifically includes the following steps:

[0013] A training dataset for fire source images was constructed, comprising uncooled infrared fire source data acquisition sequences and visible light conversion dataset sequences. The uncooled infrared fire source data was obtained by capturing real indoor and outdoor fire source scenes using an uncooled infrared imager, and the dataset was generated by extracting binarized fire source targets through threshold segmentation. The visible light conversion dataset was generated by downloading visible light images from various authoritative fire source visible light databases, and processing them through colorimetric analysis and morphological algorithms to produce a similar binary image dataset to the infrared fire source images. The dataset was converted into binary fire source target images and backgrounds through grayscale thresholding and morphological transformation, ensuring similarity between the binary image sequences converted from the uncooled infrared fire source data acquisition sequences and the visible light conversion binary image sequences.

[0014] The uncooled infrared fire source data acquisition sequence uses an uncooled infrared imager to acquire real indoor and outdoor fire source scenes, and extracts binarized fire source targets through threshold segmentation to generate a dataset; the visible light conversion dataset sequence downloads visible light images from various authoritative fire source visible light databases (including datasets published by Bilkent University in Turkey, the CV&PR Laboratory of Keimyung University in South Korea, and the University of Science and Technology of China), and processes them through colorimetric analysis and morphological algorithms to generate a similar infrared fire source binarized image dataset.

[0015] In one embodiment, the uncooled infrared fire source data acquisition sequence is converted into a binarized fire source target image. Threshold segmentation and binarization image processing are then performed on the input uncooled infrared fire source data acquisition sequence to obtain a binarized image I(x,y), as shown in the following equation:

[0016]

[0017] In the above formula, I RawThis is a 14-bit infrared raw image, where (x,y) are the column and row coordinates of the pixel, and μ(I) is the pixel coordinate. Raw ) represents the mean of the 14-bit infrared Raw image, and TH is the threshold for identifying high-heat targets, which is set to 1000 by default.

[0018] In step S2, obtaining the average accuracy of the deep neural network prediction specifically includes: combining the attention mechanism module in the two dimensions of grayscale and channel, feeding the binarized image I(x,y) into the deep neural network to predict the target at three feature scales of 13×13, 26×26 and 52×52, and using the one-stage deep network to directly obtain the classification confidence prediction and location prediction of the suspected fire source target;

[0019] Here, one-stage and two-stage refer to the classification of object detection frameworks. The YOLOv4-tiny framework is a type of one-stage detection that can directly output the target location, classification, and confidence score in the fully connected layer of a deep network. Two-stage detection, on the other hand, first detects the target location and outputs the target coordinates in the first stage, and then sends them to the second stage to detect the target classification and confidence score.

[0020] The targets include: large targets, medium targets, and small targets in the corresponding binarized image.

[0021] In one embodiment, the obtained suspected fire source targets are stored in a candidate fire source list. Connected component calculation is performed on the obtained binarized image I(x,y). The region growing method is used to scan the binarized image line by line. Each time an unlabeled non-zero pixel is encountered, an unused label is assigned, and its neighborhood is detected. The same operation is repeated until there are no non-zero pixels that can be assigned the same label in the neighborhood of the passed pixel. Then the image line scanning continues, and the same processing steps as above are performed. Until the entire image scanning is completed, connected components with different labels correspond to different candidate regions of highlighted targets in the binarized image.

[0022] Obtaining the candidate region of the highlighted target includes: comparing the target positions in the candidate region of the highlighted target with the target positions in the candidate fire source list predicted by the deep neural network, calculating the IOU between the target in the candidate region of the highlighted target and the target in the candidate fire source list, and setting the IOU matching threshold. The default IOU matching threshold is 80%.

[0023] The system matches the highlighted target candidate area with the target in the candidate fire source list. For successfully matched candidate area targets, the area variance change is analyzed in the time domain dimension. When the variance change is greater than the time domain area judgment threshold, which is 15 by default, it is judged as a real fire source and an alarm is output.

[0024] According to a second aspect of this application, the present invention provides a petrochemical zone ignition source detection system based on infrared thermal image analysis. This system is applied to a PC platform, where deep learning target recognition utilizes a GPU for prediction, and the CPU is responsible for temporal feature analysis calculations and running the aforementioned petrochemical zone ignition source detection method based on infrared thermal image analysis. The petrochemical zone ignition source detection system based on infrared thermal image analysis includes:

[0025] The binarized image sequence generation module is used to generate a binary image sequence similar to infrared thermal imaging by performing grayscale thresholding and morphological transformation on visible light images, and to generate an infrared thermal imaging binary image sequence by performing grayscale thresholding on infrared thermal imaging image sequences. The two are combined to generate a binary image sequence for fire source detection.

[0026] The deep neural network prediction module is used to obtain the average accuracy of deep neural network predictions by increasing the number of image prediction channels and adding an attention mechanism.

[0027] The fire source detection location acquisition module is used to further analyze the temporal features of the target predicted by the deep learning network, and finally extract the target that meets the conditions to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

[0028] In one embodiment, the binarized image sequence generation module includes:

[0029] A fire source image training dataset construction module is used to construct uncooled infrared fire source data acquisition sequences and visible light conversion dataset sequences;

[0030] The binarization image acquisition module is used to process infrared thermal imaging images and visible light images, converting them into binarized fire source target images and backgrounds. Through grayscale thresholding and morphological transformation, the binarized visible light images are made similar to the binarized infrared fire source images.

[0031] The deep neural network prediction module includes:

[0032] The attention mechanism module is used to provide grayscale and channel dimensions;

[0033] A deep neural network is used to predict targets in three dimensions from an input binarized image, obtaining classification confidence prediction and location prediction for suspected fire sources; the targets include: large targets, medium targets, and small targets in the corresponding binarized image;

[0034] The fire source detection location acquisition module includes:

[0035] The module for obtaining candidate areas of highlighted targets is used to store the obtained suspected fire source targets in the candidate fire source list and to perform connected component calculation on the obtained binarized image to obtain candidate areas of highlighted targets.

[0036] The real fire source acquisition module is used to compare the target location in the highlighted target candidate area with the target location in the candidate fire source list, calculate the candidate area at the same location using the IOU threshold, and analyze the area variance change of the candidate area in the time domain dimension. When the variance change is greater than a certain threshold, it is determined to be a real fire source and an alarm is output.

[0037] According to a third aspect of this application, the present invention provides a user input program storage medium, wherein the stored computer program causes an electronic device to execute the described method for detecting fire sources in petrochemical zones based on infrared thermal image analysis.

[0038] According to a fourth aspect of this application, the present invention provides an information data processing terminal, which is installed on an electronic device and provides a user input interface to implement the aforementioned method for detecting fire sources in petrochemical zones based on infrared thermal image analysis.

[0039] Combining all the above technical solutions, the advantages and positive effects of this invention are as follows:

[0040] First, in view of the technical problems existing in the prior art and the difficulty of solving these problems, and closely combining the technical solution to be protected by this invention with the results and data during the research and development process, this paper analyzes in detail how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about after solving the problems, as described in detail below:

[0041] (1) The present invention uses an infrared thermal imaging detector as the core sensor. Compared with the traditional visible light fire source detection scheme, it can effectively utilize the temperature sensing characteristics of infrared thermal imaging to detect high temperature targets earlier, thereby effectively eliminating the interference of common visible light fire source detection objects such as flags, clothes, and helmets.

[0042] (2) In view of the shortcomings of existing petrochemical zone fire source detection technology based on image analysis, this invention provides a petrochemical zone fire source detection method based on deep learning and temporal feature discrimination, which can effectively identify fire source targets and greatly reduce false alarms, which is conducive to its widespread application in petrochemical monitoring scenarios.

[0043] (3) Compared with other infrared thermal imaging fire source detection methods based on temperature measurement, this invention solves the problem of difficulty in determining alarm threshold and the need to set up a large number of false alarm zones to eliminate false alarms. It can effectively eliminate false alarms caused by high-temperature pipelines in petrochemical areas and improve the intelligence level of fire source detection equipment.

[0044] Secondly, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution protected by this invention are specifically described as follows: This invention generates an image sequence similar to infrared thermal imaging by performing grayscale thresholding and morphological transformation on visible light images, effectively solving the problem of limited infrared thermal imaging fire source image samples. By increasing image prediction channels and adding an attention mechanism, the mean average accuracy (mAP) of deep neural network prediction is effectively improved. Finally, the temporal characteristics of the targets predicted by the deep learning network are further analyzed, and targets that meet the conditions are finally extracted to obtain the final fire source detection output. This greatly reduces false alarms caused by irregular bright areas in complex oil pipelines and effectively improves the reliability of the fire source detection method.

[0045] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0046] (1) After the technical solution of the present invention is transformed, it can be applied on a large scale in the intelligent security monitoring system of petrochemical zone, and can be promoted for fire prevention applications in places such as gas stations and charging stations.

[0047] (2) This invention solves the problem that the current domestic infrared thermal imaging fire source detection relies heavily on temperature measurement methods, resulting in a high false alarm rate and making the product difficult to apply in complex scenarios in petrochemical areas. Attached Figure Description

[0048] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0049] Figure 1 This is a flowchart of the petrochemical zone fire source detection method based on infrared thermal image analysis provided in this embodiment of the invention;

[0050] Figure 2 This is a schematic diagram of the deep neural network fire source detection framework with an enhanced attention mechanism provided in this embodiment of the invention.

[0051] Figure 3 This is a schematic diagram of the method for detecting fire sources in petrochemical areas based on infrared thermal image analysis provided in this embodiment of the invention.

[0052] Figure 4 This is a schematic diagram of a petrochemical zone fire source detection system based on infrared thermal image analysis provided in an embodiment of the present invention;

[0053] Figure 5 This invention provides a solution for on-site fire source detection in petrochemical zones, featuring an explosion-proof pan-tilt design to scan the monitored area. The corresponding upper computer interface is shown in the image.

[0054] Figure 6 is a diagram showing the identification effect of the fire source detection algorithm under different petrochemical zone monitoring scenarios provided in the embodiments of the present invention.

[0055] Figure 6(a) shows the static analysis of the fire source detection algorithm identification in different petrochemical zone monitoring scenarios provided by the embodiments of the present invention, and the effect of identifying false fire sources (actually high-heat petrochemical pipelines);

[0056] Figure 6(b) shows the static analysis of the fire source detection algorithm identification in different petrochemical zone monitoring scenarios provided by the embodiments of the present invention, and the effect of identifying false fire sources (actually high-heat petrochemical pipelines);

[0057] Figure 6(c) shows the static analysis of the fire source detection algorithm identification in different petrochemical zone monitoring scenarios provided by the embodiments of the present invention, and the effect of identifying false fire sources (actually high-heat petrochemical pipelines);

[0058] Figure 6(d) shows the static analysis of the fire source detection algorithm identification in different petrochemical zone monitoring scenarios provided by the embodiments of the present invention, and the effect of identifying false fire sources (actually high-heat petrochemical pipelines);

[0059] Figure 6(e) is a diagram showing the effect of flames caused by oil and gas leaks in two actual fire hazards provided by the embodiment of the present invention;

[0060] Figure 6(f) shows the correct alarm effect diagram of the sparks generated by on-site electric welding in two actual fire hazards provided by the embodiment of the present invention, which was correctly alarmed by combining static feature analysis with the temporal feature judgment of the fire source;

[0061] Figure 6(g) is a visible light image effect diagram of Figure 6(f) provided in an embodiment of the present invention;

[0062] In the diagram: 1. Binarized image sequence generation module; 2. Deep neural network prediction module; 3. Fire source detection location acquisition module. Detailed Implementation

[0063] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0064] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0065] I. Explanation of the Implementation Example:

[0066] This invention provides a method for detecting fire sources in petrochemical areas based on infrared thermal image analysis. The method utilizes a deep neural network to obtain suspected targets, performs temporal feature analysis on the obtained suspected targets, determines the location of the fire source, and issues an alarm for dangerous fire sources in the petrochemical area based on the determination results.

[0067] Specifically, the method for detecting ignition sources in petrochemical areas based on infrared thermal image analysis includes the following steps:

[0068] Step 1: By performing grayscale thresholding and morphological transformation on the visible light image, a binary image sequence similar to infrared thermal imaging is generated, and combined with the original binary image sequence extracted from infrared thermal imaging to generate a binary fire source image training dataset.

[0069] Step 2: By increasing the number of image prediction channels and adding an attention mechanism, the mean average accuracy (mAP) of the deep neural network prediction is obtained.

[0070] Step 3: Further analyze the temporal characteristics of the target predicted by the deep learning network, and finally extract the targets that meet the criteria of static target recognition and temporal dynamic feature determination to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

[0071] Example 1

[0072] like Figure 1 As shown, the method for detecting fire sources in petrochemical areas based on infrared thermal image analysis provided in this embodiment of the invention includes the following steps:

[0073] S101, Construct a fire source image training dataset. The fire source image training dataset is mainly divided into two parts: uncooled infrared fire source data acquisition sequence and visible light conversion dataset sequence. In order to achieve data similarity between the uncooled infrared fire source data acquisition sequence and the visible light conversion dataset sequence, grayscale thresholding and morphological transformation are performed on the visible light images to generate a binary image sequence similar to infrared thermal imaging. This binary image sequence is then combined with the original binary image sequence extracted from infrared thermal imaging to generate a binary fire source image training dataset.

[0074] S102 employs a deep neural network model with an added attention mechanism to train the fire source image dataset. By increasing the image prediction channel, it effectively utilizes the feature information of the shallow network, and can further improve the network's detection performance for small fire source areas without significantly changing the network size.

[0075] S103 uses binarization segmentation combined with connected components to find suspected fire source locations and performs ROI judgment with the target locations predicted by the deep learning network. For targets that are judged to be overlapping, the area change characteristics are analyzed in the time domain, which effectively avoids false alarms caused by bright oil pipelines with static features similar to fire sources.

[0076] In step S101, a fire source image training dataset is constructed. The fire source image training dataset includes an uncooled infrared fire source data acquisition sequence and a visible light conversion dataset sequence. The uncooled infrared fire source data is obtained by acquiring real indoor and outdoor fire source scenes using an uncooled infrared imager. The dataset is generated by extracting binarized fire source targets through threshold segmentation. The visible light conversion dataset is generated by downloading visible light images from various authoritative fire source visible light databases and processing them through colorimetric analysis and morphological algorithms. The dataset is converted into a similar binary image dataset of infrared fire sources. The binary image sequence converted from the uncooled infrared fire source data acquisition sequence and the binary image sequence converted from the visible light conversion sequence are similar.

[0077] In step S102, obtaining the average accuracy of the deep neural network prediction specifically includes: combining the attention mechanism module to feed the binarized image I(x,y) into the deep neural network to predict the target at three feature scales of 13×13, 26×26 and 52×52 in both grayscale and channel dimensions, and using the one-stage deep network to directly obtain the classification confidence prediction and location prediction of the suspected fire source target;

[0078] Those skilled in the art will understand that one-stage and two-stage refer to the classification of object detection frameworks. The YOLOv4-tiny framework is a type of one-stage detection, which can directly output the target location, classification, and confidence score in the fully connected layer of a deep network. Two-stage detection, on the other hand, first detects the target location and outputs the target coordinates in the first stage, and then sends them to the second stage to detect the target classification and confidence score.

[0079] The targets include: large targets, medium targets, and small targets in the corresponding binarized image.

[0080] In this embodiment of the invention, the obtained suspected fire source targets are stored in a candidate fire source list. Connected component calculation is performed on the obtained binarized image I(x, y). A region growing method is used to scan the binarized image row by row. For each unlabeled non-zero pixel encountered, an unused label is assigned, and its neighborhood is detected. If any unlabeled non-zero pixels are found, they are assigned the same label. This operation is repeated until no non-zero pixels with the same label exist in the neighborhood of the traversed pixel. Then, the image row (column) scanning continues. If an unlabeled non-zero pixel is detected, it is assigned a new label, and the same processing is performed. The algorithm terminates when the entire image scan is complete. Connected components with different labels correspond to different candidate regions for highlighted targets in the binarized image.

[0081] Obtaining the highlighted target candidate area involves comparing the target locations in the highlighted target candidate area with those in the candidate fire source list predicted by the deep neural network, calculating the Interchange of Units (IOU) between the highlighted target candidate area and the targets in the candidate fire source list, setting an IOU matching threshold (the default matching threshold can be 80%), thereby completing the matching between the highlighted target candidate area and the targets in the candidate fire source list. For successfully matched candidate area targets, the area variance change is analyzed in the time domain dimension. When the variance change is greater than the time domain area judgment threshold, where the default judgment threshold is 15, it is judged as a real fire source and an alarm is output.

[0082] Example 2

[0083] The petrochemical zone fire source detection method based on infrared thermal image analysis provided in this embodiment of the invention is implemented using a PC platform. Deep learning target recognition is performed using a GPU for prediction, and temporal feature analysis is performed by a CPU.

[0084] (1) Convert the uncooled infrared fire source data acquisition sequence into a binary fire source target image. Perform threshold segmentation and binarization image processing on the input uncooled infrared fire source data acquisition sequence to obtain the binary image I(x,y), as shown in equation (1) below:

[0085]

[0086] In the above formula, I Raw This is a 14-bit infrared raw image, where (x,y) are the column and row coordinates of the pixel, and μ(I) is the pixel coordinate. Raw ) represents the mean of the 14-bit infrared Raw image, and TH is the threshold for identifying high-heat targets, which can be set to 1000 by default.

[0087] (2) The binarized image I(x, y) is fed into a deep neural network for prediction. The deep neural network framework is as follows: Figure 2 As shown.

[0088] Deep neural networks predict targets in three dimensions, corresponding to large, medium, and small targets in a binary image. Adding an attention mechanism module can improve the detection capability of deep neural networks in both grayscale and channel dimensions. Figure 2 The deep neural network can obtain classification confidence prediction and location prediction of suspected fire source targets.

[0089] (3) Store the suspected fire source targets obtained in step (2) in the candidate fire source list. Perform connected component calculation on the binarized image I(x, y) obtained in step (1) to obtain the candidate region of the bright target. Compare the target positions of the candidate region of the bright target with the target positions in the candidate fire source list. Calculate the candidate region of the same position using the IOU threshold. Then analyze the area variance change of the candidate region in the time domain. When the variance change is greater than a certain threshold, it is determined to be a real fire source and an alarm is output. The general principle of the fire source detection method in petrochemical areas based on infrared thermal image analysis is as follows: Figure 3 As shown.

[0090] Example 3

[0091] The general principle of the method for detecting fire sources in petrochemical areas based on infrared thermal image analysis is as follows: Figure 3 As shown. Specifically, it includes the following steps:

[0092] Step 1: Acquire infrared images;

[0093] Step 2, extract the foreground image;

[0094] Step 3, thermal target extraction;

[0095] Step 4: Deep learning for suspected fire source identification;

[0096] Step 5: Determine if there is a potential fire source; if yes, proceed to the next step; otherwise, return to step 1.

[0097] Step 6: Store the information of suspected fire sources in the candidate list;

[0098] Step 7: Extract the connected components of the highlighted target;

[0099] Step 8: Match the coordinates of connected components with suspected fire source targets;

[0100] Step 9: Record the area change information;

[0101] Step 10: Determine if the variance of the area change exceeds the determination threshold;

[0102] Step 11: Identify a fire source and trigger an alarm. If the fire source is unclear, return to step 12 and repeat the process until a clear and accurate fire source is obtained.

[0103] Example 4

[0104] This invention provides a petrochemical zone fire source detection system based on infrared thermal image analysis, applied to a PC platform. Deep learning target recognition utilizes the GPU for prediction, while temporal feature analysis is performed by the CPU, and the aforementioned petrochemical zone fire source detection method based on infrared thermal image analysis is executed.

[0105] like Figure 4As shown, the petrochemical zone ignition source detection system based on infrared thermal image analysis includes:

[0106] Binarized image sequence generation module 1 is used to generate a binary image sequence similar to infrared thermal imaging by performing grayscale thresholding and morphological transformation on a visible light image, and to generate an infrared thermal imaging binary image sequence by performing grayscale thresholding on an infrared thermal imaging image sequence. The two are combined to generate a fire source detection binary image sequence.

[0107] The deep neural network prediction module 2 is used to obtain the mean average accuracy (mAP) of deep neural network prediction by increasing the number of image prediction channels and adding an attention mechanism.

[0108] The fire source detection location acquisition module 3 is used to further analyze the temporal features of the target predicted by the deep learning network, and finally extract the target that meets the conditions to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

[0109] More preferably, the binarized image sequence generation module 1 includes:

[0110] A fire source image training dataset construction module is used to construct uncooled infrared fire source data acquisition sequences and visible light conversion dataset sequences;

[0111] The binarization image acquisition module is used to process infrared thermal imaging images and visible light images, converting them into binarized fire source target images and backgrounds. Through grayscale thresholding and morphological transformation, the binarized visible light images are made similar to the binarized infrared fire source images.

[0112] The deep neural network prediction module 2 includes:

[0113] The attention mechanism module is used to provide grayscale and channel dimensions;

[0114] A deep neural network is used to predict targets in three dimensions from an input binarized image, obtaining classification confidence prediction and location prediction for suspected fire sources; the targets include: large targets, medium targets, and small targets in the corresponding binarized image;

[0115] The fire source detection location acquisition module 3 includes:

[0116] The module for obtaining candidate areas of highlighted targets is used to store the obtained suspected fire source targets in the candidate fire source list and to perform connected component calculation on the obtained binarized image to obtain candidate areas of highlighted targets.

[0117] The real fire source acquisition module is used to compare the target location in the highlighted target candidate area with the target location in the candidate fire source list, calculate the candidate area at the same location using the IOU threshold, and analyze the area variance change of the candidate area in the time domain dimension. When the variance change is greater than a certain threshold, it is determined to be a real fire source and an alarm is output.

[0118] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0119] The information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments.

[0121] II. Application Examples:

[0122] Application examples

[0123] This invention has been practically deployed and applied in a petrochemical plant area, providing a petrochemical area fire source detection system. The system includes: multiple front-end monitoring devices, a video server, and an information processing server.

[0124] The front-end monitoring equipment is an explosion-proof fire source detection device, which includes two network video streams: infrared thermal imaging and visible light, and pushes them to the back-end in accordance with the monitoring industry standard ONVIF format.

[0125] The video server can save the video from the front-end monitoring points of the subnet segment for internal data storage within the subsystem, facilitating later problem analysis and tracing.

[0126] The information processing server is the operating platform for the backend host computer software. The host computer software receives and displays the video stream from the front-end device. For fire source detection applications, it is also necessary to decode the video stream to extract real-time infrared image data, run the fire source detection algorithm described in this invention, and overlay the alarm box back onto the video stream.

[0127] This invention provides a computer device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above-described method embodiments.

[0128] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps described in the various method embodiments above.

[0129] This invention also provides an information data processing terminal, which, when executed on an electronic device, provides a user input interface to implement the steps described in the above method embodiments. The information data processing terminal is not limited to mobile phones, computers, or switches.

[0130] This invention also provides a server that, when executed on an electronic device, provides a user input interface to implement the steps described in the above method embodiments.

[0131] This invention provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] III. Evidence of the relevant effects of the embodiments:

[0135] Experiments show

[0136] The petrochemical zone fire source detection method based on deep learning and time-domain feature discrimination provided by this invention uses a deep neural network to obtain suspected targets, and then judges the fire source by analyzing the time-domain features of the suspected targets. This effectively solves the problem of high false alarm rate based on visible light image recognition and infrared imaging temperature measurement, and can effectively improve the level of intelligent fire source detection in petrochemical zones.

[0137] To address the on-site fire source detection needs in petrochemical zones, an explosion-proof pan-tilt-zoom (PTZ) design is adopted for patrolling the monitored area. The corresponding upper-computer interface is shown below. Figure 5 As shown.

[0138] In actual working scenarios, the PTZ stays at each preset position for a period of time. During the stay time, it runs static target detection and time-domain fire source feature judgment algorithms. When the static target detection detects a suspected fire source target, it uses time-domain features for secondary judgment. When the time-domain judgment is a fire source, it outputs an alarm signal.

[0139] Figure 6(a)-6(d) shows the static analysis of fire source detection algorithm identification in different petrochemical zone monitoring scenarios. The false fire sources (actually high-heat petrochemical pipelines) were eliminated through dynamic feature judgment. They are marked with a box in the figure.

[0140] Figure 6(e) shows the flames caused by oil and gas leaks in two actual fire hazards, and Figure 6(f) shows the sparks generated by on-site electric welding in two actual fire hazards. Correct alarms were triggered by combining static feature analysis with the temporal characteristics of the fire source. Figure 6(g) is a visible light image of Figure 6(f). The white box marks the location where workers are performing electric welding. The visible light video shows that accurately identifying fires using visible light is very difficult for complex oil cracking pipelines, while the temperature sensing advantage of infrared thermal imaging is fully utilized.

[0141] Currently, commonly used fire source detection in petrochemical areas mainly relies on overheating for early warning. However, petrochemical pipelines have many bright targets, making temperature-based alarm devices prone to generating numerous false alarms, significantly reducing the user experience. Table 1 below compares the fire source detection device of this invention with commonly used domestic petrochemical area fire source detection devices.

[0142] Table 1. Comparison of the ignition source detection equipment of this invention with domestically commonly used ignition source detection equipment in petrochemical areas.

[0143] Equipment type Detection method False alarm removal method False alarm trigger frequency Mainstream domestic fire source detection equipment Over-temperature alarm False alarm zone setting Extremely high (multiple times a day) This article discusses fire source detection equipment. Deep learning video analysis Analysis of the temporal characteristics of fire source Once every few weeks

[0144] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for detecting ignition sources in petrochemical areas based on infrared thermal image analysis, characterized in that, This method utilizes deep neural networks to obtain suspected targets, performs temporal feature analysis on the obtained suspected targets and determines the location of the ignition source, and issues an alarm for dangerous ignition sources in the petrochemical zone based on the judgment results. Specifically, it includes the following steps: S1, by performing grayscale thresholding and morphological transformation on visible light images, a binary image sequence similar to infrared thermal imaging is generated, and combined with the original binary image sequence extracted from infrared thermal imaging to generate a binary fire source image training dataset. S2, by adding image prediction channels and adding an attention mechanism, uses a deep neural network to predict the binarized image and obtain the classification confidence and location information of suspected fire source targets; S3. Further analysis of the temporal characteristics of the target predicted by the deep learning network, and finally extraction of targets that meet the criteria of static target recognition and temporal dynamic feature determination, to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

2. The method for detecting fire sources in petrochemical areas based on infrared thermal image analysis according to claim 1, characterized in that, In step S1, generating a binary image sequence similar to infrared thermal imaging by performing grayscale thresholding and morphological transformation on the visible light image specifically includes the following steps: A training dataset of fire source images was constructed, including uncooled infrared fire source data acquisition sequences and visible light conversion dataset sequences. Infrared thermal imaging images and visible light images were processed and converted into binarized fire source target images and backgrounds. Through grayscale thresholding and morphological transformation, the binarized visible light conversion images and infrared fire source binarized images were made similar. The uncooled infrared fire source data acquisition sequence is obtained by acquiring real indoor and outdoor fire source scenes using an uncooled infrared imager, and a dataset is generated by extracting binarized fire source targets through threshold segmentation. The visible light conversion dataset sequence generates a dataset similar to an infrared fire source binarized image by downloading visible light images from a fire source visible light database and processing them through colorimetric analysis and morphological algorithms.

3. The method for detecting fire sources in petrochemical areas based on infrared thermal image analysis according to claim 2, characterized in that, The input uncooled infrared fire source data acquisition sequence is thresholded and converted into a binary fire source target image to obtain the binary image I(x,y), as shown in the following formula: In the above formula, IRaw is a 14-bit infrared Raw image, (x,y) are the column and row coordinates of the pixel, μ(IRaw) is the mean of the 14-bit infrared Raw image, and TH is the threshold for determining high-heat targets, which is set to 1000 by default.

4. The method for detecting ignition sources in petrochemical zones based on infrared thermal image analysis according to claim 1, characterized in that, In step S2, obtaining the classification confidence and location information of the suspected fire source target specifically includes: applying an attention mechanism in the spatial and channel dimensions, inputting the binarized image I(x,y) into a deep neural network, predicting the target at three feature scales of 13×13, 26×26 and 52×52, and directly obtaining the classification confidence prediction and location prediction of the suspected fire source target using a one-stage deep network.

5. The method for detecting ignition sources in petrochemical zones based on infrared thermal image analysis according to claim 4, characterized in that, The obtained suspected fire source targets are stored in the candidate fire source list. Connected component calculation is performed on the obtained binarized image I(x,y). The binarized image is scanned line by line using the region growing method. For each unlabeled non-zero pixel encountered, an unused label is assigned, and its neighborhood is then detected. The same operation is repeated until there are no non-zero pixels with the same label in the neighborhood of the passed pixel. Then the image line scanning continues until the entire image is scanned. The connected components with different labels correspond to different candidate regions of highlighted targets in the binarized image.

6. The method for detecting fire sources in petrochemical zones based on infrared thermal image analysis according to claim 5, characterized in that, Obtaining connected components with different labels corresponds to different candidate regions for highlighting targets in the binarized image, specifically including: The system compares the locations of targets in the highlighted target candidate area with those in the candidate fire source list predicted by the deep neural network. It calculates the Interchange of Units (IOU) between the targets in the highlighted target candidate area and those in the candidate fire source list, sets the default IOU matching threshold to 80%, and completes the matching of targets in the highlighted target candidate area and those in the candidate fire source list. For successfully matched candidate area targets, it analyzes the area variance change in the time domain dimension. When the variance change is greater than the time domain area judgment threshold of 15, it is judged as a real fire source and an alarm is output.

7. A petrochemical zone ignition source detection system based on infrared thermal image analysis, implementing the method for ignition source detection in petrochemical zones based on infrared thermal image analysis as described in any one of claims 1-6, characterized in that, The system includes: The binarized image sequence generation module (1) is used to generate a binarized image sequence similar to infrared thermal imaging by performing grayscale thresholding and morphological transformation on the visible light image, and to generate an infrared thermal imaging binarized image sequence by performing grayscale thresholding on the infrared thermal imaging image sequence. The two are combined to generate a fire source detection binarized image sequence. The deep neural network prediction module (2) is used to predict the binary image by adding image prediction channels and adding an attention mechanism, thereby obtaining the classification confidence and location information of the suspected fire source target; The fire source detection location acquisition module (3) is used to further analyze the temporal features of the target predicted by the deep learning network, and finally extract the target that meets the conditions to obtain the final fire source detection location of the irregular bright area in the oil pipeline.

8. The petrochemical zone ignition source detection system based on infrared thermal image analysis according to claim 7, characterized in that, The binarized image sequence generation module (1) includes: A fire source image training dataset construction module is used to construct uncooled infrared fire source data acquisition sequences and visible light conversion dataset sequences; The binarization image acquisition module is used to process infrared thermal imaging images and visible light images, converting them into binarized fire source target images and backgrounds. Through grayscale thresholding and morphological transformation, the binarized visible light images are made similar to the binarized infrared fire source images. The deep neural network prediction module (2) includes: The attention mechanism module is used to apply attention weights to features in both spatial and channel dimensions; A deep neural network is used to predict targets in three dimensions from an input binarized image, obtaining classification confidence prediction and location prediction for suspected fire sources; the targets include: large targets, medium targets, and small targets in the corresponding binarized image; The fire source detection location acquisition module (3) includes: The highlight target candidate region target acquisition module is used to store the obtained suspected fire source targets in the candidate fire source list, and to perform connected component calculation on the obtained binarized image to obtain the highlight target candidate region targets; The real fire source acquisition module is used to compare the target location in the highlighted target candidate area with the target location in the candidate fire source list, calculate the candidate area at the same location using the IOU threshold, and analyze the area variance change of the candidate area in the time domain dimension. When the variance change is greater than a certain threshold, it is determined to be a real fire source and an alarm is output.

9. A storage medium for receiving user input programs, characterized in that, The stored computer program causes the electronic device to execute the petrochemical zone fire source detection method based on infrared thermal image analysis as described in any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal is installed on the electronic device to provide a user input interface for implementing the petrochemical zone fire source detection method based on infrared thermal image analysis as described in any one of claims 1-6.

Images