Power plant internal fire detection method, system and equipment based on multi-source information and medium

Abstract

The invention discloses a power plant internal fire detection method, system and device based on multi-source information and a medium, and belongs to the technical field of computer vision, and the method comprises the steps: collecting multi-source image data, constructing a field picture data set, carrying out data enhancement based on the field picture data set, and expanding a power plant targeted data set; the collected real image and the composite image are preprocessed, a preprocessed data set is labeled and aligned, and fire positioning is carried out; and constructing a double-trunk feature extraction network to perform feature fusion, performing target detection by using the fused features, and obtaining a final fire detection result based on a detection result. The detection precision is improved, the algorithm detection speed is increased, and the algorithm multi-source input fusion process is simplified; fire morphological data based on the field is generated through the customized GAN, the mixed data training model greatly improves the detection and recognition precision of flames and smog in equipment and power plant areas, an accident area is quickly positioned, and quick response to the fire is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method, system, equipment, and medium for detecting internal fires in power plants based on multi-source information. Background Technology

[0002] Currently, fire detection technologies inside power plants are mainly divided into three categories: Single-sensor detection technologies: If only temperature sensors are used for detection, they can only capture temperature changes at local points and cannot achieve large-scale area monitoring. They are also susceptible to interference from ambient temperature fluctuations, resulting in a high false alarm rate. When only smoke sensors are used, alarms cannot be triggered in time when smoke has not spread to the sensor location in the early stages of a fire, resulting in detection lag. Data collection by deploying temperature sensors alone is difficult to cover complex large-scale power plant scenarios and lacks sensitivity for detecting early small fires.

[0003] Single image detection technology: If only visible light image detection is used, the clarity of visible light images will be greatly reduced in scenarios such as nighttime inspections of power plants, areas under strong light, and areas where cables and pipes are shaded, making it impossible to effectively identify signs of fire. When relying solely on infrared image detection, although temperature anomalies can be captured, there is a lack of scene texture information, making it difficult to distinguish between normal high-temperature operation of equipment and temperature anomalies in the early stages of a fire, and it is easy to misjudge normal machine heating as a fire hazard.

[0004] Simple multi-source fusion technology: Some technologies attempt to combine two types of detection data (such as temperature sensor and visible light image), but have not established a systematic information fusion and analysis mechanism. They only make judgments by simply superimposing data, without considering the spatiotemporal synchronization of different data sources. The time difference between the acquisition of sensor data and image data will lead to information deviation. Furthermore, they have not conducted customized analysis for the environmental differences in different areas of the power plant, resulting in unstable detection accuracy. In complex scenarios, they still cannot meet the high reliability fire detection requirements of power plants. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, this invention aims to solve the problems of limited coverage, detection lag, and susceptibility to environmental interference (strong light, shadow, night, temperature fluctuations) of existing single sensor or single image detection technologies, and to achieve fire monitoring of the entire area and all time of the power plant.

[0007] This addresses the issues of data asynchrony in time and space and lack of targeted analysis models in existing simple multi-source fusion technologies, thereby improving the detection accuracy after multi-source information fusion and reducing false alarm and false negative rates.

[0008] This addresses the issue that existing technologies cannot adapt to the diverse environments of different areas within a power plant, enabling customized fire detection for various scenarios and meeting the high-reliability detection requirements of complex power plant environments.

[0009] By adopting an end-to-end algorithm detection method, real-time computer vision fire smoke monitoring is achieved, improving fire detection efficiency and solving the problem of insufficient timeliness in current fire monitoring technology.

[0010] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for detecting internal fires in power plants based on multi-source information, comprising, Multi-source image data is collected to construct a live scene dataset. Data augmentation is performed on the live scene dataset to expand the power plant-specific dataset. The collected real and synthetic images are preprocessed, and the preprocessed dataset is labeled and aligned for fire location. A dual-backbone feature extraction network is constructed for feature fusion. The fused features are used for target detection, and the final fire detection result is obtained based on the detection results.

[0011] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the expanded power plant-specific dataset includes the collection of visible light or infrared images of different areas inside the power plant to construct a field image dataset. Data augmentation of power plant scenes was performed using generative adversarial networks based on a dataset of on-site footage; Generative networks are used to generate composite flame and smoke data that conforms to the power plant equipment environment, thus expanding the power plant's targeted dataset.

[0012] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the preprocessing includes preprocessing the acquired real images and synthetic images; Denoising and data augmentation are performed on real image data and synthetic images; The infrared image is filtered to remove noise and its size is standardized.

[0013] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the fire location includes: constructing a spatial alignment mechanism to correspond the camera footage with the specific equipment location, thereby achieving rapid location after a fire alarm. The dataset was manually labeled with flames and smoke. Based on the labeling results, the target was located using relative coordinates and its category was labeled. Establish a power plant spatial alignment mechanism to match camera footage with the shooting equipment, and locate the danger level and potential risk of spread hazards for different equipment and scenarios.

[0014] The beneficial effects of the preferred technical solution in the embodiments of the present invention are as follows: by constructing a spatial alignment mechanism and manual annotation, the camera image is correlated with the specific device location, thereby achieving rapid and accurate fire location, improving emergency response efficiency and reducing losses.

[0015] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the method for obtaining the final fire detection result includes: constructing a dual-backbone feature extraction network to extract features from visible light images and infrared images respectively; the dual-backbone feature extraction network includes a neck feature enhancement module and a head detection network module; the neck feature enhancement module fuses the features extracted by the two backbone networks in the channel dimension; and the head detection network detects the target. The preprocessed images are divided into training, validation, and test sets in a ratio of 7:2:1. The initial learning rate is set to 0.001. The Adam optimizer is used to train the constructed dual-backbone feature extraction network, and the CIoU loss function is used to optimize the network parameters until the model converges. The fused features are input into the trained network, which outputs the bounding box coordinates of the target. The camera RTSP push address is parsed, and the visible light image and infrared image of each frame are simultaneously input into the algorithm model to generate detection results. The detection bounding boxes of flames and smoke, as well as the detection category and confidence level, are output. Non-maximum suppression is applied to the bounding boxes to remove redundant boxes and boxes with inclusion relationships, resulting in the final detection result. The threshold for non-maximum suppression is set to 0.5. When the intersection-union ratio (IUU) of two bounding boxes is greater than 0.5, the bounding box with higher confidence is retained.

[0016] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the data enhancement includes employing two generative adversarial networks containing an encoder and a decoder, and incorporating a spatial transformation network and a feature pyramid network. The encoder Will , Two images are concatenated and fed into the network. The input size is 128×256 pixels. The encoder performs reflection filling on the reflection image, changing the input image size from 128×256 to 134×262. The main tasks are to extract image features and synthesize the images. The decoder will synthesize the image. Decomposed back into individual object images ( , ), and real object images ( , This is compared to increase the constraints on the encoder; The input to the spatial transformation network is the foreground image. and background image Processed into matrices of the same resolution size and merged along the channel dimension; The feature pyramid network inputs the synthesized image into a five-layer feature pyramid network and obtains five feature maps at different scales. The features are upsampled to 1 / 4 of the original synthesized image size and then stitched together along the channel dimension. The difference between the synthesized image generated by the encoder module and the real synthesized image is represented by the L2 loss function. ; The loss function of the decoder generator is The loss function of the spatial transformation module is The mask prediction network uses a binary cross-entropy function to calculate the loss function. The overall adversarial loss of the encoder can be obtained from the loss function of the conditional adversarial network. ; The overall algorithm loss function is: in, , , The weights are adjustable.

[0017] The beneficial effects of the preferred technical solution in the embodiments of the present invention are as follows: by generating realistic synthetic data of flames and smoke in power plant scenarios through generative adversarial networks, the diversity and relevance of the dataset are enhanced, and the generalization ability and detection accuracy of the model in real environments are improved.

[0018] As a preferred embodiment of the power plant internal fire detection method based on multi-source information described in this invention, the construction of the dual-backbone feature extraction network includes two backbone feature extraction networks with the same structure, as well as a neck feature enhancement network and a head detection network. The features at all levels extracted by the two backbone feature extraction networks are fused and enhanced by the neck network and then transmitted to the head detection network for detection, and the detection target type and location box are output. In YOLOv5, the backbone feature extraction network includes the Focus module, CBS module, ELAN module, and MPConv module; the neck feature enhancement network includes the SPPCSPC module and PANet structure; and the head detection network includes the RepConv module. When constructing the loss function, consider the target size contained in the detection box, the target's location information in the image, and the category to which the detected target belongs: in The CIOU function is used to represent the size loss function of the prediction box. This represents the target's position loss function. Use cross-entropy to create a category-specific loss function. These are the weight coefficients of the three loss functions; in, For the mask The number of detection frames, This represents the number of bounding boxes with a mask set to True. These are hyperparameters, belonging to the weight parameters, with values ​​ranging from 0.5 to 1. They determine the degree to which the model's attention mask is True. For negative sample location loss, For positive sample location loss, The cross-entropy loss is calculated as the confidence score of the current detection box containing the target compared to the true value. The bounding box regression loss.

[0019] The preferred technical solution in the embodiments of the present invention has the following beneficial effects: by using a dual backbone network to fuse visible light and infrared image features and combining it with an optimized loss function, the accuracy and robustness of fire target detection are significantly improved, and false alarms and false misses are reduced.

[0020] Another objective of this invention is to provide a power plant internal fire detection system based on multi-source information.

[0021] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a power plant internal fire detection system based on multi-source information, comprising: a data acquisition module, a data processing module, and a detection module; The data acquisition module collects multi-source image data, constructs a live scene dataset, performs data augmentation based on the live scene dataset, and expands the power plant's targeted dataset. The data processing module preprocesses the collected real and synthetic images, annotates and aligns the preprocessed dataset, and performs fire location. The detection module constructs a dual-backbone feature extraction network to perform feature fusion, uses the fused features to perform target detection, and obtains the final fire detection result based on the detection results.

[0022] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the power plant internal fire detection method based on multi-source information.

[0023] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the power plant internal fire detection method based on multi-source information.

[0024] The beneficial effects of this invention are as follows: Compared with existing technologies such as traditional single-sensor detection and single infrared image detection, this invention combines visible light images with infrared images to improve detection accuracy. Compared with existing multi-source fusion technologies, this invention is based on an end-to-end detection algorithm, which improves the algorithm's detection speed and simplifies the multi-source input fusion process. Compared with existing technologies such as traditional deep learning models that rely on real fire data, this invention generates fire morphology data based on actual conditions through customized GANs, and the mixed data training model greatly improves the accuracy of detecting and identifying flames and smoke in equipment and power plant areas. Utilizing the power plant spatial alignment mechanism, the algorithm locates cameras by numbering the video stream detected, thereby quickly locating the accident area and achieving rapid fire response. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating an internal fire detection method for power plants based on multi-source information, provided as an embodiment of the present invention.

[0027] Figure 2 This is a flowchart illustrating the data set creation process for a power plant internal fire detection method based on multi-source information, as provided in one embodiment of the present invention.

[0028] Figure 3 The present invention provides a flowchart of a fire detection method for power plant interiors based on multi-source information, which is an embodiment of the present invention.

[0029] Figure 4 The flowchart of the detection network algorithm of a power plant internal fire detection method based on multi-source information is provided as an embodiment of the present invention.

[0030] Figure 5This is a detection network structure diagram of a power plant internal fire detection method based on multi-source information, provided as an embodiment of the present invention. Detailed Implementation

[0031] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0032] Example 1, referring to Figures 1-5 This is one embodiment of the present invention, which provides a method for detecting internal fires in power plants based on multi-source information, including: S100: Collect multi-source image data, construct a live image dataset, perform data augmentation based on the live image dataset, and expand the power plant's targeted dataset. S200: Preprocess the collected real and synthetic images, label and align the preprocessed dataset, and locate the fire. S300. Construct a dual-backbone feature extraction network to perform feature fusion, use the fused features to perform target detection, and obtain the final fire detection result based on the detection result. It should be noted that existing power plant fire detection methods generally suffer from the following overall defects: they mainly rely on a single type of sensor (such as smoke or heat sensors), which can easily lead to blind spots in the vast and complex power plant environment; detection methods based on ordinary video images are easily affected by environmental interference such as changes in lighting and equipment steam, resulting in high false alarm and missed alarm rates; and they lack spatial correlation with the location of power plant equipment, making it impossible to quickly and accurately locate the fire source when an alarm is triggered.

[0033] Therefore, to address the aforementioned problems, through steps S100-S300, a dataset with power plant characteristics was constructed by acquiring and enhancing multi-source data, solving the problems of insufficient samples and scene adaptability; a spatial alignment mechanism was used to achieve accurate mapping of fire locations, solving the positioning problem in emergency response; and a dual-backbone feature fusion network effectively combined the advantages of visible light and infrared images, significantly improving the accuracy and robustness of fire detection in complex environments. Ultimately, a fast, accurate, and reliable intelligent fire detection system for power plants was constructed.

[0034] Example 2, refer to Figures 1-5 This is one embodiment of the present invention, which provides a method for detecting internal fires in power plants based on multi-source information, including: In this embodiment of the invention, step S100 involves collecting multi-source image data, constructing a live scene dataset, performing data augmentation based on the live scene dataset, and expanding the power plant-specific dataset. This includes the following steps S101-S102: S101. Collect visible light or infrared images of different areas inside the power plant and construct a live image dataset; In an embodiment of the present invention, S102, expanding the power plant-specific dataset, includes the following steps A1-A2: A1. To compensate for the lack of real power plant fire data, we use existing equipment and fire source and smoke datasets for various scenarios, as well as publicly available flame and smoke datasets, to build a dedicated GAN (Generative Adversarial Network) model for power plant scenarios. We input the original power plant environment images into the model and input images of flames and smoke into the model. A2. Generate composite flame and smoke data that conforms to the power plant equipment environment through a generative network, thereby expanding the power plant's targeted dataset.

[0035] In an optional embodiment, the expanded power plant-specific dataset in S102 can be generated by collecting existing power plant environmental images and a small number of real fire images, including flames and smoke; applying traditional image enhancement techniques, such as image rotation, scaling, flipping, brightness adjustment, contrast changes, and adding random noise, to generate multiple enhanced versions; and merging these enhanced images with the original dataset to form an expanded power plant-specific dataset for model training.

[0036] In another alternative embodiment, the extended power plant-specific dataset in S102 can also be an extended dataset based on image overlay synthesis. The method involves acquiring isolated images of flames and smoke from a public dataset and collecting background images of the power plant's interior environment. Image processing tools are used to overlay the flame and smoke images as layers onto the power plant background images. The position, size, transparency, and blending mode are adjusted to simulate the real scene. The synthesized images are post-processed (such as blurring edges or color correction) to enhance realism, and these synthesized data are added to the dataset.

[0037] In this embodiment of the invention, step S200 involves preprocessing the acquired real and synthetic images, labeling and aligning the preprocessed dataset, and performing fire location, including the following steps S201-S203: In an embodiment of the present invention, S201, preprocessing the acquired image includes the following steps B1-B2: B1. Denoise real image data and GAN-synthesized data by 5×5 Gaussian filtering, and perform data augmentation by mirroring, rotation and other methods. B2. The infrared image is denoised by a 3×3 median filter and then scaled to 1920×1080 pixels to unify the size of the visible light image and the infrared image.

[0038] In an optional embodiment, the preprocessing in S201 can be as follows: for real images and GAN-synthesized data, mean filtering (5×5 kernels) is used for noise reduction, replacing the Gaussian filtering of the original scheme; data augmentation uses translation (randomly moving image pixels) and color dithering (adjusting brightness and contrast). For infrared images, mean filtering (3×3 kernels) is used for noise reduction; similarly, the infrared images are scaled to 1920×1080 pixels to unify the size of visible light images and infrared images; however, mean filtering blurs image edge details when removing noise, affecting the accuracy of subsequent fire location; color jitter introduces unrealistic color changes, reducing the model's ability to recognize real flames and smoke.

[0039] In another optional embodiment, the preprocessing in S201 may also include denoising using median filtering (5×5 kernels) for both real images and GAN-synthesized data; and data augmentation using random cropping (cropping local regions of the image) and horizontal flipping. For infrared images, Gaussian filtering (3×3 kernels) is used for noise reduction; similarly, the infrared images are scaled to 1920×1080 pixels to unify the size of the visible light images and the infrared images; however, median filtering is less effective at removing Gaussian noise, resulting in residual noise; random cropping loses key fire features (such as flame edges) in the images, affecting the model training effect.

[0040] Furthermore, the Generative Adversarial Network (GAN) uses images of equipment scenes in a power plant without fire as base images and fire sample data as training samples. The generator uses the Adam optimizer (learning rate 0.0002, β1=0.5), and the discriminator uses the Adam optimizer (learning rate 0.0004, β1=0.5). The loss function uses a combination of adversarial loss and perceptual loss to ensure the degree of integration between the generated flames and smoke and the power plant scene. The training iteration is 200 rounds, with a batch size of 16 per round. Training stops when the discriminator's recognition accuracy for real and synthetic images approaches 50%.

[0041] The data augmentation algorithm employs two generative adversarial networks, including an encoder and a decoder, and incorporates a spatial transformation network and a feature pyramid network.

[0042] The encoder: encoder Will , Two images are concatenated and fed into the network. The input size is 128×256 pixels. The encoder first performs reflection filling on the reflection image, changing the input image size from 128×256 to 134×262. The subsequent main body consists of two parts. The first part's main function is to extract image features, mainly composed of three convolutional blocks, each consisting of a convolutional layer, a normalization layer, and an activation layer. The second part's main function is to synthesize the image, specifically composed of three residual blocks and three upsampling layers. The residual blocks are used to avoid gradient explosion. Finally, an activation function is used to generate RGB pixel values.

[0043] The decoder: The decoder is used to synthesize the image Decomposed back into individual object images ( , ), and real object images ( , This is compared to increase the constraints on the encoder. The encoder generator is then used. The generated synthetic image is input into the decoder. middle, study The mapping, the discriminator of the decoder a real single object Compared with the single object obtained from decomposition The corresponding feature vectors are obtained by inputting them into the high-level semantic feature space respectively. ( ), ( ( ))and( ( ), ( ( The true or false samples are distinguished by calculating the differences between the samples in the feature space.

[0044] The spatial transformation network The input to the spatial transformation network is the foreground image. and background image The matrix is ​​processed to the same resolution size and merged along the channel dimension. The spatial transformation module, during training, utilizes the spatial relationships encoded in the synthesized images. These synthesized images may contain information such as the relative positions and rotation angles of objects within the image. The spatial transformation module learns this information to perform relative transformations. The spatial transformation network consists of three parts: a localization network, a parameterized sampling network, and an image sampling network. Finally, the input data is interpolated using the sampling grid and corresponding parameters to obtain the affine transformed output data. , The change data here is input into the encoder's generator. middle.

[0045] The feature pyramid network: The spatial transformation network is trained before the generator to establish the size relationship between the two objects to be synthesized. To enable the discriminator to better handle situations where the foreground and background images differ significantly in size and to better extract features from high-resolution images, a feature pyramid network is added to the decoder, allowing it to better extract multi-scale features from the synthesized image. The synthesized image is input into a five-layer feature pyramid network, resulting in five feature maps at different scales. These features are then upsampled to one-quarter the size of the original synthesized image and concatenated along the channel dimension.

[0046] Mask prediction: The main function of the mask prediction module is to maintain the reasonable positioning of the foreground and background images during image synthesis in the entire image synthesis algorithm. The mask prediction module takes the synthesized image from the encoder module as input. Output object and object The prediction mask matrices in the synthesized image respectively .matrix Each element in the image is a composite image. The corresponding pixel in the middle belongs to the object The probability, matrix Each element in the image is a composite image. The corresponding pixel in the middle belongs to the object The probability. The algorithm uses Predicted object and object By considering the possible pixels occupied, two prediction matrices are obtained, which will be... The pixel value is set to 1, and the rest are 0. Similarly, by fusing these two matrices, a prediction mask matrix is ​​obtained. Simultaneously utilizing the prediction matrix and the original separated image ( , A composite image is obtained by multiplying the pixels to obtain a composite image directly generated from pixel selection. This is achieved using a prediction mask matrix. The mask is compared with the original input image's mask to constrain the network's training. The discriminator of the mask prediction network distinguishes between synthetic images generated by pixel selection and real synthetic images. The difference between them is used to train the generative adversarial network corresponding to the entire mask prediction module.

[0047] The difference between the synthesized image generated by the encoder module and the real synthesized image is represented by the L2 loss function: The formula above expresses the expected value, used to measure the average difference between the synthesized image and the real synthesized image. Represents a real synthetic image. Indicates that when a composite image is generated 0 indicates that the two images have the same features. The method used in this invention minimizes... Enhance the generator's synthesis capabilities.

[0048] The loss function of the decoder generator is as follows: For the decoder to synthesized image The feature vector obtained after decoding; the loss function of the spatial transformation module is, and STN is the spatial transformation network: For the spatially transformed target image pair, For spatial transformation networks of the original image The process of performing spatial transformations (rotation, translation, scaling); Mask prediction network The output is the predicted class of each pixel in the generated synthetic image. The mask prediction network is penalized using the real mask, hence the binary cross-entropy function is employed. The loss function formula is as follows: Where N is the total number of categories, It is the true label of the i-th pixel. It is the predicted value for a pixel, which is a probability value between 0 and 1.

[0049] The overall adversarial loss of the encoder can be obtained from the loss function of the conditional adversarial network. For the encoder input features, Features of the encoder output: in, For real samples Expectations Let x be the expectation of the generated sample (x,y); The overall algorithm loss function is: in, , , The weights are adjustable.

[0050] S202. Construct a spatial alignment mechanism to correspond the camera's captured images with the specific device locations, enabling rapid location tracking after a fire alarm. In an embodiment of the present invention, the manual annotation includes the following steps C1-C2: C1. Perform detailed manual annotation of flames and smoke on the dataset; C2. The target is located using the relative coordinates of its upper left and lower right corners, and its category is labeled accordingly. In an optional embodiment, manual annotation in S202 can be performed by using center point coordinates and width and height to perform refined manual annotation of flames and smoke in the dataset. Each target area is manually marked using visualization tools. The annotation uses the relative coordinates of the target center point, that is, the normalized x and y positions of the center point in the image, as well as the relative width and height of the target to locate it and label its category.

[0051] In another optional embodiment, manual annotation in S202 can also be performed by using polygon vertex coordinates to perform refined manual annotation of flames and smoke in the dataset. The target shape is accurately covered by drawing polygon outlines. The annotation uses the relative coordinates of multiple vertices of the polygon, that is, the normalized x and y position sequence of each vertex in the image for positioning and labeling its category.

[0052] In an embodiment of the present invention, the rapid positioning includes the following steps D1-D2: D1. Construct a power plant spatial alignment mechanism, using a one-to-one correspondence between camera numbers and room location numbers, and associate the camera images with specific shooting devices to achieve rapid location of danger after an alarm is triggered. D2. Providing rapid response based on the degree of danger of different equipment and scenarios, as well as the risk of diffusion hazards (such as the presence of toxic gases).

[0053] In an optional embodiment, the rapid positioning in S202 can be based on coordinate mapping positioning using a Geographic Information System (GIS). A GIS platform is deployed within the power plant to precisely map the location coordinates of each camera to the location coordinates of the equipment. When the cameras are installed, their field of view and azimuth angle are recorded and stored in the GIS database. When a fire detection algorithm triggers an alarm, the system retrieves the location and field of view parameters from the GIS based on the alarm camera's number. Combining this with the pixel location of the fire in the image, the system calculates the actual coordinates of the fire-affected equipment using a target detection box and geometric projection transformation, displaying the results on the power plant map for rapid positioning. Simultaneously, based on the equipment coordinates and their associated hazard level information, the system automatically initiates the corresponding emergency response process. However, in complex indoor environments, multipath errors can lead to positioning deviations.

[0054] In another optional embodiment, the rapid positioning in S202 can also be based on short-range positioning using Bluetooth beacons. Low-power Bluetooth beacons are installed on key equipment in the power plant, with each beacon broadcasting a unique identifier. Bluetooth receivers are deployed near the cameras and bound to the camera numbers. When the fire detection algorithm alarms, the system reads the signal from the Bluetooth receiver associated with the alarm camera, estimates the distance between the fire-affected equipment and the camera using the received beacon signal strength (RSSI), and triangulates the specific location of the equipment based on the camera's field of view. The system retrieves equipment information (such as the degree of danger) from the database based on the equipment identifier and triggers corresponding alarm and response measures, such as evacuation route planning or isolation operations. However, this method suffers from poor positioning stability.

[0055] In an embodiment of the present invention, S300 involves constructing a dual-backbone feature extraction network for feature fusion, using the fused features for target detection, and obtaining the final fire detection result based on the detection result, including the following steps S301-S302: S301. Construct a dual-backbone feature extraction network to extract features from visible light images and infrared images respectively; Both the visible light backbone feature extraction module and the infrared backbone feature extraction module contain a lightweight attention module CBAM (Convolutional Neural Network Architecture Attention Module). The module improves the feature extraction capability of the algorithm by combining channel attention mechanism and spatial attention mechanism. The input dimension of the visible light backbone feature extraction module is 640×640×3, and the input dimension of the infrared backbone feature extraction module is 640×640×1.

[0056] Specifically, the dual-backbone feature extraction network includes a neck feature enhancement module and a head detection network module. The neck feature enhancement module fuses the features extracted by the two backbone networks along the channel dimension, while the head detection network detects the target. The preprocessed images are divided into training, validation, and test sets in a ratio of 7:2:1. The initial learning rate is set to 0.001. The Adam optimizer is used to train the constructed dual-backbone feature extraction network, and the CIoU loss function is used to optimize the network parameters until the model converges. S302. Input the fused features into the trained network, output the bounding box coordinates of the target, parse the RTSP push address of the camera, input the visible light image and infrared image of each frame into the algorithm model at the same time, generate the detection results, and output the detection bounding box of the flame and smoke, as well as the detection category and confidence level. Non-maximum suppression is applied to the bounding boxes to remove redundant boxes and boxes with inclusion relationships, resulting in the final detection result. The threshold for non-maximum suppression is set to 0.5. When the intersection-union ratio (IUU) of two bounding boxes is greater than 0.5, the bounding box with higher confidence is retained.

[0057] Specifically, the multi-source fire detection algorithm comprises two structurally identical backbone feature extraction networks, a neck feature enhancement network, and a head detection network. The features extracted from the two backbones are fused and enhanced by the neck network, and then the final features are passed to the head detection network for detection, outputting the detected target type and location bounding box.

[0058] The backbone feature extraction network: (1) Focus Module: In YOLOv5, the Focus module was initially proposed and implemented. Its idea is to use interval pixel slicing operations on each feature map to transform the input of size (B, C, H, W) into (B, 4) pixels. Feature maps of C, H / 2, W / 2. According to the official explanation, this module reduces the overall computational cost of the network to some extent compared to using multiple convolutional operations. Assume the output size of the image sample is (1, 3, 128, 128). The output size separated by the Focus module is (1, 12, 64, 64), which is the same as performing one downsampling and one channel boosting operation.

[0059] (2) CBS Module: The initial network model consists of a continuous stack of CBS modules. The CBS module is the most basic convolutional block in the network, composed of convolutional layers, batch normalized layers, and the SiLU activation function. Compared to common activation functions such as ReLU, the SiLU activation function can more effectively capture linear or non-linear features. Its use in deep learning can effectively improve network performance; however, due to the inclusion of the sigmoid function in its formula, it has a high computational cost. The formulas for the SiLU and sigmoid activation functions are shown below: in, This represents the SiLU activation function. This represents the variable input to the activation function. This represents the sigmoid activation function.

[0060] (3) ELAN Module: The ELAN module is particularly crucial in the backbone feature extraction network. The ELAN module consists of multiple CBSs, such as... Figure 5 As shown, the ELAN module doubles the number of channels in the feature map through a series of operations. Figure 5In the diagram, C represents the number of channels in the feature map. Assuming the input has 128 channels, after passing through the first and second CBS modules from the left, two 64-channel feature maps are obtained. The feature map from the second CBS module is then stacked through two consecutive CBS modules to obtain the third feature map, which also has 64 channels. Similarly, the rightmost operation will also produce a 64-channel feature map. Concatenating these four feature maps yields a 256-channel feature map, effectively doubling the channels of the initial input features. Finally, a CBS module is used for feature fusion to obtain the output.

[0061] (4) MPConv Module: This module consists of a combination of max pooling and CBS modules. The left and right processes still use the Concat operation for channel merging. In the diagram, C represents the number of channels in the feature map. The MPConv module differs somewhat from the one in the head detection network. Figure 5 The diagram shows the structure in the backbone network, where the number of input and output channels for each module is the same. However, in the head network, the number of output channels for this module is twice the number of input channels, so this module also serves to expand the number of channels.

[0062] The neck feature enhancement network: SPPCSPC module: The SPPCSPC module is essentially an operation that uses pooling kernels of various sizes for pooling and stacking, inheriting the SPP structure used in YOLOv4. For example... Figure 5 As shown, the max-pooling kernels are 5, 9, and 13, which are then concatenated with the feature maps that have not undergone pooling. This operation can increase the receptive field in deep networks and expand the information learned by the network. The SPPC module adds an extra large residual edge to the original SPP module, and merges the original input of the module directly with the output of the SPP structure through a CBS convolutional module. This large-span skip connection operation can effectively prevent information loss in deep networks.

[0063] PANet Structure: The founding team of the PANet network structure improved upon the shortcomings of the feature pyramid structure by adding a bottom-up fusion module after the top-down pyramid structure, which allows information from the lower layers to be better transmitted to the higher layers.

[0064] The head detection network: The RepConv module differs from previous YOLO family designs in its detection head section. YOLOv7 adds a RepConv module before each of the three detection heads. This module's unique feature lies in its distinctive design, combining 3×3 and 1×1 convolutions within the same convolutional layer to achieve model reparameterization. The complex convolutional paths are omitted during inference, with only the 1×1 convolutions participating in prediction. This design approach achieves higher accuracy without increasing the number of parameters. The RepConv design disrupts the traditional residual skip connections and deep network concatenation patterns, demonstrating significant advantages in VGG networks. However, its application in networks like ResNet or DenseNet weakens the network's learning ability and leads to a decrease in accuracy. This invention also fully considers this characteristic, not directly using previously designed modules, but adding a feature path that does not involve convolutions in addition to the complex and simple convolutions.

[0065] To ensure speed in the detection task, the algorithm chooses to directly output predictions from multi-scale feature maps. When constructing the loss function, the following three aspects are crucial for network learning: First, the size of the target contained in the detection box; by comparing it with the actual labeled box, a higher degree of overlap indicates a better prediction. Second, the target's position information in the image; if all target locations are successfully detected in the prediction, zero false positives and zero false negatives will be calculated. Finally, the category of the detected target is determined to ensure consistency with the true label. in The CIOU function is used to represent the size loss function of the prediction box. This represents the target's position loss function. Use cross-entropy to create a category-specific loss function. These are the weight coefficients of the three loss functions; in general, to make the detection network pay more attention to detection accuracy and location information, the three weight hyperparameters can be set to 0.25, 0.5, and 0.25.

[0066] in, For the mask The number of detection frames, This represents the number of bounding boxes with a mask set to True. These are hyperparameters, belonging to the weight parameters, with values ​​ranging from 0.5 to 1. They determine the degree to which the model's attention mask is True. For negative sample location loss, For positive sample location loss, The cross-entropy loss is calculated as the confidence score of the current detection box containing the target compared to the true value. The bounding box regression loss.

[0067] To accelerate the loss function calculation process, the algorithm still utilizes a mask to design the loss function. In this invention, the model's input size is set to 640×640, which will produce outputs with three feature sizes: 80×80, 40×40, and 20×20.

[0068] Using an 80x80 output feature as an example, a Boolean matrix mask of the same size will be generated. Cells appearing within the bounding boxes will be marked as True, and those not appearing will be marked as False. For example... and As shown in the calculation formula, the mask matrix with a boolean value of True is calculated based on the anchor. and ,in This represents the statistics for boolean values ​​of True. Calculate the target category loss function. according to The calculation formula, when the mask is True, is based on... The calculation formula, when it is False, is based on The formula is used for calculation. In the calculation formula, there exists a hyperparameter. , is a weight parameter with a value between 0.5 and 1, which determines the degree to which the model focuses on the mask as True.

[0069] Example 3 is an embodiment of the present invention. The above is an illustrative scheme of a power plant internal fire detection method based on multi-source information. It should be noted that the technical solution of a power plant internal fire detection system based on multi-source information and the technical solution of the power plant internal fire detection method based on multi-source information described above belong to the same concept. For details not described in detail in the technical solution of the power plant internal fire detection system based on multi-source information in this embodiment, please refer to the description of the technical solution of the power plant internal fire detection method based on multi-source information described above.

[0070] This embodiment provides a power plant internal fire detection system based on multi-source information, including: a data acquisition module, a data processing module, and a detection module; The data acquisition module collects multi-source image data, constructs a live scene dataset, performs data augmentation based on the live scene dataset, and expands the power plant's targeted dataset. The data processing module preprocesses the collected real and synthetic images, annotates and aligns the preprocessed dataset, and performs fire location. The detection module constructs a dual-backbone feature extraction network to perform feature fusion, uses the fused features to perform target detection, and obtains the final fire detection result based on the detection results.

[0071] This embodiment also provides an electronic device applicable to a power plant internal fire detection method based on multi-source information, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the power plant internal fire detection method based on multi-source information as proposed in the above embodiment.

[0072] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements a power plant internal fire detection method based on multi-source information as proposed in the above embodiment.

[0073] The storage medium proposed in this embodiment belongs to the same inventive concept as the method for detecting internal fires in power plants based on multi-source information proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0074] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0075] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting internal fires in power plants based on multi-source information, characterized in that: include, Collect multi-source image data, construct a live image dataset, perform data augmentation based on the live image dataset, and expand the power plant's targeted dataset. The collected real and synthetic images are preprocessed, and the preprocessed dataset is labeled and aligned for fire location. A dual-backbone feature extraction network is constructed for feature fusion. The fused features are then used for target detection, and the final fire detection result is obtained based on the detection results.

2. The method for detecting internal fires in power plants based on multi-source information as described in claim 1, characterized in that: The targeted dataset for the expanded power plant includes the collection of visible light or infrared images of different areas inside the power plant to construct a live image dataset. Data augmentation of power plant scenes was performed using generative adversarial networks based on a dataset of on-site footage; Generative networks are used to generate composite flame and smoke data that conforms to the power plant equipment environment, thereby expanding the power plant's targeted dataset.

3. The method for detecting internal fires in power plants based on multi-source information as described in claim 2, characterized in that: The preprocessing includes preprocessing the acquired real images and synthetic images; Denoising and data augmentation are performed on real image data and synthetic images; The infrared image is filtered to remove noise and its size is standardized.

4. The method for detecting internal fires in power plants based on multi-source information as described in claim 3, characterized in that: The fire location process includes constructing a spatial alignment mechanism to correspond the camera footage with the specific device location, thereby enabling rapid location after a fire alarm is triggered. The dataset is manually labeled with flames and smoke. Based on the labeling results, the target is located using relative coordinates and its category is labeled. Establish a power plant spatial alignment mechanism to match camera footage with the shooting equipment, and locate the danger level and potential risk of spread hazards for different equipment and scenarios.

5. The method for detecting internal fires in power plants based on multi-source information as described in claim 4, characterized in that: The final fire detection result is obtained by constructing a dual-backbone feature extraction network to extract features from visible light images and infrared images respectively. The dual-backbone feature extraction network includes a neck feature enhancement module and a head detection network module. In the neck feature enhancement module, the features extracted by the two backbone networks are fused in the channel dimension, and the head detection network detects the target. The preprocessed images are divided into training, validation, and test sets in a ratio of 7:2:

1. The initial learning rate is set to 0.

001. The Adam optimizer is used to train the constructed dual-backbone feature extraction network, and the CIoU loss function is used to optimize the network parameters until the model converges. The fused features are input into the trained network, which outputs the bounding box coordinates of the target. The camera RTSP push address is parsed, and the visible light image and infrared image of each frame are simultaneously input into the algorithm model to generate detection results. The detection bounding boxes of flames and smoke, as well as the detection category and confidence level, are output. Non-maximum suppression is applied to the bounding boxes to remove redundant boxes and boxes with inclusion relationships, resulting in the final detection result. The threshold for non-maximum suppression is set to 0.

5. When the intersection-union ratio (IUU) of two bounding boxes is greater than 0.5, the bounding box with higher confidence is retained.

6. The method for detecting internal fires in power plants based on multi-source information as described in claim 5, characterized in that: The data augmentation includes employing two generative adversarial networks containing an encoder and a decoder, and incorporating a spatial transformation network and a feature pyramid network; The encoder Will , Two images are concatenated and fed into the network. The input size is 128×256 pixels. The encoder performs reflection filling on the reflection image, changing the input image size from 128×256 to 134×262. The main tasks are to extract image features and synthesize the images. The decoder will synthesize the image. Decomposed back into individual object images ( , ), and real object images ( , This is compared to increase the constraints on the encoder; The input to the spatial transformation network is the foreground image. and background image Processed into matrices of the same resolution size and merged along the channel dimension; The feature pyramid network inputs the synthesized image into a five-layer feature pyramid network and obtains five feature maps at different scales. The features are upsampled to 1 / 4 of the original synthesized image size and then stitched together along the channel dimension. The difference between the synthesized image generated by the encoder module and the real synthesized image is represented by the L2 loss function. ; The loss function of the decoder generator is The loss function of the spatial transformation module is The mask prediction network uses a binary cross-entropy function to calculate the loss function. The overall adversarial loss of the encoder can be obtained from the loss function of the conditional adversarial network. ; The overall algorithm loss function is: in, , , The weights are adjustable.

7. The method for detecting internal fires in power plants based on multi-source information as described in claim 6, characterized in that: The construction of the dual-backbone feature extraction network includes two backbone feature extraction networks with the same structure, as well as a neck feature enhancement network and a head detection network. The features at all levels extracted by the two backbone feature extraction networks are fused and enhanced by the neck network and then passed to the head detection network for detection, outputting the type and location box of the detected target. In YOLOv5, the backbone feature extraction network includes the Focus module, CBS module, ELAN module, and MPConv module; the neck feature enhancement network includes the SPPCSPC module and PANet structure; and the head detection network includes the RepConv module. When constructing the loss function, consider the target size contained in the detection box, the target's location information in the image, and the category to which the detected target belongs: in The CIOU function is used to represent the size loss function of the prediction box. This represents the target's position loss function. Use cross-entropy to create a category-specific loss function. These are the weight coefficients of the three loss functions; in, For the mask The number of detection frames, This represents the number of bounding boxes with a mask set to True. These are hyperparameters, belonging to the weight parameters, with values ​​ranging from 0.5 to 1. They determine the degree to which the model's attention mask is True. For negative sample location loss, For positive sample location loss, The cross-entropy loss is calculated as the confidence score of the current detection box containing the target compared to the true value. The bounding box regression loss.

8. A power plant internal fire detection system based on multi-source information, employing the power plant internal fire detection method based on multi-source information as described in any one of claims 1 to 7, characterized in that, include: Data acquisition module, data processing module, detection module; The data acquisition module collects multi-source image data, constructs a live scene dataset, performs data augmentation based on the live scene dataset, and expands the power plant's targeted dataset. The data processing module preprocesses the collected real and synthetic images, annotates and aligns the preprocessed dataset, and performs fire location. The detection module constructs a dual-backbone feature extraction network to perform feature fusion, uses the fused features to perform target detection, and obtains the final fire detection result based on the detection results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the power plant internal fire detection method based on multi-source information as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the power plant internal fire detection method based on multi-source information as described in any one of claims 1 to 7.

Images