Transmission line wildfire monitoring method, device and storage medium based on semantic segmentation

Abstract

The present invention discloses a method, device, and storage medium for power transmission line wildfire monitoring based on semantic segmentation. The method comprises obtaining an image to be detected and inputting it into a pretrained semantic segmentation model; the semantic segmentation model outputs a smoke image detection result based on the image to be detected; and when a smoke image is detected, an alarm signal is output. This method addresses the existing problems of the inability to accurately identify wildfire smoke, low monitoring timeliness, and poor accuracy. By innovatively utilizing the MCCL self-attention module and improving existing semantic segmentation models, the method improves the accuracy of segmentation boundaries, enhances the expression of valid features, and suppresses invalid features. This method alleviates the issues of insignificance and inter-class similarity in smoke targets, addresses information loss and dissipation, and accurately segments wildfire smoke, improving semantic segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to the fields of power transmission line wildfire monitoring, computer vision and image segmentation technology, and in particular to a power transmission line wildfire monitoring method, device and storage medium based on semantic segmentation. Background Art

[0002] With the increasing electrification of society today, people's demands for the reliability and quality of power supply are becoming increasingly stringent. As the current transmission network of my country's power grid, the safe and stable operation of overhead transmission lines is closely related to the safety of the power grid. Long-distance and large-scale high-voltage overhead transmission lines often need to cross mountainous areas. At the same time, due to the increasing demand for environmentally friendly electricity, the corridors of low-voltage distribution overhead transmission lines are often located far away from residential areas. Furthermore, due to the increased incidence of extreme weather in recent years and the ancestral worship during festivals such as Qingming Festival, wildfires have frequently occurred in my country, causing transmission line tripping and impacting grid equipment and power supply.

[0003] Therefore, specialized monitoring methods and devices are being developed specifically for wildfires along power transmission lines. These can detect fire sources in the early smoke phase of a wildfire, helping repair personnel eliminate potential hazards before the fire impacts equipment and the power grid, ensuring safe operation of the lines and reliable power supply. However, traditional wildfire smoke detection methods cannot accurately identify wildfire smoke, cannot detect it in real time, and have inaccurate smoke segmentation boundaries. They cannot effectively handle inconspicuous objects in the image background, and similar-looking smoke may be classified as wildfire smoke. Invalid features or noise are prominent, while valid features are not obvious. Furthermore, information loss and attrition during information transmission can easily lead to vanishing or exploding gradients and prevent the training of very deep networks. Furthermore, traditional wildfire monitoring devices lack an alarm module and generally only send alarm signals. They cannot sound an alarm near the fire source to alert personnel near the transmission line of a fire, hindering personnel from locating the fire source and extinguishing it promptly. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device and storage medium for power line wildfire monitoring based on semantic segmentation, so as to solve the problems in the prior art of inability to accurately identify wildfire smoke and low monitoring timeliness and accuracy.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for monitoring wildfires on power transmission lines based on semantic segmentation, comprising:

[0007] Obtain the image to be detected and input it into the pre-trained semantic segmentation model;

[0008] The semantic segmentation model outputs a smoke image detection result based on the image to be detected; when a smoke image is detected, an alarm signal is output;

[0009] The semantic segmentation model is trained by the following steps:

[0010] Obtain smoke images and construct a semantic segmentation dataset;

[0011] Preprocess the semantic segmentation dataset;

[0012] Build a semantic segmentation model using the encoder-decoder as the basic structure;

[0013] Use the preprocessed semantic segmentation dataset to train the built semantic segmentation model;

[0014] Set network training parameters and use the loss function to train the built semantic segmentation model.

[0015] In combination with the first aspect, further, obtaining the smoke image and constructing the semantic segmentation dataset includes:

[0016] Obtain surveillance video containing smoke through surveillance cameras;

[0017] In surveillance videos containing smoke, a smoke image is captured every preset number of frames as a sample, and the smoke targets are manually labeled at the pixel level;

[0018] Add synthetic smoke data and difficult samples to form the final semantic segmentation dataset.

[0019] In combination with the first aspect, further, the preprocessing of the semantic segmentation dataset includes data enhancement of the images in the semantic segmentation dataset, and the data enhancement of the images in the semantic segmentation dataset includes horizontal flipping, vertical flipping, rotation, translation, scaling, cropping, erasing, adjusting contrast, adjusting saturation and color jittering operations on the images in the semantic segmentation dataset.

[0020] In combination with the first aspect, further, the semantic segmentation model built by training using the preprocessed semantic segmentation dataset includes:

[0021] In the encoding stage, the output results of the skeleton network in the encoder are subjected to feature extraction and enhanced feature expression. The features include deep features and shallow features. In the decoding stage, the deep features and shallow features from the encoding stage are combined for fusion extraction and output to the segmentation result. Convolution processing and upsampling operations are used to make the final segmentation result size the same as the original input smoke image.

[0022] In combination with the first aspect, further, the skeleton network is an improved ResNet50 deep residual network, and the ResNet50 deep residual network includes stage one, stage two, stage three and stage four; the improvement of the ResNet50 deep residual network includes: replacing the ordinary convolution in stage three and stage four with void convolution.

[0023] In combination with the first aspect, further, performing feature extraction on the output result of the skeleton network in the encoder during the encoding stage includes:

[0024] The local features output by the first stage are fed into the decoder, and the local features output by the fourth stage are fed into the MCCL-self-attention module. The local features output by the second stage, the local features output by the third stage after convolution processing, and the local features output by the MCCL-self-attention module are merged, and the obtained results are further convolved to output global features and fed into the decoder.

[0025] The MCCL-self-attention module includes an MCCL module, a CBAM module and a self-attention module; the MCCL module includes a DPPM module;

[0026] The processing process after the local features output by the fourth stage are sent to the MCCL-self-attention module includes: sending the local features output by the fourth stage to the MCCL module, subjecting the local features to convolution processing, performing feature subtraction and fusion on the local features after the convolution processing, outputting them from the MCCL module, and sending them to the DPPM module for processing, re-fusing the local features output by the DPPM module with the fused local features output by the MCCL module, and sending them to the CBAM module and the self-attention module for processing respectively;

[0027] The CBAM module includes an SA module and a CA module. The processing process of the CBAM module includes:

[0028] The SA module and the CA module assign weights to the local features output by the fourth stage respectively, multiply the local features output by the fourth stage by the assigned weights respectively to obtain new local features, add the obtained new local features and output them to the self-attention module for processing;

[0029] The processing process of the self-attention module includes:

[0030] The local features input in stage four are convolved and dimensionally transformed to obtain local feature A. The local features output by the MCCL module are convolved and dimensionally transformed to obtain local feature B. Local feature A is multiplied by local feature B to output local feature C. The local features output by the CBAM module are convolved and dimensionally transformed to obtain local feature D. Local feature C is multiplied by local feature D. The result is dimensionally transformed and convolved, and then added to local feature A to convolve the obtained local feature.

[0031] In combination with the first aspect, further, setting network training parameters and using the loss function to train the constructed semantic segmentation model includes:

[0032] Use the images in the preprocessed semantic segmentation image dataset as input images and unify the size of the input images;

[0033] The semantic segmentation network model is trained using a stochastic gradient descent algorithm with momentum, wherein the weight initialization method in the semantic segmentation network model is Kaiming and the activation function is ReLu;

[0034] The loss function is used to predict the difference between the prediction and the actual data, and the loss function is DiceLoss+FocalLoss function;

[0035] The calculation formula of the loss function is:

[0036]

[0037] FocalLoss=-α t (1-p t ) γ log(P t )

[0038]

[0039] In the formula, DiceLoss is the similarity loss function, FocalLoss is the boundary loss function, TotalLoss is the total loss function, y i represents the label value of pixel i, Represents the predicted value of pixel i, N is the total number of pixels, p t Represents the probability of the model predicting a certain category; α t It is the value used to balance the number of positive and negative samples, and γ is the value used to adjust the imbalance problem of difficult and easy samples.

[0040] In a second aspect, the present invention further provides a power transmission line wildfire monitoring device based on semantic segmentation, comprising:

[0041] The image acquisition module is used to obtain the image to be detected and input it into the pre-trained semantic segmentation model;

[0042] An image detection module is configured to output a smoke image detection result based on the image to be detected by the semantic segmentation model; and output an alarm signal when a smoke image is detected;

[0043] The semantic segmentation model is trained by the following steps:

[0044] Obtain smoke images and construct a semantic segmentation dataset;

[0045] Preprocess the semantic segmentation dataset;

[0046] Build a semantic segmentation model using the encoder-decoder as the basic structure;

[0047] Use the preprocessed semantic segmentation dataset to train the built semantic segmentation model;

[0048] Set network training parameters and use the loss function to train the built semantic segmentation model.

[0049] In a third aspect, the present invention provides a transmission line wildfire monitoring device based on semantic segmentation, comprising a processor and a storage medium;

[0050] The storage medium is used to store instructions;

[0051] The processor is configured to operate according to the instructions to execute the steps of any one of the methods of the first aspect.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention provides a power transmission line wildfire monitoring method, device and storage medium based on semantic segmentation. The method uses the MCCL-self-attention module and improves the existing DeeplabV3 semantic segmentation model to enhance the accuracy of segmentation boundaries, enhance the expression of effective features and suppress invalid features, alleviate the problems of insignificance and inter-class similarity in smoke targets, solve the problems of information loss and dissipation, can accurately segment wildfire smoke, and improve the accuracy of semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is an overall flow chart of the method for monitoring wildfires on power transmission lines based on semantic segmentation provided in the first embodiment of the present invention;

[0056] Figure 2 This is a model structure diagram of the transmission line wildfire monitoring method based on semantic segmentation provided in the first embodiment of the present invention;

[0057] Figure 3 Schematic diagram of the ResNet50 deep residual network in the power line wildfire monitoring method based on semantic segmentation provided in Example 1 of the present invention;

[0058] Figure 4 Schematic diagram of the MCCL module in the MCCL-self-attention module in the power transmission line wildfire monitoring method based on semantic segmentation provided in Example 1 of the present invention;

[0059] Figure 5 Schematic diagram of the DPPM module in the MCCL-self-attention module in the power transmission line wildfire monitoring method based on semantic segmentation provided in the first embodiment of the present invention;

[0060] Figure 6 Schematic diagram of the CBAM module in the MCCL-self-attention module in the power line wildfire monitoring method based on semantic segmentation provided in the first embodiment of the present invention;

[0061] Figure 7 Schematic diagram of the SA module in the CBAM module in the power line wildfire monitoring method based on semantic segmentation provided in Example 1 of the present invention;

[0062] Figure 8 Schematic diagram of the CA module in the CBAM module in the power line wildfire monitoring method based on semantic segmentation provided in Example 1 of the present invention;

[0063] Figure 9 Schematic diagram of a self-attention module suitable for processing two-dimensional signals in the MCCL-self-attention module in the power transmission line wildfire monitoring method based on semantic segmentation provided in the first embodiment of the present invention;

[0064] Figure 10 Schematic diagram of the self-attention module in the power line wildfire monitoring method based on semantic segmentation provided in the first embodiment of the present invention;

[0065] Figure 11 This is a segmentation result diagram obtained by testing a semantic segmentation network model in the power line wildfire monitoring method based on semantic segmentation provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0066] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0067] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0068] Example 1:

[0069] Figure 1 This is a flow chart of a method for monitoring forest fires on power transmission lines based on semantic segmentation provided by the first embodiment of the present invention. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0070] The method for monitoring wildfires on power transmission lines based on semantic segmentation provided in this embodiment can be applied to a terminal and can be executed by a device for monitoring wildfires on power transmission lines based on semantic segmentation. The device can be implemented in software and / or hardware and can be integrated into a terminal, such as any smartphone, tablet computer, or computer device with communication capabilities. Figure 1 , the method of this implementation specifically includes the following steps:

[0071] Step 1: Obtain smoke images and build a semantic segmentation dataset;

[0072] Surveillance videos containing smoke are captured using surveillance cameras. Smoke images are captured from these videos, with one sample taken every preset number of frames. Because smoke images change slowly from frame to frame, a sample is taken every four frames. Smoke targets are manually annotated at the pixel level using Adobe Photoshop. The dataset currently constructed includes, in addition to wildfire smoke images, hundreds of real-world smoke scenes, including factories, forests, and buildings, to increase the dataset's diversity. To enhance the sample richness, some synthetic smoke data and difficult samples are also added to form the final semantic segmentation dataset.

[0073] Step 2: Use data enhancement technology to preprocess the data set;

[0074] The preprocessing of the dataset mainly uses data enhancement technology to enrich the smoke dataset. New smoke data is generated by performing horizontal flipping, vertical flipping, rotation, translation, scaling, cropping, erasing, contrast, saturation, color jittering and other techniques on the input dataset image, so as to improve the generalization ability of the model and prevent network overfitting.

[0075] Step 3: Using the encoder-decoder as the basic structure, build a semantic segmentation network model;

[0076] Among them, the encoding stage uses the improved ResNet50 deep residual network as the skeleton network, and the ResNet50 deep residual network includes stage one, stage two, stage three and stage four; the improvement of the ResNet50 deep residual network includes: replacing the ordinary convolution in stage three and stage four with hollow convolution, so that the output size of stage three and stage four remains consistent with that of stage two.

[0077] Step 4: Use the preprocessed semantic segmentation dataset to train the constructed semantic segmentation network model;

[0078] In the encoding stage, the MCCL-self-attention module is used to extract and enhance the feature expression of the output results of the skeleton network in the encoder. The features include deep features and shallow features. In the decoding stage, the deep features and shallow features from the encoding stage are combined for fusion extraction and output as segmentation results. Convolution processing and upsampling operations are used to make the final segmentation result size the same as the original input smoke image.

[0079] The encoder output features can be divided into local features and global features. The decoder further fuses its local features with the global features to improve the accuracy of the segmentation boundary.

[0080] The feature extraction of the output result of the skeleton network in the encoder during the encoding stage includes:

[0081] The first stage of the ResNet50 deep residual network performs feature extraction and outputs local features to the decoder. The fourth stage outputs local features and sends them to the MCCL-self-attention module. The local features output by the second stage, the local features output by the third stage after 1×1 convolution, and the local features output by the MCCL-self-attention module are merged. The obtained results are then processed by 1×1 convolution and the global features are output to the decoder.

[0082] The MCCL-self-attention module includes the MCCL module (multi-scale contextual contrast local module), the CBAM module (lightweight attention module) and the self-attention module (self-attention module); the MCCL module includes the DPPM module (dense pyramid module).

[0083] The MCCL module is connected to the fourth stage of the ResNet50 deep residual network. The processing process of the local features output by the fourth stage being sent to the MCCL-self-attention module includes: the local features output by the fourth stage are sent to the MCCL module, the local features are processed by four 3×3 convolutions with void convolution rates of 1, 6, 12 and 18 respectively, the local features after convolution processing are subtracted and fused, and the local features after convolution processing are output from the MCCL module and sent to the DPPM module for processing, the local features output by the DPPM module and the fused local features output by the MCCL module are re-fused, and sent to the CBAM module and the self-attention module for processing respectively. This can better process inconspicuous targets in the image background and alleviate the insignificance problem of smoke targets.

[0084] The CBAM module includes the SA module (spatial attention module) and the CA module (channel attention module), which can effectively enhance effective features and suppress invalid features or noise. The CA module and the SA module are respectively connected to the output of the MCCL module. The processing process in the CBAM module includes:

[0085] The SA module and CA module assign weights M to the local features output by stage 4 respectively. S and M C , the local features F and F' output from stage 4 are respectively compared with the assigned weights M S and M C Multiply them to get new local features, add the new local features and output them to the self-attention module for processing.

[0086] In stage four, the local features fed into the MCCL are concatenated with four 3×3 convolutions with dilated convolution rates of 1, 2, 4, and 6, respectively, and then connected to the input of the DPPM module. The DPPM module uses average pooling operations with different strides to obtain global contextual priors and contextual information for different subregions, placing small feature maps in the upper region of the module. To further enhance the module's ability to aggregate contextual information from different subregions, the DPPM module upsamples the feature maps in the upper region to the corresponding size and feeds them into the lower layers for dense feature concatenation. The fusion of features at four different scales through average pooling operations with different kernels complements the global contextual prior, effectively resolving interference from clouds and fog, which resemble smoke in appearance, and improving detection accuracy.

[0087] The self-attention module is connected to the output of stage 4, MCCL module, and CBAM module of the ResNet50 deep residual network. The processing of the self-attention module includes:

[0088] The local features input to the fourth stage of the ResNet50 deep residual network undergo 1×1 convolution and dimension transformation to obtain local feature A. The local features output by the MCCL module undergo 1×1 convolution and dimension transformation to obtain local feature B. Local feature A and local feature B are multiplied to output local feature C. The local features output by the CBAM module undergo 1×1 convolution and dimension transformation to obtain local feature D. Local feature C and local feature D are multiplied, the result is dimensionally transformed and 1×1 convolved, and then added to local feature A. The resulting local features are then subjected to 1×1 convolution. This module directly integrates the information of the input features of the fourth stage of the ResNet50 deep residual network, the MCCL module, and the CBAM module, rather than simply obtaining more global information by stacking multiple convolutional layers, providing richer semantic information for subsequent layers.

[0089] In the decoder, the shallow features output by the encoder are first input into the decoder, and the features are integrated using 1×1 convolution. Then, the deep features finally output from the encoder are upsampled by 4 times. After stacking and combining the local features and the global features, they pass through a convolution layer with a convolution kernel size of 3×3, and then are upsampled by 4 times and output.

[0090] Step 5: Set network training parameters and use the loss function to train the constructed semantic segmentation network model;

[0091] The specific steps include: using images from the preprocessed semantic segmentation image dataset as input images, and unifying the input image size; training the semantic segmentation model using stochastic gradient descent with momentum, with Kaiming weight initialization and Reinforced Lu (ReLU) activation function; and using a loss function to predict the degree of discrepancy between the predicted and actual data. The semantic segmentation model was trained using stochastic gradient descent with momentum, with 30,000 iterations, 1e-5 weight decay, 0.9 momentum, 4 batches, an initial learning rate of 0.001, and a "Poly" learning rate iteration strategy, which is an exponential transformation strategy.

[0092] The loss function is DiceLoss+Focal Loss function (similarity loss function+boundary loss function), and the calculation formula of the loss function is:

[0093]

[0094] FocalLoss=-α t (1-p t ) γ log(P t )

[0095]

[0096] In the formula, DiceLoss is the similarity loss function, Focal Loss is the boundary loss function, Total Loss is the total loss function, and y i represents the label value of pixel i, Represents the predicted value of pixel i, N is the total number of pixels, p t Represents the probability (i.e. confidence) of the model predicting a certain category; α t It is the value used to balance the number of positive and negative samples, and γ is the value used to adjust the imbalance problem of difficult and easy samples.

[0097] Among them, the larger the number of samples, the smaller α is given. t Value, the smaller the number of samples, the larger α is given t value; generally γ ≥ 1, which is used to reduce the loss of easy-to-separate samples by a power function.

[0098] The Focal Loss function is primarily designed to address the imbalance in the number of positive and negative samples, as well as the imbalance in the number of samples with different difficulty levels. The Dice Loss function can mitigate the negative impact of foreground-background imbalance in samples. This imbalance means that most areas of the image do not contain the target, while only a small part does. Dice Loss training focuses more on mining the foreground area, ensuring a lower FN, but this can lead to loss saturation. Therefore, using the Dice Loss function alone often does not yield good results and requires a combination. Therefore, the loss function in this embodiment uses the Dice Loss + Focal Loss function.

[0099] Step 6: Obtain the image to be detected and input it into the pre-trained semantic segmentation model to output the smoke image detection result.

[0100] The acquired images to be detected primarily consist of stereo environmental signals, which can include photos or video signals. These signals are then fed into a pre-trained semantic segmentation model to identify smoke within the images. Upon detecting smoke, an audible and visual alarm signal is output to alert personnel near the transmission lines of a fire, allowing them to locate the fire source and extinguish or evacuate the area promptly. The emitted bright light also facilitates pinpointing the fire's origin during the initial smoke phase of a nighttime wildfire, when the source brightness is low. This system can quickly identify all possible smoke conditions within the captured area, eliminating the need to wait for smoke to diffuse to the monitoring device or for image signals to be aggregated before performing identification. This effectively detects fire signals and enables real-time detection.

[0101] This embodiment discloses a method for monitoring wildfires on power transmission lines based on semantic segmentation. It uses the MCCL-self-attention module and improves the existing DeeplabV3 semantic segmentation model to enhance the accuracy of segmentation boundaries, enhance the expression of effective features and suppress invalid features, alleviate the problems of insignificance and inter-class similarity in smoke targets, solve the problems of information loss and dissipation, accurately segment wildfire smoke, and improve the accuracy of semantic segmentation.

[0102] Example 2:

[0103] This embodiment provides a transmission line wildfire monitoring device based on semantic segmentation, the device comprising:

[0104] The image acquisition module is used to obtain the image to be detected and input it into the pre-trained semantic segmentation model;

[0105] An image detection module is configured to output a smoke image detection result based on the image to be detected by the semantic segmentation model; and output an alarm signal when a smoke image is detected;

[0106] The semantic segmentation model is trained by the following steps:

[0107] Obtain smoke images and construct a semantic segmentation dataset;

[0108] Preprocess the semantic segmentation dataset;

[0109] Build a semantic segmentation model using the encoder-decoder as the basic structure;

[0110] Use the preprocessed semantic segmentation dataset to train the built semantic segmentation model;

[0111] Set network training parameters and use the loss function to train the built semantic segmentation model.

[0112] The transmission line wildfire monitoring device based on semantic segmentation provided in an embodiment of the present invention can execute the transmission line wildfire monitoring method based on semantic segmentation provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Example 3:

[0114] An embodiment of the present invention further provides a transmission line wildfire monitoring device based on semantic segmentation, comprising a processor and a storage medium;

[0115] The storage medium is used to store instructions;

[0116] The processor is configured to operate according to the instructions to execute the steps of the method described in embodiment 1.

[0117] Example 4:

[0118] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first embodiment when the program is executed by a processor.

[0119] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for monitoring wildfires on power transmission lines based on semantic segmentation, characterized in that: include: Obtain the image to be detected and input it into the pre-trained semantic segmentation model; The semantic segmentation model outputs a smoke image detection result based on the image to be detected; when a smoke image is detected, an alarm signal is output; The semantic segmentation model is trained by the following steps: Obtain smoke images and construct a semantic segmentation dataset; Preprocess the semantic segmentation dataset; Build a semantic segmentation model using the encoder-decoder as the basic structure; Use the preprocessed semantic segmentation dataset to train the built semantic segmentation model; Set network training parameters and use the loss function to train the built semantic segmentation model; The semantic segmentation model built by training the preprocessed semantic segmentation dataset includes: During the encoding phase, the output of the skeleton network in the encoder is subjected to feature extraction and enhanced feature expression. The features include deep features and shallow features. During the decoding phase, the deep features and shallow features from the encoding phase are combined for fusion extraction and output as segmentation results. Convolution processing and upsampling operations are then performed to ensure that the final segmentation result size is the same as the original input smoke image size. The skeleton network is an improved ResNet50 deep residual network, which includes stage one, stage two, stage three and stage four. The improvement of the ResNet50 deep residual network includes: replacing the ordinary convolution in stage three and stage four with hollow convolution; The feature extraction of the output result of the skeleton network in the encoder during the encoding stage includes: The local features output by the first stage are fed into the decoder, and the local features output by the fourth stage are fed into the MCCL-self-attention module. The local features output by the second stage, the local features output by the third stage after convolution processing, and the local features output by the MCCL-self-attention module are merged, and the obtained results are further convolved to output global features and fed into the decoder. The MCCL-self-attention module includes an MCCL module, a CBAM module and a self-attention module; the MCCL module includes a DPPM module; The processing process after the local features output by the fourth stage are sent to the MCCL-self-attention module includes: sending the local features output by the fourth stage to the MCCL module, subjecting the local features to convolution processing, performing feature subtraction and fusion on the local features after the convolution processing, outputting them from the MCCL module, and sending them to the DPPM module for processing, re-fusing the local features output by the DPPM module with the fused local features output by the MCCL module, and sending them to the CBAM module and the self-attention module for processing respectively; The CBAM module includes an SA module and a CA module. The processing process of the CBAM module includes: The SA module and the CA module assign weights to the local features output by the fourth stage respectively, multiply the local features output by the fourth stage by the assigned weights respectively to obtain new local features, add the obtained new local features and output them to the self-attention module for processing; The processing process of the self-attention module includes: The local features input in stage four are convolved and dimensionally transformed to obtain local feature A. The local features output by the MCCL module are convolved and dimensionally transformed to obtain local feature B. Local feature A is multiplied by local feature B to output local feature C. The local features output by the CBAM module are convolved and dimensionally transformed to obtain local feature D. Local feature C is multiplied by local feature D. The result is dimensionally transformed and convolved, and then added to local feature A to convolve the obtained local feature.

2. The method for monitoring power line wildfires based on semantic segmentation according to claim 1, characterized in that: The method of obtaining a smoke image and constructing a semantic segmentation dataset includes: Obtain surveillance video containing smoke through surveillance cameras; In surveillance videos containing smoke, a smoke image is captured every preset number of frames as a sample, and the smoke targets are manually labeled at the pixel level; Add synthetic smoke data and difficult samples to form the final semantic segmentation dataset.

3. The method for monitoring power line wildfires based on semantic segmentation according to claim 1, wherein: The preprocessing of the semantic segmentation dataset includes data enhancement of the images in the semantic segmentation dataset, and the data enhancement of the images in the semantic segmentation dataset includes horizontal flipping, vertical flipping, rotation, translation, scaling, cropping, erasing, contrast adjustment, saturation adjustment and color jittering operations on the images in the semantic segmentation dataset.

4. The method for monitoring power line wildfires based on semantic segmentation according to claim 1, wherein: The setting of network training parameters and the use of loss function to train the constructed semantic segmentation model include: Use the images in the preprocessed semantic segmentation image dataset as input images and unify the size of the input images; The semantic segmentation network model is trained using a stochastic gradient descent algorithm with momentum, wherein the weight initialization method in the semantic segmentation network model is Kaiming and the activation function is ReLu; The loss function is used to predict the difference between the prediction and the actual data, and the loss function is DiceLoss+FocalLoss function; The calculation formula of the loss function is: ； ； ； In the formula, DiceLoss is the similarity loss function, FocalLoss is the boundary loss function, and TotalLoss is the total loss function. represents the label value of pixel i, Represents the predicted value of pixel i, N is the total number of pixels, Represents the probability of the model predicting a certain category; is the value used to balance the number of positive and negative samples. It is a value used to adjust the imbalance problem of difficult and easy samples.

5. A transmission line wildfire monitoring device based on semantic segmentation, characterized in that: include: The image acquisition module is used to obtain the image to be detected and input it into the pre-trained semantic segmentation model; An image detection module is configured to output a smoke image detection result based on the image to be detected by the semantic segmentation model; and output an alarm signal when a smoke image is detected; The semantic segmentation model is trained by the following steps: Obtain smoke images and construct a semantic segmentation dataset; Preprocess the semantic segmentation dataset; Build a semantic segmentation model using the encoder-decoder as the basic structure; Use the preprocessed semantic segmentation dataset to train the built semantic segmentation model; Set network training parameters and use the loss function to train the built semantic segmentation model; The semantic segmentation model built by training the preprocessed semantic segmentation dataset includes: During the encoding phase, the output of the skeleton network in the encoder is subjected to feature extraction and enhanced feature expression. The features include deep features and shallow features. During the decoding phase, the deep features and shallow features from the encoding phase are combined for fusion extraction and output as segmentation results. Convolution processing and upsampling operations are then performed to ensure that the final segmentation result size is the same as the original input smoke image size. The skeleton network is an improved ResNet50 deep residual network, which includes stage one, stage two, stage three and stage four. The improvement of the ResNet50 deep residual network includes: replacing the ordinary convolution in stage three and stage four with hollow convolution; The feature extraction of the output result of the skeleton network in the encoder during the encoding stage includes: The local features output by the first stage are fed into the decoder, and the local features output by the fourth stage are fed into the MCCL-self-attention module. The local features output by the second stage, the local features output by the third stage after convolution processing, and the local features output by the MCCL-self-attention module are merged, and the obtained results are further convolved to output global features and fed into the decoder. The MCCL-self-attention module includes an MCCL module, a CBAM module and a self-attention module; the MCCL module includes a DPPM module; The processing process after the local features output by the fourth stage are sent to the MCCL-self-attention module includes: sending the local features output by the fourth stage to the MCCL module, subjecting the local features to convolution processing, performing feature subtraction and fusion on the local features after the convolution processing, outputting them from the MCCL module, and sending them to the DPPM module for processing, re-fusing the local features output by the DPPM module with the fused local features output by the MCCL module, and sending them to the CBAM module and the self-attention module for processing respectively; The CBAM module includes an SA module and a CA module. The processing process of the CBAM module includes: The SA module and the CA module assign weights to the local features output by the fourth stage respectively, multiply the local features output by the fourth stage by the assigned weights respectively to obtain new local features, add the obtained new local features and output them to the self-attention module for processing; The processing process of the self-attention module includes: The local features input in stage four are convolved and dimensionally transformed to obtain local feature A. The local features output by the MCCL module are convolved and dimensionally transformed to obtain local feature B. Local feature A is multiplied by local feature B to output local feature C. The local features output by the CBAM module are convolved and dimensionally transformed to obtain local feature D. Local feature C is multiplied by local feature D. The result is dimensionally transformed and convolved, and then added to local feature A to convolve the obtained local feature.

6. A transmission line wildfire monitoring device based on semantic segmentation, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images