A method and system for detecting forest fires on power transmission lines based on improved YOLO

Abstract

The present invention discloses a method and system for detecting wildfires on power transmission lines based on improved YOLO, which relates to the technical field of electric power safety detection. The method and system collect wildfire sample data sets of power transmission lines; the wildfire sample data sets include marked daytime sample data sets and nighttime sample data sets; the YOLO model is improved, a Backbone module and a feature fusion module are added, and a wildfire detection model is obtained; the wildfire detection model is trained using the wildfire sample data sets; a real-time transmission line image to be detected is obtained, and the real-time transmission line image is input into the wildfire detection model to obtain a wildfire detection result. By adopting the embodiment of the present invention, by adding a Backbone module and a feature fusion module to the traditional YOLO model, it is possible to focus more on the extraction of features in a specific time period, thereby improving the detection accuracy of flame features, and effectively responding to fire detection tasks in various scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of power safety detection, and in particular to a method and system for detecting forest fires on power transmission lines based on improved YOLO. Background Art

[0002] In modern society, power transmission is a core part of infrastructure, and its safety and stability are of vital importance. However, power transmission lines usually cross vast forest areas and are extremely vulnerable to wildfires. Once a fire occurs, it will not only cause large-scale power outages, but may also trigger a chain reaction and cause more serious disasters.

[0003] At present, a fire detection system model based on deep learning is usually used in combination with multimodal data analysis technology for wildfire identification. It can also analyze other relevant information in the environment, such as temperature, humidity, etc. This multi-dimensional data fusion enables the system to accurately judge the occurrence and development of fires in complex environments, thereby greatly improving the reliability and practicality of detection.

[0004] However, the scarcity of abnormal samples of wildfires around nighttime routes makes the model susceptible to noise interference when extracting flame features. In addition, other light sources in nighttime scenes, such as vehicle lights and distant city lights, may have similar spectral characteristics to flames, increasing the risk of misjudgment of the model and posing a severe challenge to accurate nighttime wildfire detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to improve the detection accuracy of wildfires at night. A method and system for wildfire detection of power transmission lines based on improved YOLO are provided. By performing wildfire detection on power transmission lines based on the wildfire detection model based on improved YOLO, it is possible to focus more on extracting features of a specific time period, thereby improving the detection accuracy of flame features, and effectively coping with fire detection tasks in various scenarios.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting forest fires on power transmission lines based on improved YOLO, comprising:

[0007] Collecting a wildfire sample data set of a power transmission line; the wildfire sample data set includes a marked daytime sample data set and a nighttime sample data set;

[0008] Improve the YOLO model by adding the Backbone module and feature fusion module to obtain a wildfire detection model;

[0009] Using the wildfire sample data set to train the wildfire detection model;

[0010] A real-time transmission line image to be detected is obtained, and the real-time transmission line image is input into the wildfire detection model to obtain a wildfire detection result.

[0011] As an improvement of the above solution, the wildfire sample data set of the power transmission line is collected, including:

[0012] Collecting historical wildfire sample data sets of power transmission lines; the historical wildfire sample data sets include daytime sample data sets and first nighttime sample data sets at different fire stages and different distances;

[0013] The CycleGAN algorithm is used to simulate and generate the second night sample data set;

[0014] The historical wildfire sample data set and the second nighttime sample data set are marked to obtain a wildfire sample data set.

[0015] As an improvement to the above solution, the YOLO model is improved by adding a Backbone module and a feature fusion module to obtain a wildfire detection model, including:

[0016] The YOLO model is improved. On the basis of the traditional YOLO model, a Backbone module and a feature fusion module are added to obtain a wildfire detection model; the wildfire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module;

[0017] Among them, the first Backbone module is used to extract daytime wildfire features; the second Backbone module is used to extract nighttime wildfire features; the feature fusion module is used to fuse the daytime wildfire features and the nighttime wildfire features to obtain day and night fusion features; the Neck module is used to perform multi-scale fusion of the day and night fusion features to obtain multi-scale fusion features; the Head module is used to perform regression prediction and generate wildfire image coordinates.

[0018] As an improvement of the above solution, the method of using the wildfire sample data set to train the wildfire detection model includes:

[0019] Performing data enhancement on the wildfire sample data set to obtain an enhanced wildfire sample data set;

[0020] Dividing the enhanced wildfire sample data set into a training set, a validation set, and a test set according to a preset ratio;

[0021] Using the training set and the validation set to train the wildfire detection model;

[0022] The trained wildfire detection model is tested using the test set to obtain performance parameters of the wildfire detection model.

[0023] As an improvement of the above solution, the method of using the wildfire sample data set to train the wildfire detection model includes:

[0024] Inputting the daytime sample data set into the first Backbone module, and extracting daytime wildfire features according to the labels of the daytime sample data set;

[0025] Inputting the nighttime sample data set into the second Backbone module, and extracting nighttime wildfire features according to the labels of the nighttime sample data set;

[0026] Inputting the daytime wildfire features and the nighttime wildfire features into the feature fusion module to obtain daytime and nighttime fusion features of different scales;

[0027] The day and night fusion features are input into the Neck module, and the day and night fusion features of different scales are bidirectionally fused to obtain multi-scale fusion features;

[0028] Inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates;

[0029] The parameters of the wildfire detection model are updated according to the wildfire image coordinates and the wildfire sample data set.

[0030] As an improvement of the above solution, the daytime wildfire features and the nighttime wildfire features are input into the feature fusion module to obtain daytime and nighttime fusion features of different scales, including:

[0031] The daytime wildfire characteristics and the nighttime wildfire characteristics are respectively compressed to obtain compressed daytime wildfire characteristics and compressed nighttime wildfire characteristics;

[0032] The compressed daytime wildfire features and the compressed nighttime wildfire features are pooled respectively to obtain pooled daytime wildfire features and pooled nighttime wildfire features;

[0033] The pooled daytime wildfire features and the pooled nighttime wildfire features are input into the multi-layer perceptron of the feature fusion module, and the output of the multi-layer perceptron is subjected to a feature-weighted splicing operation to obtain day and night fusion features of different scales.

[0034] As an improvement of the above solution, the day and night fusion features are input into the Neck module, and the day and night fusion features of different scales are bidirectionally fused to obtain multi-scale fusion features, including:

[0035] Through the feature pyramid network, the day and night fusion features of different scales are path-fused from bottom to top to obtain a first fusion result;

[0036] Through a path aggregation network, the day and night fusion features of different scales are path-fused from top to bottom to obtain a second fusion result;

[0037] A multi-scale fusion feature is obtained according to the first fusion result and the second fusion result.

[0038] As an improvement of the above solution, the multi-scale fusion feature is input into the Head module to obtain the wildfire image coordinates, including:

[0039] Inputting the multi-scale fusion features into the Head module, generating anchor boxes and the confidence of each anchor box on the multi-scale fusion features of different scales;

[0040] Use the GIoU function to perform bounding box regression on the anchor box to obtain a target bounding box;

[0041] By using a classification network, a cross entropy loss algorithm is used to calculate the category probability of the anchor box to obtain a target category; the target category includes flame and non-flame;

[0042] The target bounding box coordinates are screened according to the confidence level and the target category to obtain wildfire image coordinates.

[0043] As an improvement of the above solution, the step of acquiring a real-time transmission line image to be detected and inputting the real-time transmission line image into the wildfire detection model to obtain a wildfire detection result includes:

[0044] Acquire real-time transmission line images to be inspected;

[0045] Preprocessing the real-time transmission line image, and inputting the preprocessed real-time transmission line image into the wildfire detection model to obtain wildfire image coordinates;

[0046] Using geographic mapping, converting the wildfire image coordinates into geographic coordinates;

[0047] According to the wildfire image coordinates at different times, the fire area time series data is obtained;

[0048] Using a linear regression algorithm to process the fire area time series data to obtain a predicted fire area;

[0049] A wildfire detection result is obtained according to the geographic coordinates and the predicted fire area.

[0050] The embodiment of the present invention also provides a power transmission line wildfire detection system based on improved YOLO, including:

[0051] A sample collection module, used to collect wildfire sample data sets of power transmission lines; the wildfire sample data sets include marked daytime sample data sets and nighttime sample data sets;

[0052] The model building module is used to improve the YOLO model by adding the Backbone module and the feature fusion module to obtain the wildfire detection model;

[0053] A model training module, used to train the wildfire detection model using the wildfire sample data set;

[0054] The real-time detection module is used to obtain the real-time transmission line image to be detected, input the real-time transmission line image into the wildfire detection model, and obtain the wildfire detection result.

[0055] Compared with the prior art, the present invention discloses a method and system for detecting wildfires on power transmission lines based on improved YOLO, which collects wildfire sample data sets of power transmission lines; the wildfire sample data sets include marked daytime sample data sets and nighttime sample data sets; improves the YOLO model, adds a Backbone module and a feature fusion module, and obtains a wildfire detection model; uses the wildfire sample data sets to train the wildfire detection model; obtains a real-time power transmission line image to be detected, and inputs the real-time power transmission line image into the wildfire detection model to obtain a wildfire detection result. By adopting the embodiments of the present invention, it is possible to focus more on the extraction of features in a specific time period, thereby improving the detection accuracy of flame features, and effectively responding to fire detection tasks in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a schematic diagram of the steps of a method for detecting wildfires on power transmission lines based on improved YOLO provided by an embodiment of the present invention;

[0057] Figure 2 is a schematic diagram of a network architecture of a wildfire detection model provided by an embodiment of the present invention;

[0058] Figure 3 is a schematic structural diagram of a C3 unit provided in an embodiment of the present invention;

[0059] Figure 4 is a structural schematic diagram of a feature fusion module provided by an embodiment of the present invention;

[0060] Figure 5 It is a structural schematic diagram of a power transmission line wildfire detection system based on improved YOLO provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In the description of the specification and claims, it is to be understood that the terms first, second, etc. in the specification and claims are only used for the purpose of describing the same technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor necessarily describing the order or time sequence. The terms are interchangeable where appropriate. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0063] Traditional fire alarm methods mostly rely on smoke or temperature sensing, which works by detecting smoke particles in the air. However, this mechanism has obvious lag, because the smoke detector will only trigger the alarm when the fire produces enough smoke particles, which often means that the fire has developed to a more serious stage and it is easy to miss the best warning time. Past fire cases have shown that once the fire reaches a certain scale, the difficulty of rescue work increases sharply, and personnel evacuation and fire fighting operations face severe challenges.

[0064] The YOLO (You Only Look Once) model is a target detection model that has the advantages of strong real-time performance, high accuracy, and lightweight. It is used in a wide range of application scenarios such as object recognition, human detection, and vehicle recognition. The YOLO model provides a more advanced solution for early detection of fires in power transmission lines. Compared with traditional fire alarm methods, it is more intelligent and more efficient, which wins valuable time for timely response measures.

[0065] However, the scarcity of abnormal samples of wildfires around nighttime routes makes the model susceptible to noise interference when extracting flame features. Secondly, other light sources in nighttime scenes, such as vehicle lights and distant city lights, may have similar spectral characteristics to flames, increasing the risk of misjudgment of the model and posing severe challenges to accurate nighttime wildfire detection.

[0066] Based on the above considerations, the embodiment of the present invention provides a method for detecting forest fires on power transmission lines based on improved YOLO. Figure 1 The power transmission line wildfire detection method is specifically performed through steps S1 to S4:

[0067] S1. Collecting a wildfire sample data set of a power transmission line; the wildfire sample data set includes a marked daytime sample data set and a nighttime sample data set.

[0068] Preferably, the wildfire sample data set includes daytime sample data sets and nighttime sample data sets at different fire stages, different distances, and different angles.

[0069] Exemplarily, the fire stages include the initial stage, the middle stage and the fierce stage. In the initial stage, the fire is weak and small in scope, producing initial flames and smoke without significant spread; in the middle stage, the flames and smoke spread within a certain range, the fire gradually escalates, forming a larger-scale fire source, and approaching the transmission line; in the fierce stage, the combustion intensity is the greatest, the heat radiation is the strongest, and the line operation has been seriously affected.

[0070] S2. Improve the YOLO model by adding a Backbone module and a feature fusion module to obtain a wildfire detection model.

[0071] Since traditional sample data sets are mainly concentrated on pictures taken under visible light conditions during the day, there is a great lack of data sets under night lighting conditions. The traditional YOLO model directly uses the acquired data set for testing. In practical applications, the traditional YOLO has a weak ability to distinguish between night flames and lights, and therefore has insufficient detection accuracy for night wildfires. In an embodiment of the present invention, by adding a Backbone module and a feature fusion module to the traditional YOLO model, different Backbone modules focus on the extraction of features in different specific time periods, further deepening the model's ability to extract features of night wildfires, and can improve the detection accuracy of flame features.

[0072] S3. Use the wildfire sample data set to train the wildfire detection model.

[0073] It should be noted that the markers in the wildfire sample data set can assist the wildfire detection model in extracting flame features, and can also verify the output results.

[0074] S4. Acquire a real-time image of a power transmission line to be detected, input the real-time image of the power transmission line into the wildfire detection model, and obtain a wildfire detection result.

[0075] Furthermore, preferably, a fire alarm may also be generated according to the wildfire detection result.

[0076] It should be noted that by acquiring real-time transmission line images and inputting them into the wildfire detection model, it is possible to timely discover and identify the early stages of a fire and further identify the trend of fire changes, which is conducive to accurate detection and early warning of wildfires.

[0077] As a preferred implementation, step S1, collecting a wildfire sample data set of a power transmission line, includes:

[0078] Collecting historical wildfire sample data sets of power transmission lines; the historical wildfire sample data sets include daytime sample data sets and first nighttime sample data sets at different fire stages and different distances;

[0079] The CycleGAN (CycleGenerative Adversarial Networks) algorithm is used to simulate and generate the second night sample data set;

[0080] The historical wildfire sample data set and the second nighttime sample data set are marked to obtain a wildfire sample data set.

[0081] It should be noted that the historical wildfire sample dataset can be obtained by collecting images and video data of flames at different stages of a power transmission line wildfire, and then eliminating images or video clips that are irrelevant to the wildfire and are blurred. It can also be obtained by obtaining an open source dataset from the Internet.

[0082] In the embodiment of the present invention, considering the shortage of existing nighttime wildfire samples, the CycleGAN algorithm is introduced to generate a large number of nighttime wildfire image samples. By simulating a variety of complex nighttime scenes, the diversity of training data is improved, the generalization ability of the model is enhanced, and the problem of sample scarcity is effectively solved, making the model more stable in nighttime detection.

[0083] As a preferred implementation, step S2, improving the YOLO model, adding a Backbone module and a feature fusion module, and obtaining a wildfire detection model, including:

[0084] The YOLO model is improved. On the basis of the traditional YOLO model, a Backbone module and a feature fusion module are added to obtain a wildfire detection model; the wildfire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module (target detection head module);

[0085] Among them, the first Backbone module is used to extract daytime wildfire features; the second Backbone module is used to extract nighttime wildfire features; the feature fusion module is used to fuse the daytime wildfire features and the nighttime wildfire features to obtain day and night fusion features; the Neck module is used to perform multi-scale fusion of the day and night fusion features to obtain multi-scale fusion features; the Head module is used to perform regression prediction and generate wildfire image coordinates.

[0086] The traditional YOLO model consists of a Backbone module, a Neck module, and a Head module. Figure 2 The forest fire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module.

[0087] It should be noted that, in the embodiment of the present invention, the first Backbone module is Backbone-D, the second Backbone module is Backbone-N, and the feature fusion module includes three RR units. The first Backbone module outputs three different scales of daytime wildfire features, the second Backbone module outputs three different scales of daytime wildfire features, and the RR unit is used to perform feature fusion of daytime wildfire features and nighttime wildfire features of the same scale.

[0088] It should also be noted that the number of output features of the first Backbone module, the number of output features of the second Backbone module, and the number of RR units are only taken as examples, and the specific values ​​are the same as the number of scales that can be recognized by the wildfire detection model. Therefore, it can be understood that other values ​​can also be used in other embodiments, but there needs to be an association relationship between the three, and the association relationship generally takes the same value.

[0089] As a preferred implementation, step S3, using the wildfire sample data set to train the wildfire detection model, includes:

[0090] Performing data enhancement on the wildfire sample data set to obtain an enhanced wildfire sample data set;

[0091] Dividing the enhanced wildfire sample data set into a training set, a validation set, and a test set according to a preset ratio;

[0092] Using the training set and the validation set to train the wildfire detection model;

[0093] The trained wildfire detection model is tested using the test set to obtain performance parameters of the wildfire detection model.

[0094] Exemplarily, the wildfire sample data set is enhanced by the Mosaic data enhancement algorithm. Specifically, the coordinates (x, y) of the splicing reference points are selected for the image and video data containing flames, and 4 pictures are randomly selected and placed in the upper left, upper right, lower left and lower right positions of the large picture of the specified size after resizing and scaling. According to the size transformation method of each picture, the mapping relationship is mapped to the picture label. Finally, according to the specified horizontal and vertical coordinates, the large picture is spliced ​​to obtain an enhanced wildfire sample data set. Then, the enhanced wildfire sample data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1. The training set and the validation set are used to train the wildfire detection model, and the trained wildfire detection model is tested using the test set to obtain the performance parameters of the wildfire detection model to verify the generalization ability of the wildfire detection model.

[0095] As a preferred implementation, step S3, using the wildfire sample data set to train the wildfire detection model, is specifically performed through steps S31 to S36:

[0096] S31. Input the daytime sample data set into the first Backbone module, and extract daytime wildfire features according to the labels of the daytime sample data set.

[0097] See also Figure 2 The structural diagram of the Backbone-D module shown in the figure shows that in the embodiment of the present invention, after the daytime sample data set is input into the first Backbone module, it first passes through two layers of Conv units and then enters the first C3 unit operation; then the Conv (Conbolution) unit and the C3 unit are repeated twice, the first time forming the first daytime wildfire feature D1, and the second time forming the second daytime wildfire feature D2; by gradually extracting and processing the features of different levels in the image, it helps to improve the expression ability and detection accuracy of the model; finally, the SPPF (Spatial Pyramid Pooling-Fast) unit is used to perform spatial pyramid pooling operations to further enrich the multi-scale information in the feature map, and the input feature map is pooled in different sizes of 5x5, 9x9, and 13x13. Each pooling operation compresses the spatial information and generates feature maps of different scales. The feature maps of different pooling scales are spliced ​​to form the third daytime wildfire feature D3, which contains richer spatial information.

[0098] Exemplarily, the Conv unit includes a convolution operation, batch normalization and an activation function. The activation function is usually a LeakyReLU (Leaky Rectified Linear Unit) function. In the convolution operation, multiple 3x3 convolution kernels are used to slide and scan the entire input image. Each convolution kernel moves on the image to capture local features, such as the edge of a flame or the details of smoke. After the first convolution operation, a 320x320x32 feature map is output, indicating that the input image has been compressed and each pixel represents information of a larger area in the image. The feature map is then normalized to ensure that the output of each layer has a consistent scale to avoid problems such as gradient vanishing or explosion. Through the ReLU function, the negative values ​​in the feature map are set to zero to maintain the nonlinear ability of the network and help the wildfire detection model better learn the complex features of the flame. The first layer of Conv units is used to extract basic features of the input image, such as edges, textures, etc. Since the convolution operation is local, it can capture the patterns of local areas in the image; then enter the second layer of Conv units to further refine these initial features and transform them into more abstract, higher-level representations. The superposition of multiple layers of convolution helps to gradually transition from simple features to complex features.

[0099] See also Figure 3 The C3 unit first divides the input feature map into two parts, one part is processed by multiple bottleneck blocks (Bottleneck Blocks), and the other part is directly passed, and finally feature fusion is performed. In the bottleneck block processing, convolution is performed again, and 1x1 and 3x3 convolution operations are used in each bottleneck block to reduce the number of channels and feature complexity. After passing through multiple bottleneck blocks, the feature map is fused with the unprocessed part to generate a richer feature map (20x20x1024).

[0100] S32: Input the nighttime sample data set into the second Backbone module, and extract nighttime wildfire features according to the labels of the nighttime sample data set.

[0101] It should be noted that, in the embodiment of the present invention, the structure of the second Backbone module is the same as that of the first Backbone module, and the principles are also the same, so they will not be described in detail.

[0102] S33, inputting the daytime wildfire features and the nighttime wildfire features into the feature fusion module to obtain daytime and nighttime fusion features of different scales.

[0103] Exemplarily, in some embodiments, the daytime wildfire features include D1, D2, and D3, and the nighttime wildfire features include N1, N2, and N3. D1, D2, and D3 are fused with N1, N2, and N3 to generate 80×80 day and night fusion features for the detection of small-scale targets. D2 is fused with N2 to generate 40×40 day and night fusion features for the detection of medium-scale targets. Finally, D3 is fused with N3 to generate 20×20 day and night fusion features for the detection of large-scale targets.

[0104] S34, inputting the day and night fusion features into the Neck module, performing bidirectional path fusion on the day and night fusion features of different scales, and obtaining multi-scale fusion features.

[0105] Through bidirectional path fusion, the underlying features can be transferred to the high-level features, and the high-level features can also be transferred to the underlying features, so that the wildfire detection model can accurately detect wildfire targets of different scales in the image.

[0106] S35. Input the multi-scale fusion features into the Head module to obtain the wildfire image coordinates.

[0107] It should be noted that the Head module is used to perform classification, bounding box regression and confidence prediction on the multi-scale fusion features generated by the Neck module.

[0108] S36. Update the parameters of the wildfire detection model according to the wildfire image coordinates and the wildfire sample data set.

[0109] Further, preferably, step S33, inputting the daytime wildfire features and the nighttime wildfire features into the feature fusion module to obtain daytime and nighttime fusion features of different scales includes:

[0110] The daytime wildfire characteristics and the nighttime wildfire characteristics are respectively compressed to obtain compressed daytime wildfire characteristics and compressed nighttime wildfire characteristics;

[0111] The compressed daytime wildfire features and the compressed nighttime wildfire features are pooled respectively to obtain pooled daytime wildfire features and pooled nighttime wildfire features;

[0112] The pooled daytime wildfire features and the pooled nighttime wildfire features are input into the multi-layer perceptron of the feature fusion module, and the output of the multi-layer perceptron is subjected to a feature-weighted splicing operation to obtain day and night fusion features of different scales.

[0113] See also Figure 4In some embodiments, the daytime wildfire feature Dday and the nighttime wildfire feature Dnight are compressed through a network layer (NIN) to generate new feature representations Fday and Fnight; then, Fday and Fnight are subjected to maximum pooling and average pooling to reduce the dimension of the feature and the number of parameters of the model to obtain the pooled daytime wildfire feature and the pooled nighttime wildfire feature; the pooled daytime wildfire feature and the pooled nighttime wildfire feature are then sent to a multilayer perceptron (MLP), and then processed through an activation function to obtain the daytime feature Wday and the nighttime feature Wnight; finally, the daytime feature Wday and the nighttime feature Wnight are fused using a splicing operation.

[0114] Through this feature weighting and fusion operation, the optimal feature weights for daytime and nighttime images can be obtained, enriching the target information of feature extraction, further improving the efficiency of image analysis, and improving the accuracy of target detection.

[0115] Preferably, step S34, inputting the day and night fusion features into the Neck module, performing bidirectional path fusion on the day and night fusion features of different scales to obtain multi-scale fusion features, includes:

[0116] Through the feature pyramid network, the day and night fusion features of different scales are path-fused from bottom to top to obtain a first fusion result;

[0117] Through a path aggregation network, the day and night fusion features of different scales are path-fused from top to bottom to obtain a second fusion result;

[0118] A multi-scale fusion feature is obtained according to the first fusion result and the second fusion result.

[0119] For example, in some embodiments, a feature pyramid network (FPN) is first used to fuse feature maps through a bottom-up path, and 80x80 low-level features are gradually transferred to 40x40 and 20x20 high-level features to form a preliminary feature pyramid. This pyramid enables the model to use information from different scales to identify objects.

[0120] Then, the Path Aggregation Network (PAN) is used for top-down feature transfer and fusion, and the 20x20 high-level semantic features are gradually transferred to the 40x40 and 80x80 low-level features. In this way, the low-level features not only contain their own detailed information, but also integrate the rich semantic information from the high-level. This top-down path aggregation allows the low-level feature maps to better capture the information in small targets and complex backgrounds, while maintaining the global vision of high-level semantic features. The PAN structure fuses multi-level features through a top-down path, allowing the model to process large and small targets at the same time. Generate feature maps of different scales, such as 80x80, 40x40, and 20x20.

[0121] By using the Neck module shown in the embodiment of the present invention, the wildfire detection model can obtain various scale features at the same time, and can identify distant flames and nearby large flames.

[0122] As a preferred implementation, S35, inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates, including:

[0123] Inputting the multi-scale fusion features into the Head module, generating anchor boxes and the confidence of each anchor box on the multi-scale fusion features of different scales;

[0124] Use the GIoU (Generalized Intersection over Union) function to perform bounding box regression on the anchor box to obtain the target bounding box;

[0125] By using a classification network, a cross entropy loss algorithm is used to calculate the category probability of the anchor box to obtain a target category; the target category includes flame and non-flame;

[0126] The target bounding box coordinates are screened according to the confidence level and the target category to obtain wildfire image coordinates.

[0127] It should be noted that the parameters of the anchor box are calculated based on the wildfire sample data set. Exemplarily, after obtaining the enhanced wildfire sample data set, the K-means clustering algorithm is used to calculate the anchor box size that best suits the wildfire sample data set, and the anchor box parameters in the wildfire detection model configuration file are updated based on the anchor box size.

[0128] In some embodiments, a set of anchor boxes are first generated on feature maps of different scales to predict the position and size of the target. The anchor boxes are regressed using the GIoU loss function to obtain the target bounding box, which is a prediction of the relative position and size. The predicted coordinates include the center point (x, y), width (W) and height (H), which represent the position and size of the detection box.

[0129] Through the classification network, each or anchor box is mapped to a category probability distribution, the Softmax function (normalized exponential function) is used to convert the output score of the model into a category probability, and the cross entropy loss is used to measure the difference between the probability distribution of the predicted category and the true category distribution.

[0130] For each anchor box, a confidence value needs to be output. This value is normalized to [0, 1] through the Sigmoid function (S-shaped growth curve function), which represents the probability of the existence of a target in the box. In the embodiment of the present invention, it is the probability of the existence of a wildfire.

[0131] The prediction results are then filtered through non-maximum suppression (NMS), and the predicted bounding boxes are NMS-ed to remove boxes with large overlaps and only retain boxes with the highest confidence to ensure that the final output results are accurate and non-redundant.

[0132] It should be noted that when outputting the wildfire image coordinates, the box with the highest confidence is not taken as the output. The confidence of the box also needs to be greater than the preset confidence threshold. Only then is it determined that a wildfire has occurred and the wildfire image coordinates are obtained. Otherwise, it is determined that no wildfire has occurred.

[0133] It should also be noted that when executing step S3 through steps S31 to S36, the wildfire sample data set can also be divided in advance to obtain a training set, a validation set, and a test set. As described in the embodiment of the present invention, the training set and the validation set are used to train the wildfire detection model, and the test set is used to test the performance of the wildfire detection model.

[0134] As a preferred implementation, S4, obtaining a real-time transmission line image to be detected, inputting the real-time transmission line image into the wildfire detection model, and obtaining a wildfire detection result, including:

[0135] Acquire real-time transmission line images to be inspected;

[0136] Preprocessing the real-time transmission line image, and inputting the preprocessed real-time transmission line image into the wildfire detection model to obtain wildfire image coordinates;

[0137] Using geographic mapping, converting the wildfire image coordinates into geographic coordinates;

[0138] According to the wildfire image coordinates at different times, the fire area time series data is obtained;

[0139] Using a linear regression algorithm to process the fire area time series data to obtain a predicted fire area;

[0140] A wildfire detection result is obtained according to the geographic coordinates and the predicted fire area.

[0141] Exemplarily, the preprocessing of the real-time power transmission line image includes scaling the real-time power transmission line image according to a preset size.

[0142] Preferably, in an embodiment of the present invention, the wildfire image coordinates include the upper left corner coordinates and the lower right corner coordinates (xmin, ymin, xmax, ymax) of the output bounding box. Due to the normalized relative coordinates, they are converted into actual pixel coordinates: (xmin·W, ymin·H, xmax·W, ymax·H), where W and H are the width and height of the transmission line image, respectively. Then, geographic mapping is performed, and the image coordinates are converted into geographic coordinates using the perspective transformation matrix H.

[0143] For example, by Predict the fire area in the next few frames; among them, yes The fire area at the time, and are the parameters of the linear regression algorithm.

[0144] A power transmission line wildfire detection method based on improved YOLO provided by an embodiment of the present invention can focus more on extracting features of a specific time period by adding a Backbone module and a feature fusion module to the traditional YOLO model, thereby improving the detection accuracy of flame features and effectively coping with fire detection tasks in various scenarios.

[0145] See also Figure 5 The embodiment of the present invention provides a power transmission line wildfire detection system based on improved YOLO, which specifically includes a sample collection module 11, a model building module 12, a model training module 13 and a real-time detection module 14, wherein:

[0146] The sample collection module 11 is used to collect wildfire sample data sets of power transmission lines; the wildfire sample data sets include marked daytime sample data sets and nighttime sample data sets;

[0147] The model building module 12 is used to improve the YOLO model, add a Backbone module and a feature fusion module, and obtain a wildfire detection model;

[0148] A model training module 13, used to train the wildfire detection model using the wildfire sample data set;

[0149] The real-time detection module 14 is used to obtain a real-time image of the power transmission line to be detected, and input the real-time image of the power transmission line into the wildfire detection model to obtain a wildfire detection result.

[0150] As a preferred implementation, the sample collection module 11 includes:

[0151] A historical sample collection unit, used to collect a historical wildfire sample data set of the transmission line; the historical wildfire sample data set includes a daytime sample data set and a first nighttime sample data set at different fire stages and different distances;

[0152] A night sample simulation unit, used to simulate and generate a second night sample data set by using a CycleGAN algorithm;

[0153] The data set generation unit is used to mark the historical wildfire sample data set and the second nighttime sample data set to obtain a wildfire sample data set.

[0154] As a preferred implementation, the model building module 12 is specifically used for:

[0155] The YOLO model is improved. On the basis of the traditional YOLO model, a Backbone module and a feature fusion module are added to obtain a wildfire detection model; the wildfire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module;

[0156] Among them, the first Backbone module is used to extract daytime wildfire features; the second Backbone module is used to extract nighttime wildfire features; the feature fusion module is used to fuse the daytime wildfire features and the nighttime wildfire features to obtain day and night fusion features; the Neck module is used to perform multi-scale fusion of the day and night fusion features to obtain multi-scale fusion features; the Head module is used to perform regression prediction and generate wildfire image coordinates.

[0157] As a preferred implementation, the model training module 13 includes:

[0158] A data enhancement unit, used to perform data enhancement on the wildfire sample data set to obtain an enhanced wildfire sample data set;

[0159] A data set division unit, used to divide the enhanced wildfire sample data set into a training set, a validation set and a test set according to a preset ratio;

[0160] A model training unit, configured to train the wildfire detection model using the training set and the validation set;

[0161] The model testing unit is used to test the trained wildfire detection model using the test set to obtain performance parameters of the wildfire detection model.

[0162] As a preferred implementation, the model training module 13 includes:

[0163] A first Backbone module execution unit, configured to input the daytime sample data set into the first Backbone module, and extract daytime wildfire features according to the labels of the daytime sample data set;

[0164] A second Backbone module execution unit, configured to input the nighttime sample data set into the second Backbone module, and extract nighttime wildfire features according to the labels of the nighttime sample data set;

[0165] A feature fusion module execution unit, used for inputting the daytime wildfire feature and the nighttime wildfire feature into the feature fusion module to obtain daytime and nighttime fusion features of different scales;

[0166] A Neck module execution unit, configured to input the day and night fusion features into the Neck module, perform bidirectional path fusion on the day and night fusion features of different scales, and obtain multi-scale fusion features;

[0167] A Head module execution unit, used for inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates;

[0168] A model parameter updating unit is used to update the parameters of the wildfire detection model according to the wildfire image coordinates and the wildfire sample data set.

[0169] Further, preferably, the feature fusion module execution unit is specifically used to:

[0170] The daytime wildfire characteristics and the nighttime wildfire characteristics are respectively compressed to obtain compressed daytime wildfire characteristics and compressed nighttime wildfire characteristics;

[0171] The compressed daytime wildfire features and the compressed nighttime wildfire features are pooled respectively to obtain pooled daytime wildfire features and pooled nighttime wildfire features;

[0172] The pooled daytime wildfire features and the pooled nighttime wildfire features are input into the multi-layer perceptron of the feature fusion module, and the output of the multi-layer perceptron is subjected to a feature-weighted splicing operation to obtain day and night fusion features of different scales.

[0173] Preferably, the Neck module execution unit is specifically used for:

[0174] Through the feature pyramid network, the day and night fusion features of different scales are path-fused from bottom to top to obtain a first fusion result;

[0175] Through a path aggregation network, the day and night fusion features of different scales are path-fused from top to bottom to obtain a second fusion result;

[0176] A multi-scale fusion feature is obtained according to the first fusion result and the second fusion result.

[0177] Preferably, the Head module execution unit is specifically used for:

[0178] Inputting the multi-scale fusion features into the Head module, generating anchor boxes and the confidence of each anchor box on the multi-scale fusion features of different scales;

[0179] Use the GIoU function to perform bounding box regression on the anchor box to obtain a target bounding box;

[0180] By using a classification network, a cross entropy loss algorithm is used to calculate the category probability of the anchor box to obtain a target category; the target category includes flame and non-flame;

[0181] The target bounding box coordinates are screened according to the confidence level and the target category to obtain wildfire image coordinates.

[0182] As a preferred implementation, the real-time detection module 14 includes:

[0183] A real-time image acquisition unit, used to acquire a real-time image of a power transmission line to be detected;

[0184] A coordinate output unit, used to pre-process the real-time transmission line image, input the pre-processed real-time transmission line image into the wildfire detection model, and obtain the wildfire image coordinates;

[0185] A coordinate conversion unit, configured to convert the wildfire image coordinates into geographic coordinates using geographic mapping;

[0186] A time series data generating unit, used to obtain the fire area time series data according to the wildfire image coordinates at different times;

[0187] A fire area prediction unit, used for processing the fire area time series data by using a linear regression algorithm to obtain a predicted fire area;

[0188] The detection result generating unit is used to obtain the wildfire detection result according to the geographic coordinates and the predicted fire area.

[0189] A power transmission line wildfire detection system based on improved YOLO provided by an embodiment of the present invention can focus more on extracting features of a specific time period by adding a Backbone module and a feature fusion module to the traditional YOLO model, thereby improving the detection accuracy of flame features and effectively coping with fire detection tasks in various scenarios.

[0190] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0191] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for detecting forest fires on power transmission lines based on improved YOLO, characterized in that: include: Collect wildfire sample datasets of power transmission lines; The wildfire sample data set includes a marked daytime sample data set and a nighttime sample data set; Improve the YOLO model by adding the Backbone module and feature fusion module to obtain a wildfire detection model; Using the wildfire sample data set to train the wildfire detection model; Acquire a real-time transmission line image to be detected, input the real-time transmission line image into the wildfire detection model, and obtain a wildfire detection result; in, The YOLO model is improved by adding a Backbone module and a feature fusion module to obtain a wildfire detection model, including: The YOLO model is improved. On the basis of the traditional YOLO model, a Backbone module and a feature fusion module are added to obtain a wildfire detection model; the wildfire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module; Among them, the first Backbone module is used to extract daytime wildfire features; the second Backbone module is used to extract nighttime wildfire features; the feature fusion module is used to fuse the daytime wildfire features and the nighttime wildfire features to obtain daytime and nighttime fusion features; the Neck module is used to perform multi-scale fusion on the daytime and nighttime fusion features to obtain multi-scale fusion features; the Head module is used to perform regression prediction and generate wildfire image coordinates; The using the wildfire sample data set to train the wildfire detection model includes: Inputting the daytime sample data set into the first Backbone module, and extracting daytime wildfire features according to the labels of the daytime sample data set; Inputting the nighttime sample data set into the second Backbone module, and extracting nighttime wildfire features according to the labels of the nighttime sample data set; Inputting the daytime wildfire features and the nighttime wildfire features into the feature fusion module to obtain daytime and nighttime fusion features of different scales; The day and night fusion features are input into the Neck module, and the day and night fusion features of different scales are bidirectionally fused to obtain multi-scale fusion features; Inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates; The parameters of the wildfire detection model are updated according to the wildfire image coordinates and the wildfire sample data set.

2. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The wildfire sample data set collected for the power transmission line includes: Collecting historical wildfire sample data sets of power transmission lines; the historical wildfire sample data sets include daytime sample data sets and first nighttime sample data sets at different fire stages and different distances; The CycleGAN algorithm is used to simulate and generate the second night sample data set; The historical wildfire sample data set and the second nighttime sample data set are marked to obtain a wildfire sample data set.

3. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The using the wildfire sample data set to train the wildfire detection model includes: Performing data enhancement on the wildfire sample data set to obtain an enhanced wildfire sample data set; Dividing the enhanced wildfire sample data set into a training set, a validation set, and a test set according to a preset ratio; Using the training set and the validation set to train the wildfire detection model; The trained wildfire detection model is tested using the test set to obtain performance parameters of the wildfire detection model.

4. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The step of inputting the daytime wildfire features and the nighttime wildfire features into the feature fusion module to obtain daytime and nighttime fusion features of different scales includes: The daytime wildfire characteristics and the nighttime wildfire characteristics are respectively compressed to obtain compressed daytime wildfire characteristics and compressed nighttime wildfire characteristics; The compressed daytime wildfire features and the compressed nighttime wildfire features are pooled respectively to obtain pooled daytime wildfire features and pooled nighttime wildfire features; The pooled daytime wildfire features and the pooled nighttime wildfire features are input into the multi-layer perceptron of the feature fusion module, and the output of the multi-layer perceptron is subjected to a feature-weighted splicing operation to obtain day and night fusion features of different scales.

5. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The step of inputting the day and night fusion features into the Neck module, performing bidirectional path fusion on the day and night fusion features of different scales, and obtaining multi-scale fusion features includes: Through the feature pyramid network, the day and night fusion features of different scales are path-fused from bottom to top to obtain a first fusion result; Through a path aggregation network, the day and night fusion features of different scales are path-fused from top to bottom to obtain a second fusion result; A multi-scale fusion feature is obtained according to the first fusion result and the second fusion result.

6. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The step of inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates includes: Inputting the multi-scale fusion features into the Head module, generating anchor boxes and the confidence of each anchor box on the multi-scale fusion features of different scales; Use the GIoU function to perform bounding box regression on the anchor box to obtain a target bounding box; By using a classification network, a cross entropy loss algorithm is used to calculate the category probability of the anchor box to obtain a target category; the target category includes flame and non-flame; The target bounding box coordinates are screened according to the confidence level and the target category to obtain wildfire image coordinates.

7. A method for detecting forest fires on power transmission lines based on improved YOLO as claimed in claim 1, characterized in that: The acquiring of a real-time transmission line image to be detected, inputting the real-time transmission line image into the wildfire detection model, and obtaining a wildfire detection result includes: Acquire real-time transmission line images to be inspected; Preprocessing the real-time transmission line image, and inputting the preprocessed real-time transmission line image into the wildfire detection model to obtain wildfire image coordinates; Using geographic mapping, converting the wildfire image coordinates into geographic coordinates; According to the wildfire image coordinates at different times, the fire area time series data is obtained; Using a linear regression algorithm to process the fire area time series data to obtain a predicted fire area; A wildfire detection result is obtained according to the geographic coordinates and the predicted fire area.

8. A power transmission line fire detection system based on improved YOLO, characterized in that: include: A sample collection module, used to collect wildfire sample data sets of transmission lines; The wildfire sample data set includes a marked daytime sample data set and a nighttime sample data set; The model building module is used to improve the YOLO model by adding the Backbone module and the feature fusion module to obtain the wildfire detection model; A model training module, used to train the wildfire detection model using the wildfire sample data set; A real-time detection module, used to obtain a real-time transmission line image to be detected, input the real-time transmission line image into the wildfire detection model, and obtain a wildfire detection result; The model building module is specifically used for: The YOLO model is improved. On the basis of the traditional YOLO model, a Backbone module and a feature fusion module are added to obtain a wildfire detection model; the wildfire detection model includes a first Backbone module, a second Backbone module, a feature fusion module, a Neck module and a Head module; Among them, the first Backbone module is used to extract daytime wildfire features; the second Backbone module is used to extract nighttime wildfire features; the feature fusion module is used to fuse the daytime wildfire features and the nighttime wildfire features to obtain daytime and nighttime fusion features; the Neck module is used to perform multi-scale fusion on the daytime and nighttime fusion features to obtain multi-scale fusion features; the Head module is used to perform regression prediction and generate wildfire image coordinates; The model training module includes: A first Backbone module execution unit, configured to input the daytime sample data set into the first Backbone module, and extract daytime wildfire features according to the labels of the daytime sample data set; A second Backbone module execution unit, configured to input the nighttime sample data set into the second Backbone module, and extract nighttime wildfire features according to the labels of the nighttime sample data set; A feature fusion module execution unit, used for inputting the daytime wildfire feature and the nighttime wildfire feature into the feature fusion module to obtain daytime and nighttime fusion features of different scales; A Neck module execution unit, configured to input the day and night fusion features into the Neck module, perform bidirectional path fusion on the day and night fusion features of different scales, and obtain multi-scale fusion features; A Head module execution unit, used for inputting the multi-scale fusion features into the Head module to obtain the wildfire image coordinates; A model parameter updating unit is used to update the parameters of the wildfire detection model according to the wildfire image coordinates and the wildfire sample data set.

Images