A Highly Efficient and Intelligent Method and System for Recognizing Typical Trees in Complex Environments Based on FPN

By using deep learning algorithms based on FPN and a drone monitoring system, efficient and real-time identification and monitoring of trees along power transmission line corridors has been achieved. This solves the problems of incomplete coverage and limitations of manual inspection in traditional methods, improves identification efficiency and accuracy, and reduces costs.

CN118968283BActive Publication Date: 2026-11-17GUIZHOU POWER GRID CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410948608.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2026-11-17
Estimated Expiration
2044-07-16

AI Technical Summary

Technical Problem

Existing methods cannot achieve comprehensive coverage and real-time monitoring of transmission line corridors, and manual inspections are limited by natural conditions such as weather and terrain, making it difficult to cope with emergencies.

Method used

A deep learning algorithm based on Feature Pyramid Network (FPN) is used in conjunction with drones and cameras to collect, preprocess, build and evaluate tree data, enabling automatic identification and real-time monitoring of trees along power transmission line corridors.

Benefits of technology

It improves the efficiency and accuracy of tree identification, enables real-time monitoring of tree growth and distribution in complex environments, reduces manual inspection workload, lowers costs, and ensures the safe operation of power transmission lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968283B_ABST
    Figure CN118968283B_ABST
Patent Text Reader

Abstract

This invention discloses an efficient intelligent identification method and system for typical trees in complex environments based on Feature Pyramid Network (FPN), belonging to the field of intelligent identification technology. The method includes data collection on trees along power transmission line corridors and preprocessing the collected data; constructing a Feature Pyramid Network model and training the model; evaluating the trained model; and identifying typical trees in complex environments. This invention effectively integrates feature information at different scales, thereby improving the accuracy and efficiency of typical tree identification. By learning feature representations under different environments, the robustness and generalization ability of the model are improved, thus maintaining high identification accuracy and efficiency under complex environments such as different lighting, weather, and tree densities. Through real-time monitoring and early warning, the risk of safety accidents caused by fallen trees and branches contacting power lines can be reduced, decreasing the workload of manual inspections and lowering labor costs; at the same time, it also improves the efficiency and accuracy of identification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, and in particular to an efficient intelligent recognition method and system for typical trees in complex environments based on FPN. Background Technology

[0002] Trees are one of the main obstacles in power transmission line corridors. Due to their vigorous growth, trees may touch the transmission lines, causing short circuits, tripping, or even fires. Therefore, regular monitoring and maintenance of trees in power transmission line corridors is crucial.

[0003] Traditional tree monitoring methods require manual inspections, which are time-consuming and inefficient, and cannot achieve comprehensive coverage and real-time monitoring of power transmission line corridors. In addition, manual inspections are also limited by natural conditions such as weather and terrain, making it difficult to deal with emergencies.

[0004] Feature Pyramid Network (FPN) is a deep learning architecture for object detection that extracts information from feature maps at different scales to effectively handle objects of varying sizes. FPN constructs multi-scale feature representations through top-down and lateral connections, which is particularly useful for identifying trees of different sizes in power transmission line corridors.

[0005] The FPN-based high-efficiency intelligent tree identification technology in complex environments aims to leverage the advantages of FPN and combine it with deep learning algorithms to develop a system capable of automatically identifying trees along power transmission line corridors. This system can be deployed on drones, surveillance cameras, or other mobile devices to automatically identify the location, size, and type of trees by capturing and analyzing images in real time, thereby providing decision support for the maintenance of power transmission line corridors. Utilizing advanced computer vision technology to improve the efficiency and accuracy of tree identification helps ensure the safe operation of power transmission lines. With continuous technological advancements, this intelligent identification technology is expected to play an even greater role in the future management of power transmission line corridors. Summary of the Invention

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] Therefore, the problem to be solved by this invention is: how to solve the problem that existing methods cannot achieve full coverage and real-time monitoring of power transmission line corridors.

[0008] To address the aforementioned technical problems, this invention provides the following technical solution: an efficient and intelligent method for identifying typical trees in complex environments based on Feature Pyramid Network (FPN), comprising: collecting data on trees along power transmission line corridors and preprocessing the collected data; constructing a Feature Pyramid Network model and training the model; evaluating the trained model and identifying typical trees in complex environments.

[0009] As a preferred embodiment of the efficient intelligent identification method for typical trees in complex environments based on FPN described in this invention, the data acquisition includes using a drone equipped with a high-definition camera or a fixed camera to collect data within the power transmission line corridor, with trees as the target object. The collected grayscale images cover different weather conditions, lighting conditions, and seasonal changes, and tree growth data is collected. The tree growth data includes the tree height and height at breast height.

[0010] As a preferred embodiment of the efficient intelligent tree recognition method for complex environments based on FPN described in this invention, the preprocessing of the collected data includes image format unification and preliminary standardization, dynamic range adjustment and illumination enhancement, occlusion handling, and data annotation to construct the final dataset; the image format unification and preliminary standardization includes converting all collected images to a uniform format and size, treating fixed-pattern noise as image background noise, collecting the original image (calibration image) in a completely dark environment, and subtracting each pixel value from the corresponding pixel value of the calibration image during subsequent normal image acquisition to eliminate fixed-pattern noise; the dynamic range adjustment and illumination enhancement includes globally adjusting image contrast to compensate for the impact of illumination changes on image quality, expressed as: in, The image after contrast adjustment. For the original image, The minimum pixel value in the image. This represents the maximum pixel value in the image. After contrast adjustment, the image is divided into multiple local regions. The brightness values ​​are adjusted appropriately based on the image's overall brightness to maintain a constant brightness despite changes in lighting conditions. The average brightness value of each local region is expressed as: in, This represents the average brightness value of the local area. This represents the number of rows in the local region. The number of columns in the local region. For the local region of the first OK, For the local region of the first List, For the image in The brightness value at that location; for the average brightness value of each local area, if... Brightness greater than the local area threshold This reduces the brightness of a local image, as shown below: in, This is an image of a local region after contrast adjustment; for the average brightness value of each local region, if... Less than the local area's brightness threshold This increases the brightness of the local image, as shown below: in, This is the image after brightness adjustment.

[0011] As a preferred embodiment of the efficient intelligent tree recognition method for complex environments based on FPN described in this invention, wherein: the brightness threshold of the local region is calculated using an adaptive thresholding algorithm within each local region; the occlusion processing includes enhancing the feature representation of the occluded region in other unoccluded regions by employing local feature enhancement, as follows: in, This is a local image after local feature enhancement. This is the enhancement coefficient.

[0012] As a preferred embodiment of the efficient intelligent tree recognition method for complex environments based on FPN described in this invention, the following steps are included: Data annotation includes labeling trees in an image using an image annotation tool. Before annotation, the annotation standards and rules are determined, including location and category information and the annotation format. For tree recognition tasks, a bounding box annotation method is used to draw a rectangular box on the image, enclosing the main body of the tree. The coordinates of the bounding box are represented by the pixel coordinates of the upper left and lower right corners. Next to the bounding box or in the corresponding annotation file, the category of the tree within the box is specified. If there is only one tree category, only the bounding box is labeled; if there are multiple categories, the category of each box needs to be explicitly labeled. The final dataset construction includes proportionally distributing the preprocessed data to the training set, validation set, and test set.

[0013] As a preferred embodiment of the efficient and intelligent tree recognition method for complex environments based on FPN described in this invention, the construction of the feature pyramid network model includes using a pre-trained deep convolutional neural network as a feature extractor. The feature extractor performs convolution operations on the input image to generate feature maps at different levels. The feature pyramid network model FPN utilizes the feature maps at different levels generated by the feature extractor to construct a feature pyramid through bottom-up, lateral connections, and top-down approaches. The bottom-up approach includes progressively extracting feature maps from the bottom layer to the top layer of the VGG network, with the feature maps serving as the input for the FPN from the bottom layer to the top layer. The top-down approach includes starting from the higher layers of the VGG network and progressively generating feature maps with a scale matching that of the lower layer through upsampling. The higher resolution feature maps; the lateral connections include combining low-level feature maps with high-level feature maps in a top-down process to form a feature pyramid; in each layer of the feature maps in FPN, a channel attention mechanism is introduced to optimize feature representation; the model training includes training the model using labeled data and updating the model parameters via backpropagation algorithm, inputting training data into the network, calculating the difference between the model output and the real label to obtain the value of the loss function, calculating the gradient based on the loss function, propagating the gradient of the output layer forward, calculating the gradient of each layer until the gradient of the loss function for each parameter is calculated, and using the calculated gradients, updating the model parameters using a gradient descent optimization algorithm.

[0014] As a preferred embodiment of the efficient intelligent tree recognition method for complex environments based on FPN described in this invention, the evaluation of the trained model includes verifying and evaluating the performance of the trained model by calculating accuracy, recall and F1 score, and optimizing the model based on the evaluation results; the optimization of the model includes adding regularization to optimize the model.

[0015] Another objective of this invention is to provide a system for efficient and intelligent identification of typical trees in complex environments based on FPN, which solves the problem of efficient and intelligent identification of typical trees in complex environments based on FPN by constructing a typical tree identification system for complex environments.

[0016] To address the aforementioned technical problems, this invention provides the following technical solution: an efficient and intelligent identification system for typical trees in complex environments based on Feature Pyramid Network (FPN), comprising a data acquisition and processing module, a model building module, and a model evaluation module; the data acquisition and processing module is used to acquire data on trees along power transmission line corridors and preprocess the acquired data; the model building module is used to construct a Feature Pyramid Network model and train the model; the model evaluation module is used to evaluate the trained model and identify typical trees in complex environments.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the efficient and intelligent tree recognition method for complex environments based on FPN as described above.

[0018] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the efficient and intelligent tree recognition method for complex environments based on FPN as described above.

[0019] The beneficial effects of this invention are as follows: The efficient and intelligent identification method for typical trees in complex environments based on FPN provided by this invention effectively integrates feature information at different scales, thereby improving the identification accuracy of typical trees. Even under complex environmental conditions, it can accurately identify tree species and locations. Compared with traditional manual inspection, FPN-based intelligent identification technology can greatly improve identification efficiency. It can quickly process large amounts of image data and provide identification results in real time, thereby reducing the workload and time cost of manual inspection. In complex natural environments, factors such as changes in light, occlusion, and varied tree postures can all make identification difficult. FPN-based identification technology can adapt to various complex environments. By learning feature representations under different environments, it improves the robustness and generalization ability of the model, thereby maintaining high identification accuracy and efficiency under complex environments such as different lighting, weather, and tree density. By integrating sensing devices such as drones or cameras, FPN-based intelligent identification technology can realize real-time monitoring of trees in power transmission line corridors. This means that managers can understand the growth and distribution of trees in the corridor at any time and promptly detect potential safety hazards. The real-time monitoring function can also help managers take timely measures such as pruning and felling to ensure the safe operation of power transmission lines. In applications such as power transmission corridors, this technology can promptly identify trees that may pose a threat to power facilities, providing strong support for risk warning and safety management. Real-time monitoring and early warning can reduce the risk of safety accidents caused by fallen trees or branches contacting power lines. FPN-based intelligent identification technology not only provides tree identification results but also extracts key growth parameters, such as tree height and crown diameter. These parameters provide strong support for management decision-making. Based on these parameters, scientific and reasonable vegetation management plans can be formulated, such as selecting suitable tree species and developing pruning schedules, thereby ensuring the rational distribution and growth of vegetation within power transmission corridors. Compared with traditional manual inspections, FPN-based intelligent identification technology can significantly reduce operating and maintenance costs. It reduces the workload of manual inspections, greatly lowering labor costs; at the same time, it improves the efficiency and accuracy of identification, and reduces additional costs and losses caused by misjudgments or omissions. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 The flowchart shows the efficient and intelligent tree recognition method for complex environments based on FPN provided in the first embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of the Feature Pyramid (FPN) for the efficient intelligent identification method of typical trees in complex environments based on FPN, provided in the first embodiment of the present invention.

[0023] Figure 3 The diagram shows the attention mechanism structure of the efficient intelligent tree recognition method for complex environments based on FPN provided in the first embodiment of the present invention.

[0024] Figure 4 A schematic diagram of the RetinaNet target detection algorithm for a typical tree recognition method in a complex environment based on FPN, provided in the first embodiment of the present invention.

[0025] Figure 5 This is a structural diagram of a typical tree efficient intelligent recognition system based on FPN in complex environments, provided in the second embodiment of the present invention. Detailed Implementation

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0028] Example 1 Reference Figures 1-4 This is the first embodiment of the present invention, which provides an efficient and intelligent method for identifying typical trees in complex environments based on FPN, including: collecting data on trees in power transmission line corridors and preprocessing the collected data; constructing a feature pyramid network model and training the model; evaluating the trained model and identifying typical trees in complex environments.

[0029] S1. Collect data on trees along the power transmission line corridor and preprocess the collected data.

[0030] Data collection involved using drones equipped with either high-definition or fixed cameras to collect data within the power transmission line corridor, targeting trees along the corridor. The collected grayscale images covered different weather conditions, lighting conditions, and seasonal variations, and tree growth data was also collected. Pixel values ​​in the grayscale images represent brightness values. Tree growth data included tree height and height at breast height (DBH).

[0031] Preprocessing includes image format unification and preliminary standardization, dynamic range adjustment and illumination enhancement, occlusion handling, data annotation, and final dataset construction.

[0032] Image format unification and preliminary standardization: convert all acquired images into a uniform format and size, treat fixed-pattern noise as image background noise, acquire the original image (calibration image) in a completely dark environment, and in the subsequent normal image acquisition process, subtract each pixel value from the corresponding pixel value of the calibration image to eliminate fixed-pattern noise.

[0033] Dynamic range adjustment and illumination enhancement, globally adjusting image contrast to compensate for the impact of illumination changes on image quality, are represented as follows: in, The image after contrast adjustment. For the original image, The minimum pixel value in the image. 255 is the maximum pixel value in the image; 255 is the maximum possible pixel value in an 8-bit image.

[0034] After contrast adjustment, the image is divided into multiple local regions. The brightness values ​​are adjusted appropriately based on the image's overall brightness to maintain a constant brightness despite changes in lighting conditions. The average brightness value of each local region is expressed as: in, This represents the average brightness value of the local area. This represents the number of rows in the local region. The number of columns in the local region. For the local region of the first OK, For the local region of the first List, For the image in The brightness value at that location.

[0035] For the average brightness value of each local area, if Brightness greater than the local area threshold This reduces the brightness of a local image, as shown below: in, This is an image of a local area after contrast adjustment.

[0036] For the average brightness value of each local area, if Less than the local area's brightness threshold This increases the brightness of the local image, as shown below: in, This is the image after brightness adjustment.

[0037] The brightness threshold for each local region is calculated using an adaptive thresholding algorithm. Adaptive thresholding algorithms include those based on neighborhood averages and those based on Gaussian weighting. For the neighborhood average-based algorithm, the threshold for each pixel is set based on the average value of its neighboring (local region) pixels. The average value (or weighted average value) of pixels within a certain range around each pixel is calculated, and then this average value is subtracted to serve as the threshold for that pixel, expressed as: in, The average value of the pixels. To adjust the parameters.

[0038] For Gaussian weighted algorithms, in regions of the image where brightness changes gradually, a Gaussian function is used to weight the pixels surrounding each pixel, and then the weighted average is calculated and fine-tuned as a threshold, expressed as: in, The weights are Gaussian, decreasing according to the distance from the center point. This represents the brightness value of the neighboring pixels. The standard deviation of the Gaussian function controls the rate at which the weights decrease.

[0039] Occlusion handling employs local feature enhancement to amplify the features of the occluded region within other unoccluded regions, compensating for the impact of occlusion recognition. This is represented as follows: in, This is a local image after local feature enhancement. This is the enhancement coefficient; it controls the degree of enhancement and is usually taken between 0.5 and 1.5.

[0040] Perform data annotation by using image annotation tools (such as LabelImg or CVAT) to label the trees in the image. Before annotation, determine the annotation standards and rules, including location and category information (different types of trees) and annotation format (bounding boxes, polygons, pixel-level annotations, etc.).

[0041] Bounding box annotation: For tree recognition tasks, bounding box annotation is commonly used. A rectangular box is drawn on the image to enclose the main part of the tree. The coordinates of the bounding box are usually represented as the pixel coordinates of the top-left and bottom-right corners.

[0042] Category Labeling: Next to the bounding box or in the corresponding labeling file, you need to specify the category of the trees within the box. If there is only one tree category, you only need to label the bounding box; if there are multiple categories, you need to explicitly label the category of each box. After labeling, ensure the accuracy and consistency of the labeling, and conduct multiple rounds of review and correction if necessary.

[0043] Dataset partitioning: The labeled dataset is divided into two parts. The majority of the dataset is used to train the model, a small portion is used to verify the model's performance during training, and the remaining data is used to evaluate the model's generalization ability.

[0044] The labeled data is stored in an appropriate format, such as XML, JSON, or CSV files, which are usually stored together with the image files to facilitate subsequent model training and evaluation.

[0045] S2. Construct a feature pyramid network model and train the model.

[0046] A pre-trained deep convolutional neural network (such as ResNet) is used as a feature extractor. The feature extractor performs convolution operations on the input image to generate feature maps at different levels. These feature maps have different spatial resolutions and semantic information, and can capture information at different scales in the image.

[0047] FPN utilizes feature maps generated by a feature extractor at different levels to construct a feature pyramid through bottom-up, lateral connections, and top-down approaches. For example... Figure 2 As shown.

[0048] Building an FPN deep learning model: Bottom-up path: Extract feature maps step by step from the bottom layer to the top layer of the VGG network. These feature maps will be used as inputs for the FPN from the bottom layer to the top layer.

[0049] Top-down approach: Starting from the higher layers of the VGG network, higher resolution feature maps are gradually generated by upsampling to match the feature scale of the lower layers.

[0050] Lateral connectivity: In the top-down process, low-level feature maps are combined with high-level feature maps to integrate spatial details and semantic information, forming a feature pyramid.

[0051] Traditional FPN achieves feature fusion through simple upsampling and lateral connections. However, in complex environments, this simple fusion method may be insufficient to capture the subtle features of trees. Therefore, a more advanced feature fusion mechanism, such as an attention mechanism, is introduced here, enabling the model to automatically learn and adjust the weights between features at different scales. In addition, a global context module is introduced to enhance the model's adaptability to complex environments, thereby improving the accuracy of tree recognition.

[0052] An attention mechanism is introduced: At each layer of the feature map in the FPN, a channel attention mechanism is introduced to optimize the feature representation. Through steps such as global pooling, feature learning, feature fusion, and sigmoid activation, an attention weight is generated for each channel. The generated attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted feature map. This mechanism can adaptively adjust the weights between different channels, emphasizing useful feature channels for tree recognition while suppressing irrelevant or redundant channels. Figure 3 As shown.

[0053] Introducing Global Context Modules: Global context modules are introduced at different levels of the FPN to capture multi-scale contextual information. This can be achieved by constructing a multi-scale feature fusion structure or using other context extraction techniques. The captured global contextual information is integrated with the feature maps of the FPN to enhance the model's understanding of complex environments.

[0054] In each feature map layer of the FPN, object detection algorithms are used to generate bounding boxes for trees. To detect trees quickly, efficiently, and accurately, especially in complex environments or when trees vary significantly in size, RetinaNet uses a dense object detector to detect objects. For example... Figure 4 As shown.

[0055] RetinaNet Object Detection Algorithm: RetinaNet is a single-stage object detection algorithm designed to address the imbalance between positive and negative samples. Combined with FPN, RetinaNet generates anchor boxes on feature maps at multiple scales with different aspect ratios to cover targets of varying sizes. For each anchor box, the dense object detector predicts a class score (indicating the presence and class of the target within the anchor box) and a bounding box regression vector (used to adjust the anchor box's position and size for a more accurate fit to the target). It also uses Focal Loss to address the common positive-negative sample imbalance problem in single-stage object detection algorithms.

[0056] After processing by the dense object detector, a large number of predicted bounding boxes and corresponding class scores are obtained.

[0057] At each level of the feature pyramid, a detection head is applied to predict the class probability and bounding box coordinates of the target. For the bounding boxes output by each detection head, non-maximum suppression (NMS) is used to eliminate redundant bounding boxes that are detected repeatedly for the same tree, while retaining the best prediction result. Specific steps include sorting, selecting the highest confidence box, calculating IoU, and suppressing overlapping boxes.

[0058] Non-maximum suppression (NMS): The main purpose of non-maximum suppression (NMS) is to select the best bounding box from multiple overlapping candidate bounding boxes. In object detection, models typically generate multiple bounding boxes that may contain the same object. NMS determines which bounding box should be retained by comparing the degree of overlap between the bounding boxes (usually measured by intersection and joint ratio IoU) and the confidence of each bounding box, thereby removing redundant bounding boxes from these overlapping bounding boxes and retaining the best bounding box.

[0059] Step 1: First, sort all candidate edges in descending order according to the confidence level of each border, that is, the edge with the highest confidence level is placed at the front.

[0060] Step 2: Create an empty list to store the final retained bounding boxes.

[0061] Step 3: Select the bounding box with the highest confidence level and add it to the keep list.

[0062] Step 4: For the remaining bounding boxes, calculate their intersection and inter-boundary ratio (IoU) with the current best bounding box. If the IoU exceeds a preset threshold (e.g., 0.5), it is assumed that both boxes have detected the same target, so the box with lower confidence will be suppressed (i.e., discarded).

[0063] Step 5: Repeat steps 3 and 4 until all bounding boxes have been checked. In each iteration, the box with the highest confidence among the currently unprocessed boxes is selected as the best box, and other boxes that overlap with it by more than a threshold are suppressed.

[0064] Step 6: Finally, all bounding boxes in the list will be output as the detection results.

[0065] A deep learning model based on Feature Pyramid Network (FPN) was constructed. By combining feature maps from different network layers, the detection accuracy of tree targets was improved, enabling the model to capture both fine-grained (high-resolution) and coarse-grained (low-resolution) features of the image. During target detection, appropriate feature layers were selected based on the tree size. Large trees used high-level features to capture semantics, while small trees used low-level features to preserve details. In this way, the model effectively utilizes the Feature Pyramid Network (FPN) to accurately detect and classify trees at multiple scales. The model training process included training the model using labeled data and updating the model parameters via backpropagation. Training data was input into the network, and the difference between the model output and the ground truth labels was calculated to obtain the loss function value. The gradient of the loss function was calculated, and the gradient of the output layer was propagated forward, calculating the gradient of each layer step by step until the gradient of the loss function for each parameter was calculated. Finally, the model parameters were updated using a gradient descent optimization algorithm based on the calculated gradients.

[0066] The model is initialized with pre-trained base network weights to accelerate convergence and improve performance. Training is performed on a labeled dataset over multiple epochs, each epoch including forward propagation, loss function calculation, backpropagation, and parameter updates. Model performance on the validation set is also monitored; training stops once performance no longer improves to avoid overfitting.

[0067] Loss function definition: To address the imbalance between positive and negative samples, the Focal Loss function is introduced. A modulation factor is used to reduce the weight of easily classified samples, causing the model to focus more on difficult-to-classify samples during training. This is expressed as: in, For loss function, These are the class weights (used to balance positive and negative samples). To predict the probability that a sample belongs to the true class for the model. To focus the parameters (used to control the weights of easy and difficult samples), during training, the model outputs a predicted probability for each sample. Then, the loss value is calculated according to the focus loss formula.

[0068] Training Process: In each training batch, labeled image data is sent to the RetinaNet model for forward propagation to obtain prediction results. The Focal Loss function is used to calculate the difference between the predicted result and the actual label to obtain the loss value. The model parameters are updated based on the loss value using the backpropagation algorithm to optimize model performance. The above steps are repeated iteratively until the model reaches a satisfactory performance state.

[0069] Training Strategy: Optimizer Selection: When training the FPN model, we chose Adam as the optimization algorithm to update the model parameters. Learning Rate Adjustment: Set an appropriate learning rate and use learning rate decay or learning rate warm-up strategies during training. Batch Size: Determine the number of images used in each iteration, which usually requires a trade-off between computational resources and convergence speed. The Adam algorithm combines momentum and adaptive learning rate features, making it suitable for deep learning tasks and capable of adaptively adjusting the learning rate on different parameters. The Adam optimizer is created using the `torch.optim.Adam()` function. Hyperparameters such as the learning rate and weight decay can be set.

[0070] Learning rate warm-up strategy: To ensure more stable model training and avoid instability in the early stages of training, a learning rate warm-up strategy is employed. Specifically, at the start of training, a small initial learning rate ηstart is chosen. Over the first few epochs, the learning rate is gradually increased until the set target learning rate ηend is reached. After the warm-up phase, regular training is performed using the set target learning rate. The learning rate warm-up strategy effectively maintains model training stability and improves the model's convergence rate, especially having a significant impact on the model at the very beginning of training.

[0071] Training loop: The model is trained using labeled data, and its parameters are updated via backpropagation. Forward propagation: Training data is input into the network, and the difference between the model output and the true labels is calculated to obtain the loss function. Backpropagation: The gradient of the loss function with respect to the output layer is calculated, and the gradient of the output layer is propagated forward, calculating the gradient of each layer step by step until the gradient of the loss function for each parameter is calculated. Using the calculated gradients, optimization algorithms such as gradient descent are employed to update the model parameters.

[0072] The calculation process of backpropagation can be expressed by the following mathematical formula: Calculate the gradient of the output layer: Backpropagation gradient: Parameter update: Parameter update: in, This represents the error term of the l-th layer. and These represent the weights and biases of the l-th layer, respectively. Indicates the learning rate. Represents the loss function. This indicates element-wise multiplication. This represents the derivative of the activation function.

[0073] Validation: Evaluate model performance on the validation set and monitor loss and accuracy metrics.

[0074] Model tuning: Adjust the model structure, hyperparameters, or training strategy based on the validation results.

[0075] Through these methods, the FPN model is ensured to learn the feature information in tree images stably and effectively during training, thereby achieving accurate detection of tree targets.

[0076] S3. Evaluate the trained model and identify typical trees in complex environments.

[0077] The trained model is validated and its performance is evaluated by calculating metrics such as accuracy, recall, and F1 score, and the model is optimized based on the evaluation results.

[0078] Accuracy refers to the proportion of samples correctly predicted as positive by the classifier out of the total number of samples correctly predicted as positive. The calculation formula is as follows: in, This indicates a true positive instance. This indicates a false positive. This indicates the number of positive class samples that the model successfully predicts as positive. This indicates the number of times the model incorrectly predicts a negative class as a positive class.

[0079] Recall is the proportion of samples correctly predicted as positive by the classifier out of the total number of samples that are actually positive. The formula is as follows: in, This indicates a true positive. This indicates a false negative. This indicates the number of positive class samples that the model successfully predicted as positive. This indicates the number of times the model incorrectly predicts a positive class as a negative class.

[0080] The F1 score is a comprehensive evaluation metric that considers both accuracy and recall, making it particularly suitable for handling imbalanced datasets. The calculation formula is as follows: By calculating accuracy, recall, and F1 score, the performance of a classification model can be evaluated more comprehensively.

[0081] Further optimization of the model based on the evaluation results may include adjusting the feature pyramid structure, adding regularization (such as Dropout), and using more sophisticated data augmentation strategies.

[0082] To avoid the impact of noise on model recognition and reduce the model's over-reliance on training data, thereby improving the model's generalization ability, regularization techniques can be applied. This involves adding a penalty term related to model complexity to the loss function, thus helping the model improve its robustness to noise.

[0083] L1 regularization: By adding L1 norm constraints to the model parameters, some parameters tend to become sparse, thereby achieving the effect of feature selection. This helps to handle the complexity of the model and improves the model's generalization ability.

[0084] L2 regularization, also known as weight decay, adds L2 norm constraints to model parameters, causing them to tend towards smaller values. This helps reduce model complexity and avoid overfitting. In deep learning, L2 regularization is often used in conjunction with weight decay, which is achieved by directly reducing the weights during gradient updates.

[0085] Dropout: During training, a subset of neurons are randomly dropped, ensuring the model encounters different network structures in each iteration. This method reduces the correlation between neurons and the model's dependence on specific neurons, thereby improving the model's ability to generalize to unfamiliar data.

[0086] The process of adjusting the feature pyramid structure involves multiple steps, including the selection of the number of pyramid layers, the generation and fusion of feature maps, model design and training, and model evaluation and optimization. It requires comprehensive consideration of the specific needs and actual situation of the object detection or image segmentation task.

[0087] In addition, the model is optimized to meet the hardware and performance requirements of real-world environments, including techniques such as model compression, quantization, and acceleration, in order to improve the model's speed and performance.

[0088] Example 2 Reference Figure 5 This is the second embodiment of the present invention, which differs from the previous embodiment in that it provides an efficient intelligent identification system for typical trees in complex environments based on FPN, including: a data acquisition and processing module, a model building module, and a model evaluation module.

[0089] The data acquisition and processing module is used to collect data on trees along the power transmission line corridor and to preprocess the collected data.

[0090] The model building module is used to build the feature pyramid network model and train the model.

[0091] The model evaluation module is used to evaluate the trained model and identify typical trees in complex environments.

[0092] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0094] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0095] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0096] Example 3 The third embodiment of the present invention differs from the first two embodiments in that it is used to verify and explain the technical effects adopted in the present invention, so as to verify the real effect of the method.

[0097] Method of the present invention: Data Acquisition: Using drones equipped with high-definition cameras, 1,000 grayscale images were collected within the power transmission line corridor, covering different weather conditions, lighting conditions, and seasonal changes.

[0098] Data preprocessing: image format unification and standardization, dynamic range adjustment and illumination enhancement, occlusion handling, data annotation, and construction of the final dataset.

[0099] Model building and training: A feature pyramid network model is built using a pre-trained deep convolutional neural network as a feature extractor and then trained.

[0100] Model evaluation and optimization: The model performance is evaluated using metrics such as accuracy, recall, and F1 score, and the model is optimized based on the evaluation results.

[0101] Testing environment: The model was evaluated using a separate test set, which included images under different lighting, weather, and seasonal conditions.

[0102] Simulation conditions for traditional manual inspection methods: Data collection: Manual on-site inspections are conducted within the transmission line corridor to collect tree data.

[0103] Working hours: Each working day is calculated as 8 hours. It is estimated that it will take 9 working days (approximately 72 hours) to complete the manual inspection of 1,000 images.

[0104] Environmental adaptability: Due to limitations imposed by natural conditions such as weather and terrain, inspection efficiency may decrease due to severe weather or complex terrain.

[0105] Recognition accuracy: Due to human factors, the recognition accuracy is low and the false judgment rate is high.

[0106] This embodiment uses both the conventional method and our invented method for simultaneous detection, and the comparison results are shown in the table below: Table 1 Overall Comparison Table

[0107] The comparative data above clearly demonstrates the significant advantages of the method of this invention in terms of identification accuracy, processing time, environmental adaptability, labor costs, coverage, real-time performance, false positive rate, and maintenance costs. These advantages effectively improve the efficiency and accuracy of tree monitoring along power transmission line corridors, contributing to ensuring the safe operation of power transmission lines.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for efficient intelligent recognition of typical trees in complex environments based on FPN, characterized in that: include, Data was collected from trees along the power transmission line corridor, and the collected data was preprocessed. Construct a feature pyramid network model and train the model; The trained model is evaluated to identify typical trees in complex environments; The data collection includes using drones equipped with high-definition or fixed cameras to collect data within the power transmission line corridor, targeting trees along the corridor. The collected grayscale images cover different weather conditions, lighting conditions, and seasonal changes, and tree growth data is also collected. The tree growth data includes tree height and height at breast height; The preprocessing of the collected data includes image format unification and preliminary standardization, dynamic range adjustment and illumination enhancement, occlusion processing, data annotation, and final dataset construction. The image format unification and preliminary standardization include converting all acquired images into a uniform format and size, treating fixed-pattern noise as image background noise, acquiring the original image (calibration image) in a completely dark environment, and subtracting each pixel value from the corresponding pixel value of the calibration image during subsequent normal image acquisition to eliminate fixed-pattern noise. The dynamic range adjustment and illumination enhancement include globally adjusting image contrast to compensate for the impact of illumination changes on image quality, expressed as follows: wherein, is the image after contrast adjustment, is the original image, is the minimum pixel value in the image, is the maximum pixel value in the image; After contrast adjustment, the image is divided into multiple local regions. The brightness values ​​are adjusted appropriately based on the image's overall brightness to maintain a constant brightness despite changes in lighting conditions. The average brightness value of each local region is represented as... wherein, is the average luminance value of the local region, is the number of rows of the local region, is the number of columns of the local region, is the first row of the local region, is the first column of the local region, is the luminance value of the image at ; For the average brightness value of each local area, if Brightness greater than the local area threshold This reduces the brightness of the local image, as shown below. in, This is an image of a local area after contrast adjustment; For the average brightness value of each local area, if Less than the local area's brightness threshold This increases the brightness of the local image, represented as, in, This is the image after brightness adjustment; The brightness threshold of each local region is calculated using an adaptive thresholding algorithm within that local region. The occlusion processing includes using local feature enhancement to enhance the feature representation of the occluded region in other unoccluded regions. in, This is a local image after local feature enhancement. For enhancement coefficient; The data annotation includes using image annotation tools to annotate trees in the image. Before annotation, the annotation standards and rules are determined, including location and category information and annotation format. For tree recognition tasks, the bounding box annotation method is used to draw a rectangle on the image to enclose the main part of the tree. The coordinates of the bounding box are represented by the pixel coordinates of the top left and bottom right corners. Next to the bounding box or in the corresponding annotation file, specify the category of the trees within the box. If there is only one tree category, only the bounding box is annotated. If there are multiple categories, the category of each box needs to be explicitly annotated. The final dataset construction includes proportionally allocating the preprocessed data to the training set, validation set, and test set; The construction of the feature pyramid network model includes using a pre-trained deep convolutional neural network as a feature extractor. The feature extractor performs convolution operations on the input image to generate feature maps at different levels. The Feature Pyramid Network (FPN) model utilizes feature maps generated by a feature extractor at different levels to construct a feature pyramid through bottom-up, lateral connections, and top-down approaches. The bottom-up approach includes extracting feature maps step by step from the bottom layer to the top layer of the VGG network, and the feature maps serve as the input for the FPN from the bottom layer to the top layer. The top-down approach includes starting from the higher layers of the VGG network and gradually generating higher-resolution feature maps that match the feature scale of the lower layers through upsampling. The lateral connection includes combining low-level feature maps with high-level feature maps in a top-down process to form a feature pyramid; In each layer of the feature map in FPN, a channel attention mechanism is introduced to optimize the feature representation; The training of the model includes: training the model using labeled data and updating the model parameters via backpropagation algorithm; inputting training data into the network; calculating the difference between the model output and the real label to obtain the value of the loss function; calculating the gradient based on the loss function; propagating the gradient of the output layer forward; calculating the gradient of each layer until the gradient of the loss function for each parameter is calculated; and using the calculated gradient, updating the model parameters using gradient descent optimization algorithm. A feature fusion mechanism is introduced to enable the model to automatically learn and adjust the weights between features at different scales; a global context module is introduced to enhance the model's adaptability to complex environments. An attention mechanism is introduced in each layer of the feature map in FPN to optimize the feature representation. Through global pooling, feature learning, feature fusion and sigmoid activation steps, an attention weight is generated for each channel. The generated attention weight is multiplied with each channel of the original feature map to obtain the attention-weighted feature map. A global context module is introduced at different levels of FPN to capture multi-scale contextual information. The captured global contextual information is then integrated with the feature maps of FPN to enhance the model's understanding of complex environments.

2. The efficient and intelligent tree identification method for complex environments based on FPN as described in claim 1, characterized in that: The evaluation of the trained model includes verifying and evaluating the performance of the trained model by calculating accuracy, recall and F1 score, and optimizing the model based on the evaluation results. The optimization of the model includes adding regularization to optimize the model.

3. A system employing the efficient and intelligent tree recognition method based on FPN in complex environments as described in any one of claims 1 to 2, characterized in that: It includes a data acquisition and processing module, a model building module, and a model evaluation module; The data acquisition and processing module is used to collect data on trees along the power transmission line corridor and to preprocess the collected data. The model building module is used to build a feature pyramid network model and train the model; The model evaluation module is used to evaluate the trained model and identify typical trees in complex environments.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the efficient and intelligent tree recognition method for complex environments based on FPN as described in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the efficient and intelligent tree recognition method for complex environments based on FPN as described in any one of claims 1 to 2.

Citation Information

Patent Citations

  • Target detection and classification method and computer readable storage medium

    CN113887602A

  • Method and system for identifying single tree in power transmission corridor based on hierarchical graph algorithm

    CN117576565A