Model training method based on deep learning and transmission tower corrosion detection method

Abstract

The invention discloses a model training method based on deep learning and a transmission tower corrosion detection method, and the method comprises the steps: constructing an original sample image set of a transmission tower, marking the original sample image set according to the corrosion level, and obtaining a first sample image set; preprocessing the first sample image set to obtain a second sample image set; a preset deep learning model is trained according to the second sample image set, a trained deep learning model is obtained, and a network architecture of the deep learning model comprises Faster R-CNN, ResNet, U-Net and ViT; and optimizing the trained deep learning model to obtain a corrosion detection model, and carrying out transmission tower corrosion detection according to the corrosion detection model. Therefore, the image features can be automatically learned, manual intervention is not needed, the method can adapt to the image features under complex backgrounds and different illumination conditions, and the accuracy and robustness of transmission tower corrosion detection are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and transmission tower detection technology, and in particular to a model training method based on deep learning, a transmission tower corrosion detection method, device, computer-readable storage medium and terminal equipment. Background Art

[0002] In the power transmission system, the transmission tower is the physical structure responsible for carrying the high-voltage power transmission line, and is an important infrastructure supporting the transmission line. Due to long-term exposure to the outdoor environment, the transmission tower is often corroded by natural environmental factors such as wind, rain, moisture, and salt spray. Especially in some highly corrosive environments, the metal materials of the transmission tower are easily corroded, resulting in a decrease in its structural strength, thereby affecting the safety and stability of the transmission line. In addition, the corrosion problem of the transmission tower will not only shorten the service life of the tower and increase maintenance costs, but may also cause major safety hazards and even lead to large-scale power outages. Therefore, timely and effective detection and identification of the corrosion of the transmission tower is of great significance to ensure the safe operation of the power system.

[0003] Traditional transmission tower corrosion detection mainly relies on manual inspection and simple image processing technology. Manual inspection is a regular on-site inspection of transmission towers by professionals, who judge the degree of corrosion based on experience. Although this method can play a monitoring role to a certain extent, it has significant limitations. For example, manual inspection is limited by environmental conditions and human resources, which is not only costly but also inefficient. Especially in remote areas or complex terrain environments, manual inspection is difficult to perform. In addition, due to the fatigue and experience of the inspectors, the accuracy of the judgment results is difficult to guarantee. In addition, the periodic arrangement of manual inspections makes it difficult to achieve real-time monitoring of tower corrosion, which may lead to potential risks not being discovered in time and then accumulating into disasters.

[0004] Traditional automated detection methods based on image processing have improved the limitations of manual inspections to a certain extent. Especially with the development of drone technology, using drones equipped with cameras to collect images of transmission towers has become a new means of monitoring. Automated detection methods based on image processing usually include steps such as image acquisition, image preprocessing, feature extraction and classification. By analyzing the corrosion features (such as rust, cracks, etc.) on the surface of the transmission tower, the corrosion situation can be detected and identified. However, traditional image processing technology often relies on manually designed feature extraction rules and cannot effectively cope with the changes in the morphology, color, texture and other diverse features of transmission tower corrosion. In addition, such methods have poor image processing capabilities under different lighting conditions and complex backgrounds, and are difficult to cope with the corrosion detection tasks of transmission towers under different lighting, shadows and background interference, thus affecting the accuracy and robustness of corrosion detection. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a model training method based on deep learning, a transmission tower corrosion detection method, device, computer-readable storage medium and terminal equipment, which can automatically learn image features without human intervention, and can adapt to image features under complex backgrounds and different lighting conditions, thereby solving the problems existing in traditional manual inspections and automated detection based on image processing, and effectively improving the accuracy and robustness of transmission tower corrosion detection.

[0006] In order to achieve the above objectives, the first aspect of the present invention provides a model training method based on deep learning, comprising:

[0007] Constructing an original sample image set of transmission towers;

[0008] Annotate each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set;

[0009] Preprocessing each first sample image in the first sample image set to obtain a second sample image set;

[0010] Training a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes Faster R-CNN, ResNet, U-Net and ViT;

[0011] The trained deep learning model is optimized to obtain a corrosion detection model, so as to perform transmission tower corrosion detection according to the corrosion detection model.

[0012] Furthermore, the preprocessing of each first sample image in the first sample image set to obtain the second sample image set specifically includes:

[0013] Each first sample image in the first sample image set is resized, normalized, and data enhanced to obtain a second sample image set.

[0014] Furthermore, the step of training a preset deep learning model according to the second sample image set to obtain a trained deep learning model specifically includes:

[0015] Using Faster R-CNN to perform corrosion area detection on the second sample image set to obtain corrosion candidate areas; wherein the feature extractor of Faster R-CNN is ResNet;

[0016] Using ResNet to extract deep features of the corrosion candidate area to obtain deep features of the corrosion area;

[0017] Using a channel attention mechanism, assigning different weights to each channel of a feature map constructed based on the depth features of the corrosion area;

[0018] The U-Net is used to segment the corrosion region based on the weighted corrosion region depth features to obtain the corrosion region segmentation mask; wherein the encoder part of the U-Net is ResNet;

[0019] Using ViT to classify the corrosion level of the corrosion area segmentation mask to obtain a corrosion level classification result;

[0020] According to the corrosion grade classification result and the annotated data of the second sample image set, a preset joint loss function is used to adjust the model parameters to obtain a trained deep learning model.

[0021] Furthermore, L total =α*L loc +β*L cls +γ*L seg , where L total represents the joint loss function, L loc represents the position loss, L cls represents the classification loss, L seg represents segmentation loss, α, β and γ represent weight coefficients, and the position loss is L1 loss or smooth L1 loss, the classification loss is cross entropy loss or focal loss, and the segmentation loss is Dice loss or cross entropy loss.

[0022] Furthermore, the training deep learning model is optimized to obtain a corrosion detection model, specifically including:

[0023] Performing a performance evaluation on the trained deep learning model using k-fold cross validation to obtain a first evaluation result;

[0024] Using precision, recall and IoU indicators to perform performance evaluation on the trained deep learning model to obtain a second evaluation result;

[0025] The trained deep learning model is optimized according to the first evaluation result and the second evaluation result to obtain a corrosion detection model.

[0026] In order to achieve the above-mentioned object, a second aspect of the present invention provides a transmission tower corrosion detection method based on deep learning, comprising:

[0027] Acquire an image of a transmission tower to be inspected;

[0028] Preprocessing the image to be detected;

[0029] Performing corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain a corrosion detection result of the transmission tower to be detected;

[0030] Wherein, the corrosion detection model is obtained by training using the deep learning-based model training method described in any one of the first aspects above.

[0031] In order to achieve the above-mentioned purpose, the third aspect of the present invention further provides a model training device based on deep learning, comprising:

[0032] A sample image set construction module is used to construct an original sample image set of a transmission tower;

[0033] A sample image set annotation module is used to annotate each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set;

[0034] A sample image set preprocessing module, used for preprocessing each first sample image in the first sample image set to obtain a second sample image set;

[0035] A deep learning model training module, used to train a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes FasterR-CNN, ResNet, U-Net and ViT;

[0036] The deep learning model optimization module is used to optimize the trained deep learning model to obtain a corrosion detection model so as to perform transmission tower corrosion detection according to the corrosion detection model.

[0037] In order to achieve the above-mentioned object, the fourth aspect of the present invention further provides a transmission tower corrosion detection device based on deep learning, comprising:

[0038] The module for acquiring the image to be detected is used to acquire the image to be detected of the transmission tower to be detected;

[0039] A preprocessing module for the image to be detected, used for preprocessing the image to be detected;

[0040] The transmission tower corrosion detection module is used to perform corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain the corrosion detection result of the transmission tower to be detected;

[0041] Wherein, the corrosion detection model is obtained by training using the deep learning-based model training method described in any one of the first aspects above.

[0042] The fifth aspect embodiment of the present invention also provides a computer-readable storage medium, including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the deep learning-based model training method described in any one of the first aspects above, or to execute the deep learning-based transmission tower corrosion detection method described in any one of the second aspects.

[0043] The sixth aspect embodiment of the present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when executing the computer program, the processor implements the deep learning-based model training method described in any one of the first aspect above, or implements the deep learning-based transmission tower corrosion detection method described in any one of the second aspect.

[0044] Compared with the prior art, the embodiments of the present invention provide a model training method based on deep learning, a transmission tower corrosion detection method, an apparatus, a computer-readable storage medium and a terminal device, wherein the training method comprises: constructing an original sample image set of a transmission tower; annotating each original sample image in the original sample image set according to a pre-divided corrosion level to obtain a first sample image set; preprocessing each first sample image in the first sample image set to obtain a second sample image set; training a preset deep learning model according to the second sample image set to obtain a trained deep learning model, wherein the network architecture of the deep learning model comprises Faster R-CNN, ResNet, U-Net and ViT; optimizing the trained deep learning model to obtain a corrosion detection model, so as to perform transmission tower corrosion detection according to the corrosion detection model. The embodiment of the present invention combines the positioning capability of Faster R-CNN, the feature extraction capability of ResNet, the segmentation capability of U-Net and the context analysis capability of ViT to form a multi-stage corrosion detection model based on deep learning. By introducing deep learning into the corrosion detection of transmission towers and constructing a multi-layer neural network, image features can be automatically learned from images without human intervention, and can also adapt to image features under complex backgrounds and different lighting conditions, thereby solving the problems existing in traditional manual inspections and automated detection based on image processing, and effectively improving the accuracy and robustness of transmission tower corrosion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of a model training method based on deep learning provided by an embodiment of the first aspect of the present invention;

[0046] Figure 2 It is a flow chart of a transmission tower corrosion detection method based on deep learning provided by an embodiment of the second aspect of the present invention;

[0047] Figure 3 It is a structural block diagram of a model training device based on deep learning provided by an embodiment of the third aspect of the present invention;

[0048] Figure 4 It is a structural block diagram of a transmission tower corrosion detection device based on deep learning provided by an embodiment of the fourth aspect of the present invention;

[0049] Figure 5 It is a structural block diagram of a terminal device provided in an embodiment of the sixth aspect of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this technical field without creative work are within the scope of protection of the present invention.

[0051] The first embodiment of the present invention provides a model training method based on deep learning, see Figure 1 , is a flowchart of a model training method based on deep learning provided by an embodiment of the first aspect of the present invention, the method comprising steps S11 to S15:

[0052] Step S11, constructing an original sample image set of a transmission tower;

[0053] Step S12, labeling each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set;

[0054] Step S13, preprocessing each first sample image in the first sample image set to obtain a second sample image set;

[0055] Step S14: training a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes Faster R-CNN, ResNet, U-Net and ViT;

[0056] Step S15: Optimize the trained deep learning model to obtain a corrosion detection model, so as to perform transmission tower corrosion detection according to the corrosion detection model.

[0057] Specifically, first, a large number of transmission tower images with different degrees of corrosion are collected to construct an original sample image set of the transmission tower; then, according to the pre-divided corrosion level, the corrosion area and corrosion level of each original sample image in the constructed original sample image set are respectively annotated, and the annotated original sample image is obtained, which is recorded as the first sample image, then, all the obtained first sample images are collected together to obtain the first sample image set; then, each first sample image in the obtained first sample image set is preprocessed, and the preprocessed first sample image is obtained, which is recorded as the second sample image, then, all the obtained second sample images are collected together to obtain the second sample image set; then, according to the obtained second sample image set, a pre-set deep learning model (the network architecture of the deep learning model includes Faster R-CNN, ResNet, U-Net and ViT) are trained to obtain a trained deep learning model; finally, the trained deep learning model is optimized to obtain an optimized deep learning model, and the optimized deep learning model is used as a corrosion detection model to perform transmission tower corrosion detection according to the obtained corrosion detection model, so as to detect the corrosion level of the transmission tower.

[0058] It should be noted that the corrosion level (i.e., degree of corrosion) of transmission towers can be divided according to the rust characteristics of the transmission tower surface and the integrity of the coating. For example, it can be divided into five levels:

[0059] Level Ⅰ (no corrosion): The surface is smooth, with no signs of rust or corrosion; the coating is intact, not peeling or damaged;

[0060] Level II (minor corrosion): There is a small amount of rust on the surface (such as pitting), but it does not affect the structural safety; the coating has slight peeling or wear, but still has a certain protective effect;

[0061] Level III (moderate corrosion): obvious rust appears on the surface, the coverage area increases, and there may be slight corrosion pits; the coating peels off a lot, affecting the protection effect, and maintenance or repair is required;

[0062] Level IV (severe corrosion): large areas of surface rust, obvious corrosion pits, which may affect the bearing capacity of the structure; the coating peels off severely, and the protective effect is basically lost;

[0063] Level V (extremely severe corrosion): The surface is severely corroded and the structure may have been damaged or failed.

[0064] It is understandable that when the corrosion area and corrosion level are marked for each original sample image in the original sample image set, manual marking may be used, or other applicable marking methods may be used, which is not specifically limited in the embodiment of the present invention.

[0065] A model training method based on deep learning is provided in an embodiment of the first aspect of the present invention, which combines the positioning capability of FasterR-CNN, the feature extraction capability of ResNet, the segmentation capability of U-Net and the context analysis capability of ViT to form a multi-stage corrosion detection model based on deep learning. By introducing deep learning into transmission tower corrosion detection and constructing a multi-layer neural network, it can automatically learn image features from images without human intervention, and exhibits superior performance in tasks such as image classification and target detection. It can also adapt to image features under complex backgrounds and different lighting conditions, thereby solving the problems existing in traditional manual inspections and automated detection based on image processing, and effectively improving the accuracy and robustness of transmission tower corrosion detection.

[0066] In one optional embodiment, preprocessing each first sample image in the first sample image set to obtain the second sample image set specifically includes:

[0067] Each first sample image in the first sample image set is resized, normalized, and data enhanced to obtain a second sample image set.

[0068] Specifically, in combination with the above embodiment, when preprocessing each first sample image in the first sample image set, each first sample image can be resized, normalized, and data enhanced (for example, rotated, cropped, etc.) respectively, and a corresponding preprocessed first sample image is obtained and recorded as a second sample image. Then, all the obtained second sample images are grouped together to obtain the second sample image set.

[0069] It should be noted that through image preprocessing, the image quality can be enhanced, noise and interference can be reduced, thereby enhancing the robustness of the final model, and the image preprocessing step is usually performed after the image annotation step. This is because image annotation needs to be performed on the original image without any modification to ensure the accuracy of the annotation, thereby further improving the accuracy of subsequent transmission tower corrosion detection.

[0070] In one optional embodiment, training a preset deep learning model according to the second sample image set to obtain a trained deep learning model specifically includes:

[0071] Using Faster R-CNN to perform corrosion area detection on the second sample image set to obtain corrosion candidate areas; wherein the feature extractor of Faster R-CNN is ResNet;

[0072] Using ResNet to extract deep features of the corrosion candidate area to obtain deep features of the corrosion area;

[0073] Using a channel attention mechanism, assigning different weights to each channel of a feature map constructed based on the depth features of the corrosion area;

[0074] The U-Net is used to segment the corrosion region based on the weighted corrosion region depth features to obtain the corrosion region segmentation mask; wherein the encoder part of the U-Net is ResNet;

[0075] Using ViT to classify the corrosion level of the corrosion area segmentation mask to obtain a corrosion level classification result;

[0076] According to the corrosion grade classification result and the annotated data of the second sample image set, a preset joint loss function is used to adjust the model parameters to obtain a trained deep learning model.

[0077] Specifically, in combination with the above embodiments, based on the network architecture of the deep learning model (including Faster R-CNN, ResNet, U-Net and ViT), when the preset deep learning model is trained according to the second sample image set, the training process is as follows:

[0078] 1. Input the obtained second sample image set into Faster R-CNN, and use Faster R-CNN to detect corrosion areas to locate and identify possible corrosion areas, and obtain corrosion candidate areas accordingly; it should be noted that:

[0079] (1.1) In an embodiment of the present invention, a pre-trained ResNet (e.g., ResNet-101, etc.) is used as a feature extractor for Faster R-CNN. In a specific implementation, the pre-trained ResNet weights on ImageNet can be loaded based on the deep learning PyTorch framework and used as a feature extractor for Faster R-CNN. This process helps to utilize the pre-trained ResNet weights, reduce the model training time, and improve the initial performance of the model.

[0080] (1.2) Based on the second sample image set input, the feature extractor ResNet of Faster R-CNN is first used to extract features, and then the extracted features are input into the RPN (Region Proposal Network) of Faster R-CNN for corrosion region detection, so as to generate candidate boxes that may contain corrosion regions (i.e., obtain corrosion candidate regions); by combining the features extracted by ResNet, RPN can generate more accurate candidate boxes.

[0081] 2. Input the detected corrosion candidate area into ResNet (for example, ResNet-101, etc.), use ResNet to extract deep features, obtain the corresponding deep features of the corrosion area, and combine the channel attention mechanism to assign different weights to each channel of the feature map constructed based on the deep features of the corrosion area, so as to highlight the key corrosion features in the deep features of the corrosion area; Among them, it should be noted that:

[0082] (2.1) Deep feature extraction: The detected corrosion candidate areas are input into the deeper network of ResNet-101 to further extract higher-level deep features. The deep features here refer to the specific visual features related to corrosion, including the color change, surface roughness, edge texture details and shape of the corrosion area. These features can help the model better capture the details of the corrosion area and accurately identify and quantify the degree of corrosion.

[0083] (2.2) Attention mechanism: Combined with the channel attention mechanism (Squeeze-and-Excitation), a feature map is constructed based on the deep features of the corrosion candidate area extracted above, and global average pooling is performed on each channel of the feature map to generate a feature vector describing the channel; the feature vector is processed through a fully connected layer to generate channel weights accordingly; the generated weights are used to readjust each channel of the original constructed feature map to highlight important channels and suppress unimportant channels; this process can achieve the effect of highlighting the important features of the corrosion area and suppressing the interference of irrelevant background information.

[0084] 3. Input the reallocated weighted corrosion area deep features into U-Net, and use the encoder-decoder architecture of U-Net to perform fine segmentation of the corrosion area to clarify the boundary of the corrosion area and obtain the corrosion area segmentation mask accordingly; Among them, it should be noted that:

[0085] (3.1) U-Net architecture: The encoder part of U-Net can extract multi-scale features based on ResNet. Multi-scale features mainly include low-level features and high-level features. Among them, low-level features are details and texture information of the eroded area at high spatial resolution, such as edges and small structural features; high-level features are semantic information at low spatial resolution, such as shape and overall area features; these features are captured by the encoder part of U-Net, which can help better define the boundaries of the eroded area; the decoder part of U-Net restores the spatial resolution by gradually upsampling, and combines the jump connection in the encoder to achieve fine boundary segmentation;

[0086] (3.2) Multi-scale fusion: Low-level features (such as texture) and high-level features (such as semantic information) can be fused through skip connections, thus ensuring that the boundaries of the eroded area are clear and precise.

[0087] 4. Input the obtained corrosion area segmentation mask into ViT (Vision Transformer), and use ViT to perform global context analysis on the segmented corrosion area to further evaluate the severity of corrosion according to the corrosion level, and obtain the corresponding corrosion level classification result (based on the corrosion level classification mentioned above, the corrosion severity evaluated here is expressed as level I, level II, level III, level IV or level V); Among them, it should be noted that:

[0088] (4.1) Image block processing: The segmented eroded area is divided into several fixed-size image patches and input into ViT. Each image patch is flattened into a vector and spatial information is added through position encoding.

[0089] (4.2) Global feature modeling: ViT uses a multi-head self-attention mechanism to analyze the corrosion area from a global perspective, capture contextual information, and evaluate the severity of corrosion. Unlike traditional CNNs (Convolutional Neural Networks), ViT can effectively handle long-distance dependencies and understand the corrosion situation more comprehensively.

[0090] 5. According to the obtained corrosion grade classification results and the annotated data obtained by annotating the second sample image set, a preset joint loss function is used to adjust the model parameters, and a trained deep learning model is obtained accordingly; wherein, it should be noted that:

[0091] (5.1) Using supervised learning methods, end-to-end model training is performed using corrosion grade classification results and labeled data;

[0092] (5.2) Optimizer selection: Use the Adam optimizer for training and combine it with the learning rate scheduler to optimize the training process.

[0093] In one of the optional embodiments, L total =α*L loc +β*L cls +γ*L seg , where L total represents the joint loss function, L loc represents the position loss, L cls represents the classification loss, L segrepresents segmentation loss, α, β and γ represent weight coefficients, and the position loss is L1 loss or smooth L1 loss, the classification loss is cross entropy loss or focal loss, and the segmentation loss is Dice loss or cross entropy loss.

[0094] Specifically, in combination with the above embodiments, in the end-to-end training process, the embodiment of the present invention defines a joint loss function, which comprehensively considers the task requirements of target detection, segmentation and corrosion severity detection. The joint loss function specifically includes:

[0095] Position loss (denoted as L loc ): used for Faster R-CNN bounding box positioning (adjusting the position and size of the bounding box), you can use L1 loss or smooth L1 loss;

[0096] Classification loss (denoted as L cls ): used for target classification and severity classification tasks (distinguishing foreground from background), usually using cross entropy loss or focal loss;

[0097] Segmentation loss (denoted as L seg ): Used for segmentation tasks, commonly used Dice loss or cross entropy loss, especially suitable for unbalanced data sets.

[0098] By weighted summation, a joint loss function (denoted as L total ), and the weight coefficient is set based on the importance of the task and the numerical range of the loss. The corresponding expression of the joint loss function is as follows:

[0099] L total =α*L loc +β*L cls +γ*L seg , where α, β, and γ represent weight coefficients used to balance the contribution of different losses.

[0100] It should be noted that in the actual model training process, these coefficients are continuously adjusted through experiments to achieve the best training effect, which can enable the model to show good performance in detection, classification and segmentation, ensuring that the corrosion area and corrosion level are accurately detected.

[0101] In one optional embodiment, optimizing the trained deep learning model to obtain a corrosion detection model specifically includes:

[0102] Performing a performance evaluation on the trained deep learning model using k-fold cross validation to obtain a first evaluation result;

[0103] Using precision, recall and IoU indicators to perform performance evaluation on the trained deep learning model to obtain a second evaluation result;

[0104] The trained deep learning model is optimized according to the first evaluation result and the second evaluation result to obtain a corrosion detection model.

[0105] Specifically, in combination with the above embodiments, when optimizing the obtained trained deep learning model, the performance of the trained deep learning model can be evaluated by using k-fold cross validation to obtain a first evaluation result, and the performance of the trained deep learning model can be evaluated by using evaluation indicators such as precision, recall rate, and IoU (Intersection over Union), to obtain a second evaluation result; then, the trained deep learning model is optimized according to the first evaluation result and the second evaluation result, to obtain an optimized deep learning model, and the optimized deep learning model is used as a corrosion detection model.

[0106] It should be noted that k-fold cross validation is to divide the training data set (i.e., the second sample image set) into k non-overlapping subsets (folds), use k-1 subsets for training each time, and use the remaining 1 subset for validation; repeat this process k times, each time selecting a different subset as the validation set; in each fold, adjust the hyperparameters (such as learning rate, batch size, etc.) and evaluate the model performance, and obtain the model performance evaluation results under different hyperparameter combinations accordingly, and finally select the hyperparameter combination that performs best in all folds. By using k-fold cross validation to evaluate model performance and optimize hyperparameters, the generalization ability of the model can be ensured.

[0107] It should be noted that by using the model prediction results and the real labeled data to calculate the accuracy, recall rate, IoU and other evaluation indicators, it can be used to evaluate the performance of the model in corrosion detection and segmentation tasks; among them,

[0108] Precision: used to evaluate the accuracy of model prediction, that is, the proportion of correctly predicted positive examples to all predicted positive examples. The calculation expression is: Precision = TP / (TP+FP), TP (True Positive) indicates the number of correctly classified as positive examples, that is, the number of examples that are actually positive and predicted as positive by the model, and FP (False Positive) indicates the number of incorrectly classified as positive examples, that is, the number of examples that are actually negative but predicted as positive by the model;

[0109] Recall: It is used to measure the sensitivity of the model, that is, the ratio of correctly predicted positive examples to all actual positive examples. Its calculation expression is: Recall = TP / (TP+FN), where FN (False Negative) represents the number of examples that are mistakenly classified as negative examples, that is, the number of examples that are actually positive but predicted as negative examples by the model.

[0110] IoU: used for segmentation tasks to calculate the degree of overlap between the predicted area and the true area. The calculation expression is: IoU = TP / (TP+FP+FN).

[0111] It should be noted that by combining the first evaluation result and the second evaluation result, a more comprehensive model evaluation result can be obtained, and then adjusting and improving the model according to the first evaluation result and the second evaluation result can enable the model to obtain the optimal training result.

[0112] The second aspect of the present invention provides a method for detecting corrosion of a transmission tower based on deep learning. Figure 2 As shown, it is a flowchart of a transmission tower corrosion detection method based on deep learning provided by an embodiment of the second aspect of the present invention, and the method includes steps S21 to S23:

[0113] Step S21, obtaining an image of a transmission tower to be detected;

[0114] Step S22, preprocessing the image to be detected;

[0115] Step S23, performing corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain a corrosion detection result of the transmission tower to be detected;

[0116] Among them, the corrosion detection model is trained using the deep learning-based model training method described in any embodiment of the first aspect above.

[0117] Specifically, after obtaining the corrosion detection model by adopting the deep learning-based model training method described in any one of the embodiments of the first aspect above, when actually performing corrosion detection on the transmission tower, the image to be detected of the transmission tower to be detected can be first obtained, and the obtained image to be detected can be preprocessed (to ensure that the format of the image to be detected is consistent with that of the second sample image used in training), and then the corrosion detection model that has been obtained can be used to perform corrosion grade classification detection on the preprocessed image to be detected, and the corrosion grade detection result of the transmission tower to be detected can be obtained accordingly.

[0118] It should be noted that the transmission tower corrosion detection method based on deep learning provided in the embodiment of the second aspect of the present invention uses the corrosion detection model obtained by the deep learning-based model training method described in any embodiment of the first aspect to perform transmission tower corrosion detection. The technical effects that can be achieved are the same as the technical effects that can be achieved in the embodiments of the first aspect, and will not be repeated here.

[0119] It should be noted that the embodiments of the present invention can not only detect the presence of corrosion, but also classify the severity of corrosion. Combined with the maintenance needs of power operation and maintenance personnel, it can provide intelligent decision support. In addition, the embodiments of the present invention are easy to use, scientific and effective, suitable for the detection needs of power systems, and provide a new technical means for the intelligent operation and maintenance of power systems. At the same time, the embodiments of the present invention introduce deep learning into transmission tower corrosion detection, which can not only improve the efficiency, accuracy and robustness of corrosion detection, but also reduce the maintenance cost of transmission towers through continuous monitoring and early warning, extend the life of equipment, and ensure the stability of power supply.

[0120] The third aspect of the present invention further provides a model training device based on deep learning, which is used to implement the model training method based on deep learning described in any embodiment of the first aspect above, see Figure 3 , which is a structural block diagram of a model training device based on deep learning provided by an embodiment of the third aspect of the present invention, wherein the training device comprises:

[0121] The sample image set construction module 11 is used to construct an original sample image set of a transmission tower;

[0122] The sample image set annotation module 12 is used to annotate each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set;

[0123] A sample image set preprocessing module 13, configured to preprocess each first sample image in the first sample image set to obtain a second sample image set;

[0124] A deep learning model training module 14 is used to train a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes FasterR-CNN, ResNet, U-Net and ViT;

[0125] The deep learning model optimization module 15 is used to optimize the trained deep learning model to obtain a corrosion detection model so as to perform transmission tower corrosion detection according to the corrosion detection model.

[0126] Preferably, the sample image set preprocessing module 13 is specifically used for:

[0127] Each first sample image in the first sample image set is resized, normalized, and data enhanced to obtain a second sample image set.

[0128] Preferably, the deep learning model training module 14 specifically includes:

[0129] A Faster R-CNN processing unit, configured to perform corrosion region detection on the second sample image set using Faster R-CNN to obtain corrosion candidate regions; wherein the feature extractor of Faster R-CNN is ResNet;

[0130] A ResNet processing unit, used to extract deep features of the corrosion candidate area using ResNet to obtain deep features of the corrosion area;

[0131] A channel attention mechanism processing unit, used for assigning different weights to each channel of the feature map constructed based on the depth features of the corrosion area by using the channel attention mechanism;

[0132] A U-Net processing unit is used to segment the corrosion region using the weighted corrosion region depth features using U-Net to obtain a corrosion region segmentation mask; wherein the encoder part of the U-Net is ResNet;

[0133] A ViT processing unit, used for performing corrosion grade classification on the corrosion region segmentation mask by using ViT to obtain a corrosion grade classification result;

[0134] A loss function processing unit is used to adjust model parameters using a preset joint loss function according to the corrosion grade classification result and the annotation data of the second sample image set to obtain a trained deep learning model.

[0135] Preferably, L total =α*L loc +β*L cls +γ*L seg , where L total represents the joint loss function, L loc represents the position loss, L cls represents the classification loss, L seg represents segmentation loss, α, β and γ represent weight coefficients, and the position loss is L1 loss or smooth L1 loss, the classification loss is cross entropy loss or focal loss, and the segmentation loss is Dice loss or cross entropy loss.

[0136] Preferably, the deep learning model optimization module 15 specifically includes:

[0137] A first model evaluation unit is used to perform a performance evaluation on the trained deep learning model by using k-fold cross validation to obtain a first evaluation result;

[0138] A second model evaluation unit is used to perform performance evaluation on the trained deep learning model using precision, recall rate and IoU indicators to obtain a second evaluation result;

[0139] A deep learning model optimization unit is used to optimize the trained deep learning model according to the first evaluation result and the second evaluation result to obtain a corrosion detection model.

[0140] It should be noted that the model training device based on deep learning provided in the embodiment of the third aspect of the present invention can implement all the processes of the model training method based on deep learning described in any embodiment of the first aspect above. The functions of each module and unit in the training device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the model training method based on deep learning described in the embodiment of the first aspect above, and will not be repeated here.

[0141] The fourth aspect of the present invention also provides a transmission tower corrosion detection device based on deep learning, which is used to implement the transmission tower corrosion detection method based on deep learning described in any embodiment of the second aspect. Figure 4 FIG. 1 is a block diagram of a transmission tower corrosion detection device based on deep learning provided in an embodiment of the fourth aspect of the present invention, wherein the detection device comprises:

[0142] The image acquisition module 21 is used to acquire the image of the transmission tower to be detected;

[0143] The image preprocessing module 22 is used to preprocess the image to be detected;

[0144] The transmission tower corrosion detection module 23 is used to perform corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain the corrosion detection result of the transmission tower to be detected;

[0145] Among them, the corrosion detection model is trained using the deep learning-based model training method described in any embodiment of the first aspect above.

[0146] It should be noted that the deep learning-based transmission tower corrosion detection device provided in the fourth aspect of the present invention can realize all the processes of the deep learning-based transmission tower corrosion detection method described in any embodiment of the second aspect above. The functions of each module in the detection device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the deep learning-based transmission tower corrosion detection method described in the second aspect above, and will not be repeated here.

[0147] The fifth aspect embodiment of the present invention also provides a computer-readable storage medium, including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the deep learning-based model training method described in any embodiment of the first aspect above, or executes the deep learning-based transmission tower corrosion detection method described in any embodiment of the second aspect.

[0148] The sixth aspect of the present invention further provides a terminal device, see Figure 5 As shown, it is a structural block diagram of a terminal device provided by an embodiment of the sixth aspect of the present invention, the terminal device includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10, and the processor 10 implements the deep learning-based model training method described in any embodiment of the first aspect when executing the computer program, or implements the deep learning-based transmission tower corrosion detection method described in any embodiment of the second aspect.

[0149] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0150] The processor 10 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 10 may be any conventional processor. The processor 10 is the control center of the terminal device, and various parts of the terminal device are connected using various interfaces and lines.

[0151] The memory 20 mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc., and the data storage area can store related data, etc. In addition, the memory 20 can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, and a flash card (Flash Card), etc., or the memory 20 can also be other volatile solid-state storage devices.

[0152] It should be noted that the above terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 5 The structural block diagram shown is only a structural example of the above-mentioned terminal device and does not constitute a structural limitation of the above-mentioned terminal device. The above-mentioned terminal device may include more or fewer components than shown in the figure, or combine certain components, or different components.

[0153] In summary, the model training method based on deep learning, the transmission tower corrosion detection method, device, computer-readable storage medium and terminal device provided in the embodiments of the present invention combine the positioning capability of Faster R-CNN, the feature extraction capability of ResNet, the segmentation capability of U-Net and the context analysis capability of ViT to form a multi-stage corrosion detection model based on deep learning. By introducing deep learning into transmission tower corrosion detection and constructing a multi-layer neural network, image features can be automatically learned from images without human intervention. It shows superior performance in tasks such as image classification and target detection, and can also adapt to image features under complex backgrounds and different lighting conditions, thereby solving the problems existing in traditional manual inspections and automated detection based on image processing, and effectively improving the accuracy and robustness of transmission tower corrosion detection.

[0154] In addition, the embodiments of the present invention can not only detect the presence of corrosion, but also classify the severity of corrosion. Combined with the maintenance needs of power operation and maintenance personnel, it can provide intelligent decision support. Moreover, the embodiments of the present invention are easy to use, scientific and effective, and suitable for the detection needs of power systems. It provides a new technical means for the intelligent operation and maintenance of power systems. At the same time, the embodiments of the present invention introduce deep learning into transmission tower corrosion detection, which can not only improve the efficiency, accuracy and robustness of corrosion detection, but also reduce the maintenance cost of transmission towers through continuous monitoring and early warning, extend the life of equipment, and ensure the stability of power supply.

[0155] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A model training method based on deep learning, characterized in that: include: Constructing an original sample image set of transmission towers; Annotate each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set; Preprocessing each first sample image in the first sample image set to obtain a second sample image set; Training a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes Faster R-CNN, ResNet, U-Net and ViT; The trained deep learning model is optimized to obtain a corrosion detection model, so as to perform transmission tower corrosion detection according to the corrosion detection model.

2. The model training method based on deep learning according to claim 1, characterized in that: The preprocessing of each first sample image in the first sample image set to obtain the second sample image set specifically includes: Each first sample image in the first sample image set is resized, normalized, and data enhanced to obtain a second sample image set.

3. The model training method based on deep learning according to claim 1, characterized in that: The step of training a preset deep learning model according to the second sample image set to obtain a trained deep learning model specifically includes: Using Faster R-CNN to perform corrosion area detection on the second sample image set to obtain corrosion candidate areas; wherein the feature extractor of Faster R-CNN is ResNet; Using ResNet to extract deep features of the corrosion candidate area to obtain deep features of the corrosion area; Using a channel attention mechanism, assigning different weights to each channel of a feature map constructed based on the depth features of the corrosion area; The U-Net is used to segment the corrosion region based on the weighted corrosion region depth features to obtain the corrosion region segmentation mask; wherein the encoder part of the U-Net is ResNet; Using ViT to classify the corrosion level of the corrosion area segmentation mask to obtain a corrosion level classification result; According to the corrosion grade classification result and the annotated data of the second sample image set, a preset joint loss function is used to adjust the model parameters to obtain a trained deep learning model.

4. The model training method based on deep learning as claimed in claim 3, characterized in that: L total =α*L loc +β*L cls +γ*L seg , where L total represents the joint loss function, L loc represents the position loss, L cls represents the classification loss, L seg represents segmentation loss, α, β and γ represent weight coefficients, and the position loss is L1 loss or smooth L1 loss, the classification loss is cross entropy loss or focal loss, and the segmentation loss is Dice loss or cross entropy loss.

5. The deep learning-based model training method according to any one of claims 1 to 4, characterized in that: Optimizing the trained deep learning model to obtain a corrosion detection model specifically includes: Performing a performance evaluation on the trained deep learning model using k-fold cross validation to obtain a first evaluation result; Using precision, recall and IoU indicators to perform performance evaluation on the trained deep learning model to obtain a second evaluation result; The trained deep learning model is optimized according to the first evaluation result and the second evaluation result to obtain a corrosion detection model.

6. A transmission tower corrosion detection method based on deep learning, characterized in that: include: Acquire an image of a transmission tower to be inspected; Preprocessing the image to be detected; Performing corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain a corrosion detection result of the transmission tower to be detected; Wherein, the corrosion detection model is obtained by training using the deep learning-based model training method described in any one of claims 1 to 5.

7. A model training device based on deep learning, characterized in that: include: A sample image set construction module is used to construct an original sample image set of a transmission tower; A sample image set annotation module is used to annotate each original sample image in the original sample image set according to the pre-classified corrosion level to obtain a first sample image set; A sample image set preprocessing module, used for preprocessing each first sample image in the first sample image set to obtain a second sample image set; A deep learning model training module, used to train a preset deep learning model according to the second sample image set to obtain a trained deep learning model; wherein the network architecture of the deep learning model includes Faster R-CNN, ResNet, U-Net and ViT; The deep learning model optimization module is used to optimize the trained deep learning model to obtain a corrosion detection model so as to perform transmission tower corrosion detection according to the corrosion detection model.

8. A transmission tower corrosion detection device based on deep learning, characterized in that: include: The module for acquiring the image to be detected is used to acquire the image to be detected of the transmission tower to be detected; A preprocessing module for the image to be detected, used for preprocessing the image to be detected; The transmission tower corrosion detection module is used to perform corrosion detection on the preprocessed image to be detected according to a preset corrosion detection model to obtain the corrosion detection result of the transmission tower to be detected; Wherein, the corrosion detection model is obtained by training using the deep learning-based model training method described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that: It includes a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the model training method based on deep learning as described in any one of claims 1 to 5, or executes the transmission tower corrosion detection method based on deep learning as described in claim 6.

10. A terminal device, characterized in that: The invention comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When executing the computer program, the processor implements the model training method based on deep learning as described in any one of claims 1 to 5, or implements the transmission tower corrosion detection method based on deep learning as described in claim 6.

Images