Toppling detection method and system of transmission tower, medium and equipment

Abstract

The invention discloses a power transmission tower toppling detection method, system, medium and device, and belongs to the technical field of power transmission tower toppling detection, and the method comprises the steps: obtaining a to-be-detected and reference remote sensing image of a power transmission tower; an image to be detected is input into the multi-branch feature YOLO model trained by a normal state tower sample, multi-scale features containing pixel details and texture information are generated through multi-stage extraction, secondary extraction and deep fusion are performed after fusion, diversified tower features containing semantic understanding are output, and recognition information of a target to be detected is obtained through recognition; and inputting the reference image into the model to obtain reference identification information, comparing the identification information of the target to be detected with the reference identification information, and outputting a tower toppling state identification result. Therefore, by implementing the method, the problem that in the prior art, a deep learning method depends on a toppling pole tower sample, so that engineering application in power transmission pole tower toppling detection is limited can be solved.

Description

Technical Field

[0001] This invention belongs to the field of power transmission tower tilting detection technology, and relates to a method, system, medium and equipment for power transmission tower tilting detection. Background Technology

[0002] With global warming and the increasing frequency of extreme weather events, transmission towers face greater risks of deformation or tilting. Severe tilting can lead to line breaks, power outages, and other accidents, causing significant economic losses and social impact. Therefore, routine and regular monitoring of transmission line tilt and collapse, and timely detection of the target location of tilted or collapsed transmission towers, is a crucial part of daily operation, maintenance, and emergency response. Currently, traditional methods for monitoring tower tilt and collapse include manual line inspection, drone inspection, infrared thermal imaging, laser scanning, and ground-based sensor monitoring. In addition, methods based on high-resolution remote sensing satellite imagery for tower tilt identification have emerged, such as tilt analysis based on tower shadows and single-phase or multi-phase tower tilt identification models based on artificial intelligence.

[0003] However, traditional manual line inspection methods are inefficient, have limited coverage, and pose safety hazards to personnel. Ground sensor monitoring methods have high deployment and maintenance costs, drone inspections have long single inspection cycles, laser scanning methods have difficult data processing and poor timeliness, and remote sensing image-based methods either rely on good imaging weather conditions or on a large number of diverse and high-quality samples of toppled target towers. At present, the accumulation of remote sensing image targets of toppled transmission towers is relatively scarce, which limits the application of these methods in engineering practice. Summary of the Invention

[0004] This application provides a method, system, medium, and equipment for detecting the collapse of power transmission towers, which can solve the problem that the engineering application of deep learning methods in power transmission tower collapse detection is limited due to their reliance on collapsed tower samples.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for detecting the tilting of transmission towers, comprising:

[0006] Acquire the current remote sensing image of the transmission tower to be detected, as well as the corresponding reference remote sensing image;

[0007] The remote sensing image to be detected is input into a preset multi-branch feature YOLO model, which performs multi-stage feature extraction on the remote sensing image to be detected, generating multi-scale tower features containing pixel-level details and low-level texture information. The multi-scale tower features are fused, and secondary feature extraction and deep fusion are performed to output diversified tower features containing semantic understanding. Based on the diversified tower features, the target recognition information of the tower to be detected is output. The multi-branch feature YOLO model is trained on remote sensing image samples of transmission towers in a normal, undamaged state.

[0008] The target identification information of the tower to be detected is compared with the target identification information of the reference tower, and the tilting state identification result of the transmission tower corresponding to the remote sensing image to be detected is output; wherein, the target identification information of the reference tower is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

[0009] Compared with existing technologies, the embodiments of this application have the following beneficial effects: by acquiring the current remote sensing image of the transmission tower to be detected and the corresponding reference remote sensing image, a data foundation is provided for subsequent comparative analysis; the remote sensing image to be detected is input into a preset multi-branch feature YOLO model, which generates multi-scale tower features containing pixel-level details and low-level texture information through multi-stage feature extraction, thereby improving the feature capture capability of towers at different scales; the multi-scale tower features are fused and secondary feature extraction and deep fusion are performed to output diverse tower features containing semantic understanding, further enhancing the richness and representation capability of the features; the target identification information of the tower to be detected is output based on the diverse tower features, thereby achieving accurate classification and positioning of the tower target; the target identification information of the tower to be detected is compared with the reference tower target identification information to output the tilting state identification result, and anomaly detection is achieved by judging the state change; wherein, the multi-branch feature YOLO model is trained only using remote sensing image samples of transmission towers in normal state, without relying on scarce tilting tower samples, thus reducing the dependence threshold on abnormal samples. Meanwhile, multi-stage feature extraction ensures the comprehensiveness of features, secondary fusion enhances semantic understanding, normal sample training solves the problem of sample scarcity, and the comparison mechanism realizes state change detection. These features work together to solve the core problem of limited engineering application caused by reliance on the overturning sample library in existing technologies, and significantly improve the practicality and generalization ability of transmission tower overturning detection.

[0010] In some embodiments of the first aspect of this application, the multi-branch feature YOLO model includes a backbone network, a feature fusion network, and a classification and recognition network, wherein the backbone network is formed by stacking and connecting several transmission feature extraction modules in sequence;

[0011] The step of performing multi-stage feature extraction on the remote sensing image to be detected, generating multi-scale tower features containing pixel-level details and low-level texture information, includes:

[0012] In each stage, according to the current transmission feature extraction module, the input signal is subjected to convolution and channel splitting operations to obtain the first feature branch stream and the second feature branch stream. The second feature branch stream is then subjected to convolution to increase the number of channels, depth-separable convolution operation, and point convolution to reduce the number of channels to obtain the third feature branch stream. The first feature branch stream and the third feature branch stream are then spliced ​​and fused to generate the multi-scale tower features corresponding to each stage.

[0013] The input signal for the first stage is the remote sensing image to be detected, and the input signals for the remaining stages are the multi-scale tower features output from the previous stage.

[0014] Compared with existing technologies, the above embodiments have the following beneficial effects: The backbone network is composed of several transmission feature extraction modules stacked and connected sequentially. In each stage, the current transmission feature extraction module performs convolution and channel splitting operations on the input signal to obtain the first feature branch stream and the second feature branch stream. The multi-dimensional information of the original features is preserved through parallel branch processing. The second feature branch stream is processed by convolution to increase the number of channels, depthwise separable convolution operation, and point convolution to reduce the number of channels to obtain the third feature branch stream. Among them, depthwise separable convolution reduces the amount of computation while maintaining the feature extraction capability, and the dynamic adjustment of the number of channels enhances the flexibility of feature expression. The first feature branch stream and the third feature branch stream are spliced ​​and fused to generate multi-scale tower features corresponding to each stage, realizing the effective integration of detailed features and deep features. The first stage uses the remote sensing image to be detected as the input signal, and the other stages use the multi-scale tower features output from the previous stage as the input. Through the hierarchical feature extraction method, the hierarchical representation of tower features is gradually deepened, improving the systematicness and completeness of feature extraction.

[0015] In some embodiments of the first aspect of this application, the feature fusion network includes a plurality of shallow transmission feature fusion modules, and a transmission feature extraction module and a deep transmission feature fusion module that are sequentially connected to each of the shallow transmission feature fusion modules.

[0016] The feature fusion network is used to fuse the multi-scale tower features and perform secondary feature extraction and deep fusion to output diverse tower features with semantic understanding, including:

[0017] Based on the shallow transmission feature fusion modules, the multi-scale tower features output by each transmission feature extraction module in the backbone network are downsampled and channel adjusted, and then spliced ​​and fused to output the composite features.

[0018] Based on each transmission feature extraction module of the feature fusion network, secondary feature extraction is performed on each composite feature to output feature maps of different resolutions;

[0019] According to the deep transmission feature fusion modules, each feature map is sampled and the channels are adjusted, and the features are stitched together to obtain the diverse tower features.

[0020] Compared with existing technologies, the above embodiments have the following beneficial effects: by downsampling and adjusting the channels of multi-scale tower features in the backbone network through each shallow transmission feature fusion module, and splicing and fusing them to output various composite features, the initial integration of multi-scale features is achieved; each transmission feature extraction module of the feature fusion network performs secondary feature extraction on each composite feature to output feature maps of different resolutions, further improving the feature representation capability; each deep transmission feature fusion module samples and adjusts the channels of each feature map and splices the features to obtain diversified tower features, enhancing the interaction between deep features and shallow features, and improving the network's recognition accuracy of tower targets in complex backgrounds.

[0021] In some embodiments of the first aspect of this application, the multi-branch feature YOLO model is trained based on remote sensing image samples of transmission towers in a normal, undamaged state, including:

[0022] Acquire the original remote sensing images of each power transmission tower;

[0023] Each of the original remote sensing images is preprocessed to obtain a remote sensing image; wherein the preprocessing includes any one or more combinations of the following: geometric correction, radiometric correction and target correction;

[0024] Based on a preset pole identification model, multiple rounds of identification and labeling are performed on each unlabeled pole target in each remote sensing image until all remote sensing images are labeled, and each remote sensing image sample is output; after each round of labeling, the labeling results of the current round are verified, and the pole identification model is iteratively updated based on all the currently verified labeling results.

[0025] Based on the remote sensing image samples, a preset original multi-branch feature YOLO model is trained to obtain the multi-branch feature YOLO model.

[0026] Compared with existing technologies, the above embodiments have the following beneficial effects: acquiring original remote sensing images of transmission towers provides basic data for model training; preprocessing each original remote sensing image, including geometric correction, radiometric correction, and target correction, is performed, where geometric correction corrects geometric distortion of the image, radiometric correction eliminates radiometric distortion, and target correction unifies the tower imaging perspective, improving image quality and consistency; based on a preset tower recognition model, multiple rounds of recognition and annotation are performed on unlabeled tower targets in each remote sensing image, and the results are verified and the tower recognition model is iteratively updated after each round of annotation until all remote sensing images are annotated and each remote sensing image sample is output. The progressive annotation through human-computer interaction improves the annotation efficiency and accuracy of the sample library; the preset original multi-branch feature YOLO model is trained based on each remote sensing image sample to obtain a multi-branch feature YOLO model, ensuring that the model can fully learn the features of normal towers and improve recognition performance.

[0027] In some embodiments of the first aspect of this application, the target correction includes:

[0028] Acquire sensor parameter data corresponding to each of the original remote sensing images, as well as digital elevation model data, ground control point data, and preset tower sample data for the same area;

[0029] Based on a preset feature point matching algorithm, the overlapping feature points in the overlapping areas of each of the original remote sensing images are identified, a connection point network is constructed, and local area network adjustment is performed in combination with the ground control point data to output the spatial location relationship of the images.

[0030] Based on the tower sample data, the digital elevation model data is supplemented and fused to obtain digital tower elevation model data. Combined with the sensor parameter data, the projection difference correction value of each pixel in each of the original remote sensing images is calculated according to the preset projection difference calculation formula.

[0031] Based on the spatial relationship of the images and the ground control point data, a geometric correction model is constructed using a polynomial fitting method. Combined with the digital tower elevation model data and the projection difference correction values, the coordinate transformation of each pixel in each of the original remote sensing images is adjusted to obtain each of the remote sensing images without projection difference.

[0032] Compared with existing technologies, the above embodiments have the following beneficial effects: In target correction, sensor parameter data corresponding to each original remote sensing image, as well as digital elevation model data, ground control point data, and preset tower sample data of the same area are acquired to provide multi-source auxiliary information for correction; based on the preset feature point matching algorithm, overlapping feature points in the overlapping areas of each original remote sensing image are identified, a connection point network is constructed, and local area network adjustment processing is performed in combination with ground control point data to output the spatial position relationship of the images, thereby improving the relative positioning accuracy between images; the digital elevation model data is supplemented and fused according to the tower sample data to obtain... The digital tower elevation model data, combined with sensor parameter data, calculates the projection difference correction value for each pixel in each original remote sensing image according to a preset projection difference calculation formula, solving the projection difference problem caused by the lack of tower height information in traditional digital elevation models. Based on the spatial position relationship of the images and ground control point data, a geometric correction model is constructed using a polynomial fitting method. Combining the digital tower elevation model data and each projection difference correction value, the coordinate transformation of each pixel in each original remote sensing image is adjusted to obtain remote sensing images without projection difference. This can eliminate the tower imaging tilt caused by different satellite perspectives and unify the initial geometric attitude of the tower.

[0033] In some embodiments of the first aspect of this application, training a preset original multi-branch feature YOLO model based on each of the remote sensing image samples to obtain the multi-branch feature YOLO model includes:

[0034] The loss function that combines localization loss and classification loss is constructed as follows:

[0035] L total =a·L cls +b·L reg Among them, the classification loss function L cls and localization loss function L reg They are represented as follows:

[0036] L cls =-α(1-p t ) γ log(p t );

[0037]

[0038] Where a and b represent weights, α is the weight balancing the positive and negative samples, and p t γ represents the probability that the model's predicted result belongs to the normal tower category, γ is the focusing factor, and t i Let t be the parameters of the i-th bounding box predicted by the model, where (x, y), w, and h represent the coordinates of the center point, width, and height of the bounding box, respectively. i *is the true value of the i-th bounding box parameter, and β is the threshold that controls the squared loss and the linear loss;

[0039] By combining the remote sensing image samples and the loss function, the original multi-branch feature YOLO model is trained to obtain the multi-branch feature YOLO model.

[0040] Compared to existing technologies, the above embodiments have the following beneficial effects: A loss function integrating localization and classification losses is constructed, where the classification loss weight and localization loss weight in the total loss function are used to balance the contribution ratios of classification loss and localization loss, ensuring that the model simultaneously focuses on classification accuracy and localization accuracy; the weight parameter balancing positive and negative samples in the classification loss function is used to address the class imbalance problem; the probability value of the model's prediction belonging to the normal tower category quantifies the model's prediction confidence for the target category; the focusing factor is used to reduce the contribution of easily classified samples to the total loss, making the model focus more on difficult samples; the localization loss function calculates the loss value through a piecewise function design. When the absolute error between the predicted tower target bounding box parameters and the true bounding box parameters is less than the threshold controlling the squared loss and linear loss, the squared loss is used to promote accurate parameter convergence; when the absolute error is greater than or equal to the threshold, the linear loss is used to avoid gradient explosion. This piecewise calculation method balances the loss contribution under different error magnitudes, improving the stability and convergence speed of model training.

[0041] In some embodiments of the first aspect of this application, comparing the target identification information of the tower to be detected with the target identification information of a reference tower, and outputting the tilting state identification result of the transmission tower corresponding to the remote sensing image to be detected, includes:

[0042] Extract the first position information and the first aspect ratio of the target identification frame of the target tower from the target identification information to be detected, and extract the second position information and the second aspect ratio of the target identification frame of the target tower from the reference target identification information;

[0043] By combining a preset threshold with the first position information and the second position information, as well as the aspect ratio of the first border and the aspect ratio of the second border, the tilting state recognition result is output.

[0044] Compared with existing technologies, the above embodiments have the following advantages: First position information and first frame aspect ratio of the target identification box in the target identification information of the tower to be detected are extracted, along with second position information and second frame aspect ratio of the target identification box in the reference tower target identification information, to obtain the key parameters required for comparison; the first and second position information are compared using a preset threshold to determine whether the tower has experienced a positional shift, and the first and second frame aspect ratios are compared to determine whether the tower has undergone a shape change; the tilting state identification result is output through multi-dimensional parameter comparison, improving the accuracy and reliability of anomaly detection.

[0045] Secondly, the present invention also provides a toppling detection system for transmission towers, comprising: a data acquisition module, a model recognition module, and a result output module;

[0046] The data acquisition module is used to acquire the current remote sensing image of the transmission tower to be detected, as well as the corresponding reference remote sensing image.

[0047] The model recognition module is used to input the remote sensing image to be detected into a preset multi-branch feature YOLO model, so that the multi-branch feature YOLO model performs multi-stage feature extraction on the remote sensing image to be detected, generates multi-scale tower features containing pixel-level details and low-level texture information, fuses the multi-scale tower features, and performs secondary feature extraction and deep fusion to output diversified tower features containing semantic understanding. Based on the diversified tower features, it outputs the target recognition information of the tower to be detected; wherein, the multi-branch feature YOLO model is trained on remote sensing image samples of transmission towers in a normal, untilted and undamaged state;

[0048] The result output module is used to compare the target identification information of the tower to be detected with the target identification information of the reference tower, and output the identification result of the tilting state of the transmission tower corresponding to the remote sensing image to be detected; wherein, the target identification information of the reference tower is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

[0049] Compared with existing technologies, the above embodiments of this application have the following beneficial effects: by acquiring the current remote sensing image of the transmission tower to be detected and the corresponding reference remote sensing image, a data foundation is provided for subsequent comparative analysis; the remote sensing image to be detected is input into a preset multi-branch feature YOLO model, which generates multi-scale tower features containing pixel-level details and low-level texture information through multi-stage feature extraction, thereby improving the feature capture capability of towers at different scales; the multi-scale tower features are fused and secondary feature extraction and deep fusion are performed to output diverse tower features containing semantic understanding, further enhancing the richness and representation capability of the features; the target identification information of the tower to be detected is output based on the diverse tower features, thereby achieving accurate classification and positioning of the tower target; the target identification information of the tower to be detected is compared with the reference tower target identification information to output the tilting state identification result, and anomaly detection is achieved by judging the state change; wherein, the multi-branch feature YOLO model is trained only using remote sensing image samples of transmission towers in normal state, without relying on scarce tilting tower samples, thus reducing the dependence threshold on abnormal samples. Meanwhile, multi-stage feature extraction ensures the comprehensiveness of features, secondary fusion enhances semantic understanding, normal sample training solves the problem of sample scarcity, and the comparison mechanism realizes state change detection. These features work together to solve the core problem of limited engineering application caused by reliance on the overturning sample library in existing technologies, and significantly improve the practicality and generalization ability of transmission tower overturning detection.

[0050] Thirdly, the present invention also provides a device for detecting the collapse of transmission towers, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, it implements the steps of any of the methods for detecting the collapse of transmission towers according to the present invention.

[0051] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the power transmission tower tilting detection methods of the present invention. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a method for detecting the collapse of a power transmission tower provided in some embodiments of the present invention.

[0053] Figure 2 This is a schematic diagram of the structure of a power transmission tower tilting detection system provided in some embodiments of the present invention.

[0054] Figure 3 This is a structural diagram of a power transmission tower tilting detection device provided in some embodiments of the present invention.

[0055] Figure 4This is a schematic diagram illustrating the generation of a digital tower elevation model provided in some embodiments of the present invention.

[0056] Figure 5 This is a flowchart illustrating a method for constructing a pole and tower sample library according to some embodiments of the present invention.

[0057] Figure 6 This is a structural diagram of a multi-branch feature YOLO model provided in some embodiments of the present invention.

[0058] Figure 7 This is a structural diagram of a transmission feature extraction module provided in some embodiments of the present invention.

[0059] Figure 8 This is a schematic diagram of the structure of a shallow transmission feature fusion module provided in some embodiments of the present invention.

[0060] Figure 9 This is a schematic diagram of the structure of a deep transmission feature fusion module provided in some embodiments of the present invention.

[0061] Figure 10 This is a flowchart of a method for comparing identification results provided in some embodiments of the present invention.

[0062] Figure 11 This is a comparison diagram of recognition results provided in some embodiments of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1:

[0065] Please refer to Figure 1 To address the limitation of existing deep learning methods in transmission tower collapse detection due to their reliance on collapsed tower samples, an embodiment of this invention provides a method for detecting the collapse of transmission towers, comprising steps S1 to S3:

[0066] Step S1: Obtain the current remote sensing image of the transmission tower to be detected, and the corresponding reference remote sensing image.

[0067] Step S2: Input the remote sensing image to be detected into a preset multi-branch feature YOLO model, so that the multi-branch feature YOLO model performs multi-stage feature extraction on the remote sensing image to be detected, generates multi-scale tower features containing pixel-level details and low-level texture information, fuses the multi-scale tower features, performs secondary feature extraction and deep fusion, outputs diversified tower features containing semantic understanding, and outputs tower target recognition information to be detected based on the diversified tower features.

[0068] The multi-branch feature YOLO model is trained based on remote sensing image samples of power transmission towers in a normal, undamaged state.

[0069] Furthermore, the training of the multi-branch feature YOLO model can be specifically implemented through the following preferred embodiments, including steps S21-S23, as follows:

[0070] S21: Acquire the original remote sensing images of each transmission tower.

[0071] In practice, when acquiring data, we first prioritize remote sensing images of areas with high tower density to ensure the efficiency of tower sample labeling. Secondly, we select remote sensing images of areas with diverse terrain conditions and complex backgrounds to ensure that the constructed tower samples have diverse scenes and richness.

[0072] S22: Preprocess each of the original remote sensing images to obtain each remote sensing image; wherein, the preprocessing includes any one or more combinations of the following: geometric correction, radiometric correction and target correction;

[0073] In practice, after preparing abundant raw remote sensing images from multiple sources, necessary preprocessing is required, including:

[0074] Geometric correction: Geometric correction of remote sensing images is performed by using mathematical models to correct geometric distortions caused by factors such as sensor attitude, Earth curvature, terrain undulation, and atmospheric refraction. This ensures that the geometric position, shape, and size of ground features (such as tower targets) in the image are consistent with the real geographic space, which helps to achieve accurate identification and positioning of tower targets.

[0075] Radiometric correction: In order to eliminate radiometric distortion caused by factors such as sensor response characteristics, atmospheric scattering / absorption, and differences in lighting conditions during the imaging process of remote sensing images, radiometric correction restores the true reflectivity or radiance of ground objects and displays the electromagnetic wave reflectivity information of real tower targets for accurate classification and identification.

[0076] Furthermore, considering that differences in satellite imaging perspectives can lead to inconsistent views of the same ground features in remote sensing images, especially for tall, slender targets (towers), this phenomenon is more pronounced. For example, the same power transmission tower may exhibit significant differences in geometric position and attitude under different satellite perspectives, which can severely interfere with the algorithm's identification and judgment of the tower's tilting state.

[0077] Therefore, in order to correct the difference in the geometric attitude of ground objects caused by different satellite perspectives, this application proposes a target correction method for towers, as follows:

[0078] Furthermore, the target correction in step S22 can be implemented through the following preferred embodiments, including steps S221-S224, as follows:

[0079] S221: Obtain sensor parameter data corresponding to each of the original remote sensing images, as well as digital elevation model data, ground control point data, and preset tower sample data for the same area;

[0080] S222: Based on a preset feature point matching algorithm, identify the overlapping feature points in the overlapping areas of each of the original remote sensing images, construct a connection point network, and combine it with the ground control point data to perform local area network adjustment processing and output the spatial location relationship of the images.

[0081] S223: Based on the tower sample data, perform data supplementation and fusion on the digital elevation model data to obtain digital tower elevation model data, and combine the sensor parameter data to calculate the projection difference correction value of each pixel in each of the original remote sensing images according to the preset projection difference calculation formula.

[0082] S224: Based on the spatial positional relationship of the images and the ground control point data, a geometric correction model is constructed using a polynomial fitting method. Combined with the digital tower elevation model data and the projection difference correction values, the coordinate transformation of each pixel in each of the original remote sensing images is adjusted to obtain each of the remote sensing images without projection difference.

[0083] In practical implementation, geometric and radiometric corrections can be performed first, followed by target correction. If this is done, the original remote sensing images in steps S221-S224 are actually the images after geometric and radiometric corrections. Specifically, in step S222, the feature point matching algorithm can use the SIFT feature point matching algorithm to recover the spatial relationship between different images using local area network adjustment. In step S223, considering that the Digital Elevation Model (DEM) only contains topographic elevation information, although it has height information for basic common objects such as buildings, bridges, and trees, it usually does not contain vertical structural information for tower-like targets. This causes displacement of the tower top due to projection differences (specifically, the tower top area deviates from its actual position in the image), leading to tower tilting or misalignment in the corrected image. To address this problem, this invention constructs a preset tower sample database (covering preset tower models with multiple types of different heights and tower type parameters) and constructs a Digital Transmission Elevation Model (DTEM) based on the original DEM. The process is as follows: Figure 4 The diagram illustrates the generation of a digital pole elevation model (DTEM). It combines DEM data with preset pole models. Specifically, different preset pole models have different heights, shapes, and structural parameters. Different models are selected based on the type of pole, and these models are fused with the DEM data according to latitude and longitude. This means accurately placing the poles at their corresponding actual locations on the DEM, resulting in the DTEM. After obtaining the DTEM, the remote sensing image is corrected pixel-by-pixel for projection differences caused by terrain and feature heights. The formula for calculating the projection difference αd is as follows:

[0084] Where h is the elevation difference, θ is the sensor's field of view, and H is the flight altitude.

[0085] Based on the above formula, the projection difference caused by the difference in elevation of ground features can be calculated, that is, the projection difference correction value of each pixel can be obtained.

[0086] In step S224, this application optimizes the sensor's exterior orientation elements (i.e., parameters such as sensor spatial position and attitude angle) through control points (GCPs) to establish a precise mapping relationship between image coordinates and ground coordinates, i.e., the geometric correction model. The specific implementation method is as follows:

[0087] First, we construct a general polynomial fitting empirical model, as shown below:

[0088]

[0089] Here, Line and Sample represent the row and column numbers of pixels in the remote sensing image, i.e., the pixel coordinates of the corresponding points. Lat, Lon, and H are the latitude, longitude, and elevation information in the ground coordinate system. f1 to f4 are ternary cubic polynomials of the same form but with different coefficients. Each polynomial f k The format is as follows:

[0090]

[0091] The above shows the expression of a ternary cubic polynomial. After expansion, it has more than 20 polynomial coefficients, including constant terms, linear terms, quadratic cross terms, pure quadratic terms, cubic cross terms, mixed quadratic terms, and pure cubic terms, denoted as a0 to a19. Subsequently, by using the least squares method to solve these polynomial coefficients with known ground control points, a geometric correction model can be obtained. This geometric correction model describes the mapping relationship between ground points and pixel coordinates.

[0092] During the solution process, cubic polynomial theory requires at least 10 control points. Preferably, this invention recommends 30 control points, which will yield good results even under complex terrain conditions. Therefore, considering the insufficient number of control points, algorithms such as SIFT feature matching can be used to identify corresponding points between remote sensing images, assisting in optimizing the geometric model and ultimately obtaining an accurate mapping relationship between image coordinates and ground coordinates.

[0093] After constructing the geometric correction model, the coordinates of each pixel in the remote sensing image are transformed by combining DTEM data and projection difference correction values. Starting from the orthophoto grid, the coordinates of the original image are inferred from the geometric model and projection difference correction values, and finally the true orthophoto image without projection difference is output.

[0094] After the above processing steps, the deformation of ground objects (especially tower targets) in the original remote sensing image caused by terrain and height differences can be restored, and the initial geometric attitude of the tower targets in the restored data can be obtained as the true angle of view.

[0095] In this preferred embodiment, during target correction, sensor parameter data corresponding to each original remote sensing image, as well as digital elevation model data, ground control point data, and preset tower sample data for the same area are acquired to provide multi-source auxiliary information for correction. Based on a preset feature point matching algorithm, overlapping feature points in the overlapping areas of each original remote sensing image are identified, a connection point network is constructed, and local area network adjustment is performed in conjunction with ground control point data to output the spatial position relationship of the images, thereby improving the relative positioning accuracy between images. Based on the tower sample data, the digital elevation model data is supplemented and fused to obtain digital tower elevation model data. Combined with sensor parameter data, the projection difference correction value of each pixel in each original remote sensing image is calculated according to a preset projection difference calculation formula, which solves the projection difference problem caused by the lack of tower height information in traditional digital elevation models. Based on the spatial position relationship of the images and ground control point data, a geometric correction model is constructed using a polynomial fitting method. Combined with the digital tower elevation model data and each projection difference correction value, the coordinate transformation of each pixel in each original remote sensing image is adjusted to obtain each remote sensing image without projection difference, which can eliminate the tower imaging tilt caused by different satellite perspectives and unify the initial geometric attitude of the tower.

[0096] S23: Based on the preset pole identification model, perform multiple rounds of identification and labeling on each unlabeled pole target in each remote sensing image until all remote sensing images are labeled, and output each remote sensing image sample; and after each round of labeling, verify the labeling result of the current round, and iteratively update the pole identification model based on all currently verified labeling result data;

[0097] In practical implementation, after obtaining the preprocessed remote sensing images from step S22, it is necessary to annotate and construct a training sample library, such as... Figure 5 The flowchart shown illustrates a method for constructing a pole and tower sample library, which includes the following main steps:

[0098] First, perform manual visual interpretation and annotation of the tower targets, construct an initial sample library I using the annotated samples, train a pre-set tower recognition model based on the initial sample library I, and obtain the tower recognition model weight I;

[0099] II. Based on the weight I of the pole identification model, the unlabeled pole targets in the remote sensing image are automatically identified. Then, the erroneous samples are manually filtered out, and the correct samples are retained. The retained correct samples are fused with the initial sample library I to obtain the initial sample library II.

[0100] Third, train the pole recognition model using the initial sample library II to obtain pole recognition model weights II. Subsequently, use pole recognition model weights II to identify unlabeled pole targets in remote sensing images. Again, manually filter out erroneous samples (as the model trains, erroneous samples will gradually decrease, and the manual workload will gradually decrease), retain the correct samples, and further fuse the correct samples with the initial sample library II to obtain the initial sample library III.

[0101] Fourth, train the pole recognition model based on the initial sample library III to obtain the pole recognition model weight III. Then, perform the same processing as above and continue iterative processing until all unlabeled remote sensing images are processed.

[0102] The semi-automatic labeling method for pole and tower samples designed above gradually increases the number of original sample libraries while reducing manual workload. By alternating and coordinating manual labeling and machine recognition, both labeling efficiency and labeling accuracy of pole and tower targets are ensured. In specific engineering applications, this method can quickly obtain a large number of labeled normal pole and tower sample targets.

[0103] S24: Based on each of the remote sensing image samples, train the preset original multi-branch feature YOLO model to obtain the multi-branch feature YOLO model.

[0104] In this preferred embodiment, original remote sensing images of transmission towers are acquired to provide basic data for model training. Preprocessing is performed on each original remote sensing image, including geometric correction, radiometric correction, and target correction. Geometric correction corrects geometric distortion, radiometric correction eliminates radiometric distortion, and target correction unifies the tower imaging perspective, improving image quality and consistency. Based on a preset tower recognition model, unlabeled tower targets in each remote sensing image are identified and labeled in multiple rounds. After each round of labeling, the results are verified, and the tower recognition model is iteratively updated until all remote sensing images are labeled, outputting each remote sensing image sample. This progressive labeling through human-computer interaction improves the labeling efficiency and accuracy of the sample library. A preset original multi-branch feature YOLO model is trained based on each remote sensing image sample to obtain a multi-branch feature YOLO model, ensuring that the model can fully learn the features of normal towers and improve recognition performance.

[0105] Furthermore, step S24 can be implemented through the following preferred embodiments, including steps S241-S242, as follows:

[0106] S241: Construct a loss function that combines localization loss and classification loss, expressed as follows:

[0107] L total =a·L cls +b·L regAmong them, the classification loss function L cls and localization loss function L reg They are represented as follows:

[0108] L cls =-α(1-p t ) γ log(p t );

[0109]

[0110] Where a and b represent weights, which can be adjusted to control the contribution of classification loss and localization loss to the overall loss. α is the weight for balancing positive and negative samples (compensating for a low proportion of positive samples), and p... t γ represents the probability that the model's predicted result belongs to the normal tower category. γ is a focusing factor used to reduce the contribution of easily classifiable samples (simple samples) to the total loss. This application addresses the class imbalance problem through a classification loss function, making the model focus more on difficult samples (such as blurred, occluded, and small target towers). i Let t be the parameters of the i-th bounding box predicted by the model, where (x, y), w, and h represent the coordinates of the center point, width, and height of the bounding box, respectively. i * Let be the true value of the i-th bounding box parameter, and β be the threshold that controls the division between squared loss and linear loss. When the error is small, the localization loss function uses a quadratic function to promote accurate convergence, while when the error is large, a linear function is used to avoid gradient explosion.

[0111] S242: Combine the remote sensing image samples and the loss function to train the original multi-branch feature YOLO model to obtain the multi-branch feature YOLO model.

[0112] In the actual training process, the input data can first be standardized based on the mean and standard deviation, and the sample library can be divided into training, validation, and test sets in a 7:2:1 ratio to ensure consistent data distribution. Data augmentation can then be performed using methods such as random horizontal / vertical flipping, rotation, and cropping.

[0113] In this preferred embodiment, a loss function integrating localization loss and classification loss is constructed. The classification loss weight and localization loss weight in the total loss function are used to balance the contribution ratios of classification loss and localization loss, ensuring the model simultaneously focuses on classification accuracy and localization accuracy. The weight parameter balancing positive and negative samples in the classification loss function addresses class imbalance. The probability value of the model's predicted result belonging to the normal tower category quantifies the model's prediction confidence for the target category. The focusing factor reduces the contribution of easily classified samples to the total loss, making the model focus more on difficult samples. The localization loss function calculates the loss value through a piecewise function design. When the absolute error between the predicted tower target bounding box parameters and the true bounding box parameters is less than the threshold controlling the squared loss and linear loss, the squared loss is used to promote accurate parameter convergence. When the absolute error is greater than or equal to this threshold, the linear loss is used to avoid gradient explosion. This piecewise calculation method balances the loss contribution under different error magnitudes, improving the stability and convergence speed of model training.

[0114] Furthermore, considering the limitations of existing models in efficiently and adaptively integrating high-level semantic information and low-level spatial information simultaneously, this application designs a novel pole recognition model architecture, MTF-YOLO (Multi-BranchFeature YOLO), as shown in the overall structure below. Figure 6 The diagram shown illustrates a multi-branch feature YOLO model structure, a novel object detection framework with a multifunctional neckline. The Shallow Transmission Feature Fusion (STFF) module is designed to combine the output of the backbone with the neckline, preserving optimal shallow information for subsequent learning. Simultaneously, the Deep Transmission Feature Fusion (DTFF) module, deeply embedded in the neckline, transmits more diverse gradient information to the output layer. The internal data flow of the model is summarized below:

[0115] The multi-branch feature YOLO model includes a backbone network, a feature fusion network, and a classification and recognition network. The backbone network is composed of several transmission feature extraction modules stacked and connected in sequence.

[0116] The multi-stage feature extraction in step S2 can be implemented through the following preferred implementation method, including step S25, as follows:

[0117] S25: In each stage, according to the current transmission feature extraction module, the input signal is subjected to convolution and channel splitting operations to obtain the first feature branch stream and the second feature branch stream. The second feature branch stream is then subjected to convolution to increase the number of channels, depth-separable convolution operation, and point convolution to reduce the number of channels to obtain the third feature branch stream. The first feature branch stream and the third feature branch stream are then spliced ​​and fused to generate the multi-scale tower features corresponding to each stage.

[0118] The input signal for the first stage is the remote sensing image to be detected, and the input signals for the remaining stages are the multi-scale tower features output from the previous stage.

[0119] In this preferred embodiment, the backbone network is composed of several transmission feature extraction modules stacked sequentially. In each stage, the current transmission feature extraction module performs convolution and channel splitting operations on the input signal to obtain a first feature branch stream and a second feature branch stream. The multi-dimensional information of the original features is preserved through parallel branch processing. The second feature branch stream is processed by convolution to increase the number of channels, depthwise separable convolution, and point convolution to reduce the number of channels to obtain a third feature branch stream. Among them, depthwise separable convolution reduces the amount of computation while maintaining the feature extraction capability, and the dynamic adjustment of the number of channels enhances the flexibility of feature expression. The first feature branch stream and the third feature branch stream are spliced ​​and fused to generate multi-scale tower features corresponding to each stage, realizing the effective integration of detailed features and deep features. The first stage uses the remote sensing image to be detected as the input signal, and the other stages use the multi-scale tower features output from the previous stage as the input. Through the hierarchical feature extraction method, the hierarchical representation of tower features is gradually deepened, improving the systematicness and completeness of feature extraction.

[0120] Furthermore, the feature fusion network includes several shallow transmission feature fusion modules, as well as a transmission feature extraction module and a deep transmission feature fusion module that are sequentially connected to each of the shallow transmission feature fusion modules.

[0121] The feature fusion network is used to fuse the multi-scale tower features in step S2, and to perform secondary feature extraction and deep fusion to output diverse tower features containing semantic understanding. This can be achieved through the following preferred implementation method, including steps S26-S28, as follows:

[0122] S26: According to each of the shallow transmission feature fusion modules, the multi-scale tower features output by each transmission feature extraction module in the backbone network are downsampled and channel adjusted, and then spliced ​​and fused to output each composite feature;

[0123] S27: Based on each transmission feature extraction module of the feature fusion network, perform secondary feature extraction on each composite feature and output feature maps of different resolutions;

[0124] S28: Based on the depth transmission feature fusion modules, sample and adjust the channels of each feature map and perform feature stitching to obtain the diverse tower features.

[0125] In this preferred embodiment, the shallow transmission feature fusion modules downsample and adjust the channels of the multi-scale tower features in the backbone network and then stitch them together to output composite features, achieving preliminary integration of multi-scale features. The transmission feature extraction modules of the feature fusion network perform secondary feature extraction on each composite feature and output feature maps of different resolutions, further improving the feature representation capability. The deep transmission feature fusion modules sample and adjust the channels of each feature map and stitch them together to obtain diverse tower features, enhancing the interaction between deep and shallow features and improving the network's recognition accuracy of tower targets in complex backgrounds.

[0126] Specifically, in the backbone network, multiple Transmission Feature Extraction (TFE) modules are stacked sequentially, forming four different stages: F2, F3, F4, and F5. The shallow transmission feature fusion (STFF) module is responsible for the initial auxiliary fusion of the multi-scale features extracted from the backbone network. Then, the TFE module is used for further tower feature extraction. Simultaneously, the deep transmission feature fusion (DTFF) module collects gradient information from each layer through a denser connection in a second top-down path. Finally, the head obtains diverse tower feature output information at multiple resolutions, and the tower target recognition loss function is calculated based on the output. Backpropagation training is then used to optimize the model's parameters. The specific structures of each module are as follows:

[0127] refer to Figure 7The diagram illustrates the structure of a transmission feature extraction module. The input information first undergoes a 1×1 convolution and a channel splitting operation, generating two feature branch streams. One branch stream retains the original feature information and directly enters the Concat feature concatenation operation, while the other stream is processed through N bottleneck structural units. The outputs of the bottleneck structures are ultimately concatenated together using Concat feature concatenation. In the bottleneck structure, a 1×1 convolution is first used to increase the number of channels, followed by a k×k depthwise separable convolution operation, and finally a 1×1 point convolution is used to reduce the number of channels and compensate for potential information loss caused by the depthwise separable convolution. This transmission feature extraction module can efficiently acquire multi-scale feature information about tower targets in the input data, further improving the accuracy of subsequent tower identification.

[0128] refer to Figure 8 The diagram shows a shallow transmission feature fusion module. Considering that preserving shallow spatial information in the backbone network is crucial for improving the detection capability of smaller tower targets, but since the information provided by the backbone network is relatively basic and easily interfered with, this application integrates shallow information as an auxiliary branch into the deep network and designs a shallow transmission feature fusion module, namely STFF-Module. Its main goal is to combine deep-level features in the backbone network with high-resolution shallow features, aiming to preserve rich localization details and abstract classification information about tower targets, so as to improve the network's spatial representation capability and feature stability of tower targets. In this module, F1 (shallow feature maps of the backbone network) serves as the primary input, and F2 (auxiliary deep features) as the secondary input. F1 first undergoes downsampling, reducing resolution through pooling or strided convolution. The downsampled features are then processed using 1×1 convolutions to adjust the number of channels, generating a channel-optimized feature C. F2 undergoes 2C channel processing (doubling the number of channels) to enhance the expressive power between feature channels. The channel-adjusted C is then concatenated with the enhanced 2C features across dimensions (concat channel addition) to form a composite 3C feature output. This STFF module is suitable for visual tasks requiring multi-scale feature fusion (such as multi-scale tower target recognition). Through cascaded dimensionality reduction and expansion operations, it achieves cross-layer interaction of multi-scale information while maintaining feature resolution.

[0129] refer to Figure 9 The diagram shows a schematic of a deep transmission feature fusion module. In this module, 1×1 convolutions are used to adjust the channels, controlling the number of channels for each input and output feature, ensuring that different input features occupy the corresponding proportions in the feature stitching operation. As shown in the figure, F1, F2, F3, and F4 represent feature maps at different resolutions. The fused features output by the DTFF-Module can simultaneously fuse information from four different scale layers, thereby significantly improving the recognition performance of various types of tower targets.

[0130] Step S3: Compare the target identification information of the tower to be detected with the target identification information of the reference tower, and output the identification result of the tilting state of the transmission tower corresponding to the remote sensing image to be detected;

[0131] The reference tower target identification information is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

[0132] Furthermore, step S3 can be implemented through the following preferred embodiments, including steps S31-S32, as follows:

[0133] S31: Extract the first position information and the first aspect ratio of the target identification frame of the target tower in the target tower identification information to be detected, and extract the second position information and the second aspect ratio of the target identification frame of the target tower in the reference target identification information;

[0134] S32: Combine the preset threshold, compare the first position information and the second position information, as well as the aspect ratio of the first border and the aspect ratio of the second border, and output the tilting state recognition result.

[0135] In specific implementation, such as Figure 10 The flowchart shown illustrates a method for comparing recognition results. This application designs a method to analyze and judge the state of tower targets based on the changes in the target bounding boxes of tower states in two time phases and the recognition and interpretation of the model. First, a reference remote sensing image (in which all towers are in a normal, untilted state) is used as the model input. The trained tower recognition model, namely the aforementioned multi-branch feature YOLO model, is used to recognize the target, obtaining the reference tower target recognition result (i.e., reference tower target recognition information). The target location of the real tower in the area is determined, and a list of tower state parameters (such as the aspect ratio of the bounding box) is constructed based on the target bounding boxes in normal state. Second, a remote sensing image of the same area to be detected (to detect towers that may contain abnormal states) is used as the model input. The model is used for recognition to obtain the corresponding recognition results, as follows. Figure 11 The comparison chart of the recognition results shown includes the following situations:

[0136] 1. At the same location in both the baseline remote sensing image and the remote sensing image to be detected, tower targets are identified, but the status box parameters change, and these towers are marked as abnormal. Figure 11 Case 1.

[0137] Second: If a tower is identified in the baseline remote sensing image, but not in the same location in the remote sensing image to be detected, it is marked as an abnormal tower. Figure 11 Situation 2.

[0138] 3. If the tower target is correctly identified at the same location in both the reference remote sensing image and the remote sensing image to be detected, and the status box remains unchanged, then the tower is judged to be in normal condition. Figure 11 Situation 3.

[0139] If it is case 1, then the target frame state parameters of the identified tower are further calculated, such as aspect ratio. By setting a threshold for comparison, if the aspect ratio of the same tower target changes significantly in the two consecutive remote sensing images, it indicates that the target tower may have tilted or even collapsed without overall structural damage. If the aspect ratio does not change significantly, it is marked as... Figure 11 The tower in normal condition as shown in case 3.

[0140] If the identification result is Case 2, it may indicate that the target shape of the pole at this location may be severely damaged or broken, resulting in a large difference from the original pole, and thus the model will identify it as a non-pole target. In this case, the pole point of this type can be marked as a pole target with abnormal condition and damage.

[0141] If the identification result is Case 3, that is, the same tower target is detected in both the reference remote sensing image and the remote sensing image to be detected, and the state parameters of the target box have not changed significantly, then the tower target is considered to be in an abnormal state.

[0142] In this preferred embodiment, the first position information and the first aspect ratio of the target identification box of the tower to be detected are extracted from the target identification information of the tower to be detected, and the second position information and the second aspect ratio of the target identification box of the tower to be detected are extracted from the target identification information of the reference tower to obtain the key parameters required for comparison. The first position information and the second position information are compared with a preset threshold to determine whether the tower has shifted position, and the first aspect ratio and the second aspect ratio of the second aspect ratio are compared to determine whether the tower has changed shape. The tilting state identification result is output through multi-dimensional parameter comparison, which improves the accuracy and reliability of anomaly detection.

[0143] In summary, compared with the prior art, the above embodiments of this application have the following beneficial effects: by acquiring the current remote sensing image of the transmission tower to be detected and the corresponding reference remote sensing image, a data foundation is provided for subsequent comparative analysis; the remote sensing image to be detected is input into a preset multi-branch feature YOLO model, which generates multi-scale tower features containing pixel-level details and low-level texture information through multi-stage feature extraction, thereby improving the feature capture capability of towers at different scales; the multi-scale tower features are fused and secondary feature extraction and deep fusion are performed to output diverse tower features containing semantic understanding, further enhancing the richness and representation capability of the features; the target identification information of the tower to be detected is output based on the diverse tower features, thereby achieving accurate classification and positioning of the tower target; the target identification information of the tower to be detected is compared with the reference tower target identification information to output the tilting state identification result, and anomaly detection is achieved by judging the state change; wherein, the multi-branch feature YOLO model is trained only using remote sensing image samples of transmission towers in normal state, without relying on scarce tilting tower samples, thus reducing the dependence threshold on abnormal samples. Meanwhile, multi-stage feature extraction ensures the comprehensiveness of features, secondary fusion enhances semantic understanding, normal sample training solves the problem of sample scarcity, and the comparison mechanism realizes state change detection. These features work together to solve the core problem of limited engineering application caused by reliance on the overturning sample library in existing technologies, and significantly improve the practicality and generalization ability of transmission tower overturning detection.

[0144] Example 2:

[0145] Please refer to Figure 2 Based on the same inventive concept, the present invention discloses a power transmission tower tilting detection system, comprising: a data acquisition module M1, a model recognition module M2, and a result output module M3;

[0146] The data acquisition module M1 is used to acquire the current remote sensing image of the transmission tower to be detected, as well as the corresponding reference remote sensing image.

[0147] The model recognition module M2 is used to input the remote sensing image to be detected into a preset multi-branch feature YOLO model, so that the multi-branch feature YOLO model performs multi-stage feature extraction on the remote sensing image to be detected, generates multi-scale tower features containing pixel-level details and low-level texture information, fuses the multi-scale tower features, performs secondary feature extraction and deep fusion, outputs diversified tower features containing semantic understanding, and outputs tower target recognition information to be detected based on the diversified tower features.

[0148] The multi-branch feature YOLO model is trained based on remote sensing image samples of power transmission towers in a normal, undamaged state.

[0149] Furthermore, the multi-branch feature YOLO model includes a backbone network, a feature fusion network, and a classification and recognition network. The backbone network is composed of several transmission feature extraction modules stacked and connected in sequence.

[0150] The model recognition module M2 includes: a multi-stage feature extraction unit;

[0151] The multi-stage feature extraction unit is used to perform convolution and channel splitting operations on the input signal according to the current transmission feature extraction module in each stage to obtain a first feature branch stream and a second feature branch stream. The second feature branch stream is then subjected to convolution to increase the number of channels, depth-separable convolution operation, and point convolution to decrease the number of channels to obtain a third feature branch stream. The first feature branch stream and the third feature branch stream are then spliced ​​and fused to generate multi-scale tower features corresponding to each stage.

[0152] The input signal for the first stage is the remote sensing image to be detected, and the input signals for the remaining stages are the multi-scale tower features output from the previous stage.

[0153] In this preferred embodiment, the backbone network is composed of several transmission feature extraction modules stacked sequentially. In each stage, the current transmission feature extraction module performs convolution and channel splitting operations on the input signal to obtain a first feature branch stream and a second feature branch stream. The multi-dimensional information of the original features is preserved through parallel branch processing. The second feature branch stream is processed by convolution to increase the number of channels, depthwise separable convolution, and point convolution to reduce the number of channels to obtain a third feature branch stream. Among them, depthwise separable convolution reduces the amount of computation while maintaining the feature extraction capability, and the dynamic adjustment of the number of channels enhances the flexibility of feature expression. The first feature branch stream and the third feature branch stream are spliced ​​and fused to generate multi-scale tower features corresponding to each stage, realizing the effective integration of detailed features and deep features. The first stage uses the remote sensing image to be detected as the input signal, and the other stages use the multi-scale tower features output from the previous stage as the input. Through the hierarchical feature extraction method, the hierarchical representation of tower features is gradually deepened, improving the systematicness and completeness of feature extraction.

[0154] Furthermore, the feature fusion network includes several shallow transmission feature fusion modules, as well as a transmission feature extraction module and a deep transmission feature fusion module that are sequentially connected to each of the shallow transmission feature fusion modules.

[0155] The model recognition module M2 further includes: a fusion unit, a secondary extraction unit, and a splicing unit;

[0156] The fusion unit is used to downsample and adjust the multi-scale tower features output by each transmission feature extraction module in the backbone network according to each of the shallow transmission feature fusion modules, and then splice and fuse them to output each composite feature.

[0157] The secondary extraction unit is used to perform secondary feature extraction on each composite feature according to each transmission feature extraction module of the feature fusion network, and output feature maps of different resolutions.

[0158] The splicing unit is used to sample and adjust the channels of each feature map according to each of the depth transmission feature fusion modules, and then splice the features to obtain the diverse tower features.

[0159] In this preferred embodiment, the shallow transmission feature fusion modules downsample and adjust the channels of the multi-scale tower features in the backbone network and then stitch them together to output composite features, achieving preliminary integration of multi-scale features. The transmission feature extraction modules of the feature fusion network perform secondary feature extraction on each composite feature and output feature maps of different resolutions, further improving the feature representation capability. The deep transmission feature fusion modules sample and adjust the channels of each feature map and stitch them together to obtain diverse tower features, enhancing the interaction between deep and shallow features and improving the network's recognition accuracy of tower targets in complex backgrounds.

[0160] Furthermore, the model recognition module M2 also includes: a training data acquisition unit, a preprocessing unit, a labeling unit, and a training unit;

[0161] The training data acquisition unit is used to acquire each original remote sensing image of the transmission tower.

[0162] The preprocessing unit is used to preprocess each of the original remote sensing images to obtain each remote sensing image; wherein, the preprocessing includes any one or more combinations of the following: geometric correction, radiometric correction and target correction;

[0163] The annotation unit is used to perform multiple rounds of identification and annotation of each unannotated pole target in each remote sensing image based on a preset pole identification model, until all remote sensing images are annotated and output each remote sensing image sample; and after each round of annotation, the annotation result of the current round is verified and the pole identification model is iteratively updated based on all currently verified annotation result data.

[0164] The training unit is used to train a preset original multi-branch feature YOLO model based on each remote sensing image sample to obtain the multi-branch feature YOLO model.

[0165] The original remote sensing images of transmission towers were acquired to provide basic data for model training. Preprocessing was performed on each original remote sensing image, including geometric correction, radiometric correction, and target correction. Geometric correction corrected geometric distortion, radiometric correction eliminated radiometric distortion, and target correction unified the tower imaging perspective, improving image quality and consistency. Based on a pre-set tower recognition model, unlabeled tower targets in each remote sensing image were identified and labeled in multiple rounds. After each round of labeling, the results were verified, and the tower recognition model was iteratively updated until all remote sensing images were labeled, outputting each remote sensing image sample. Progressive labeling through human-computer interaction improved the labeling efficiency and accuracy of the sample library. A pre-set original multi-branch feature YOLO model was trained based on each remote sensing image sample to obtain a multi-branch feature YOLO model, ensuring that the model could fully learn the features of normal towers and improve recognition performance.

[0166] Furthermore, the preprocessing unit includes: a target correction data acquisition subunit, an adjustment subunit, a projection difference correction calculation subunit, and a coordinate transformation subunit;

[0167] The target correction data acquisition subunit is used to acquire sensor parameter data corresponding to each of the original remote sensing images, as well as digital elevation model data, ground control point data and preset tower sample data for the same area.

[0168] The adjustment subunit is used to identify overlapping feature points in overlapping areas of each of the original remote sensing images based on a preset feature point matching algorithm, construct a connection point network, and combine the ground control point data to perform local area network adjustment processing and output the spatial position relationship of the images.

[0169] The projection difference correction calculation subunit is used to supplement and fuse the digital elevation model data according to the tower sample data to obtain digital tower elevation model data, and combine the sensor parameter data to calculate the projection difference correction value of each pixel in each of the original remote sensing images according to the preset projection difference calculation formula.

[0170] The coordinate transformation subunit is used to construct a geometric correction model based on the spatial positional relationship of the images and ground control point data, using a polynomial fitting method. It then combines the digital tower elevation model data and the projection difference correction values ​​to perform coordinate transformation adjustment on each pixel in the original remote sensing images, thereby obtaining remote sensing images without projection difference.

[0171] In this preferred embodiment, during target correction, sensor parameter data corresponding to each original remote sensing image, as well as digital elevation model data, ground control point data, and preset tower sample data for the same area are acquired to provide multi-source auxiliary information for correction. Based on a preset feature point matching algorithm, overlapping feature points in the overlapping areas of each original remote sensing image are identified, a connection point network is constructed, and local area network adjustment is performed in conjunction with ground control point data to output the spatial position relationship of the images, improving the relative positioning accuracy between images. Based on the tower sample data, the digital elevation model data is supplemented and fused to obtain digital tower elevation model data. Combined with sensor parameter data, the projection difference correction value of each pixel in each original remote sensing image is calculated according to a preset projection difference calculation formula, solving the projection difference problem caused by the lack of tower height information in traditional digital elevation models. Based on the spatial position relationship of the images and ground control point data, a geometric correction model is constructed using a polynomial fitting method. Combined with the digital tower elevation model data and each projection difference correction value, the coordinate transformation of each pixel in each original remote sensing image is adjusted to obtain each remote sensing image without projection difference, which can eliminate the tower imaging tilt caused by different satellite perspectives and unify the initial geometric attitude of the tower.

[0172] Furthermore, the training unit includes: a loss function construction subunit and a training subunit;

[0173] The loss function construction subunit is used to construct a loss function that fuses localization loss and classification loss, as shown below:

[0174] L total =a·L cls +b·L reg Among them, the classification loss function L cls and localization loss function L reg They are represented as follows:

[0175] L cls =-α(1-p t ) γ log(p t );

[0176]

[0177] Where a and b represent weights, α is the weight balancing the positive and negative samples, and p t γ represents the probability that the model's predicted result belongs to the normal tower category, γ is the focusing factor, and t i Let t be the parameters of the i-th bounding box predicted by the model, where (x, y), w, and h represent the coordinates of the center point, width, and height of the bounding box, respectively. i * is the true value of the i-th bounding box parameter, and β is the threshold that controls the squared loss and the linear loss;

[0178] The training subunit is used to train the original multi-branch feature YOLO model by combining each of the remote sensing image samples and the loss function, so as to obtain the multi-branch feature YOLO model.

[0179] In this preferred embodiment, a loss function integrating localization loss and classification loss is constructed. The classification loss weight and localization loss weight in the total loss function are used to balance the contribution ratios of classification loss and localization loss, ensuring the model simultaneously focuses on classification accuracy and localization accuracy. The weight parameter balancing positive and negative samples in the classification loss function addresses class imbalance. The probability value of the model's predicted result belonging to the normal tower category quantifies the model's prediction confidence for the target category. The focusing factor reduces the contribution of easily classified samples to the total loss, making the model focus more on difficult samples. The localization loss function calculates the loss value through a piecewise function design. When the absolute error between the predicted tower target bounding box parameters and the true bounding box parameters is less than the threshold controlling the squared loss and linear loss, the squared loss is used to promote accurate parameter convergence. When the absolute error is greater than or equal to this threshold, the linear loss is used to avoid gradient explosion. This piecewise calculation method balances the loss contribution under different error magnitudes, improving the stability and convergence speed of model training.

[0180] The result output module M3 is used to compare the target identification information of the tower to be detected with the target identification information of the reference tower, and output the tilting state identification result of the transmission tower corresponding to the remote sensing image to be detected;

[0181] The reference tower target identification information is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

[0182] Furthermore, the result output module M3 includes: an information extraction unit and a comparison unit;

[0183] The information extraction unit is used to extract the first position information and the first aspect ratio of the pole target identification box in the pole target identification information to be detected, and to extract the second position information and the second aspect ratio of the pole target identification box in the reference pole target identification information.

[0184] The comparison unit is used to combine a preset threshold, compare the first position information and the second position information, as well as the aspect ratio of the first border and the aspect ratio of the second border, and output the tilting state recognition result.

[0185] In this preferred embodiment, the first position information and the first aspect ratio of the target identification box of the tower to be detected are extracted from the target identification information of the tower to be detected, and the second position information and the second aspect ratio of the target identification box of the tower to be detected are extracted from the target identification information of the reference tower to obtain the key parameters required for comparison. The first position information and the second position information are compared with a preset threshold to determine whether the tower has shifted position, and the first aspect ratio and the second aspect ratio of the second aspect ratio are compared to determine whether the tower has changed shape. The tilting state identification result is output through multi-dimensional parameter comparison, which improves the accuracy and reliability of anomaly detection.

[0186] In summary, compared with the prior art, the embodiments of this application have the following beneficial effects: by acquiring the current remote sensing image of the transmission tower to be detected and the corresponding reference remote sensing image, a data foundation is provided for subsequent comparative analysis; the remote sensing image to be detected is input into a preset multi-branch feature YOLO model, which generates multi-scale tower features containing pixel-level details and low-level texture information through multi-stage feature extraction, thereby improving the feature capture capability of towers at different scales; the multi-scale tower features are fused and secondary feature extraction and deep fusion are performed to output diverse tower features containing semantic understanding, further enhancing the richness and representation capability of the features; the target identification information of the tower to be detected is output based on the diverse tower features, thereby achieving accurate classification and positioning of the tower target; the target identification information of the tower to be detected is compared with the reference tower target identification information to output the tilting state identification result, and anomaly detection is achieved by judging the state change; wherein, the multi-branch feature YOLO model is trained only using remote sensing image samples of transmission towers in normal state, without relying on scarce tilting tower samples, thus reducing the dependence threshold on abnormal samples. Meanwhile, multi-stage feature extraction ensures the comprehensiveness of features, secondary fusion enhances semantic understanding, normal sample training solves the problem of sample scarcity, and the comparison mechanism realizes state change detection. These features work together to solve the core problem of limited engineering application caused by reliance on the overturning sample library in existing technologies, and significantly improve the practicality and generalization ability of transmission tower overturning detection.

[0187] Example 3:

[0188] Figure 3 A structural diagram of a power transmission tower tilting detection device according to this application is provided. (See diagram below.) Figure 3 As shown, the overturning detection device for the transmission tower may include: a processor N1, a memory N2, a data interface N3, and a communication bus N4.

[0189] Wherein: processor N1, memory N2, and data interface N3 communicate with each other through communication bus N4; data interface N3 is used for data communication with other devices such as input devices or output devices; processor N1 is used to execute program N5, which can specifically execute the relevant steps in any of the above embodiments of the transmission tower tilting detection method.

[0190] Specifically, program N5 may include program code, which includes computer-executable instructions.

[0191] The processor N1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The power transmission tower tilting detection device includes one or more processors, which may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0192] Memory N2 is used to store program N5. Memory N2 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0193] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments in this application are not directed to any particular programming language.

[0194] Example 4:

[0195] This invention also provides a computer-readable storage medium storing at least one executable instruction that, when executed on a power transmission tower tilting detection device / system, causes the power transmission tower tilting detection device / system to perform one of the power transmission tower tilting detection methods described in any of the above method embodiments.

[0196] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the purpose of simplification and aiding understanding of one or more aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0197] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A method of detecting a collapse of a power transmission tower, characterized by, The method comprises the following steps: acquire the current remote sensing image to be detected of a power transmission tower and a corresponding reference remote sensing image; input the remote sensing image to be detected into a preset multi-branch feature YOLO model, so that the multi-branch feature YOLO model performs multi-stage feature extraction on the remote sensing image to be detected, generates each multi-scale tower feature containing pixel-level details and low-level texture information, fuses each multi-scale tower feature, and performs secondary feature extraction and deep fusion to output diversified tower features containing semantic understanding, and according to the diversified tower features, output tower target recognition information to be detected; wherein the multi-branch feature YOLO model is obtained by training remote sensing image samples of power transmission towers in a normal, non-toppling and undamaged state; compare the tower target recognition information to be detected with reference tower target recognition information, and output the toppling state recognition result of the power transmission tower corresponding to the remote sensing image to be detected; wherein the reference tower target recognition information is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

2. A method of collapse detection of a power transmission tower as claimed in claim 1, wherein, The multi-branch feature YOLO model comprises a backbone network, a feature fusion network and a classification recognition network, and the backbone network is formed by stacking and connecting a plurality of transmission feature extraction modules in sequence; the multi-stage feature extraction on the remote sensing image to be detected to generate each multi-scale tower feature containing pixel-level details and low-level texture information comprises: in each stage, according to the current transmission feature extraction module, the input signal is subjected to convolution and channel splitting operation to obtain a first feature branch stream and a second feature branch stream, and the second feature branch stream is subjected to convolution expansion channel number processing, depth separable convolution operation and point convolution channel number reduction processing to obtain a third feature branch stream, and then the first feature branch stream and the third feature branch stream are spliced and fused to generate a multi-scale tower feature corresponding to each stage; wherein the input signal of the first stage is the remote sensing image to be detected, and the input signal of the remaining stages is the multi-scale tower feature output by the previous stage.

3. A method of collapse detection of a power transmission tower as claimed in claim 2, wherein, The feature fusion network comprises a plurality of shallow transmission feature fusion modules, and a transmission feature extraction module and a deep transmission feature fusion module connected in sequence with each of the shallow transmission feature fusion modules; the feature fusion network is used for fusing each multi-scale tower feature, performing secondary feature extraction and deep fusion, and outputting diversified tower features containing semantic understanding, which comprises: according to each shallow transmission feature fusion module, each multi-scale tower feature output by each transmission feature extraction module in the backbone network is subjected to down-sampling and channel adjustment, and spliced and fused to output each composite feature; according to each transmission feature extraction module of the feature fusion network, each composite feature is subjected to secondary feature extraction to output feature maps of different resolutions; according to each deep transmission feature fusion module, each feature map is sampled and channel adjusted and spliced to obtain the diversified tower features.

4. A method of collapse detection of a power transmission tower as claimed in claim 1, wherein, The multi-branch feature YOLO model is trained according to remote sensing image samples of power transmission towers in a normal, non-toppling and non-damaged state, and comprises: Obtaining each original remote sensing image of a power transmission tower; Pretreating each of the original remote sensing images to obtain each remote sensing image; wherein the pretreatment comprises any one or a combination of the following: geometric correction, radiation correction and target correction; Based on a preset tower identification model, each unlabeled tower target in each remote sensing image is identified and labeled in multiple rounds until all remote sensing images are labeled, and each remote sensing image sample is output; and after each round of labeling, the labeling results of the current round are approved, and the tower identification model is iteratively updated based on all currently approved labeling results data; According to each of the remote sensing image samples, the pre-set original multi-branch feature YOLO model is trained to obtain the multi-branch feature YOLO model.

5. A method of collapse detection of a power transmission tower as claimed in claim 4 wherein, The target correction comprises: Obtaining sensor parameter data corresponding to each of the original remote sensing images, as well as digital elevation model data, ground control point data and preset tower sample data of the same region; Based on a preset feature point matching algorithm, each coincident feature point in the overlapping region of each of the original remote sensing images is identified, a connection point network is constructed, and local network adjustment processing is performed in combination with the ground control point data to output an image spatial position relationship; According to the tower sample data, the digital elevation model data is supplemented and fused to obtain digital tower elevation model data, and in combination with the sensor parameter data, the projection difference correction value of each pixel in each of the original remote sensing images is calculated according to a preset projection difference calculation formula; Based on the image spatial position relationship and the ground control point data, a geometric correction model is constructed by a polynomial fitting method, and in combination with the digital tower elevation model data and each of the projection difference correction values, coordinate transformation and adjustment are performed on each pixel in each of the original remote sensing images to obtain each of the remote sensing images without projection difference.

6. A method of collapse detection of a power transmission tower as claimed in claim 4 wherein, The training of the pre-set original multi-branch feature YOLO model according to each of the remote sensing image samples to obtain the multi-branch feature YOLO model comprises: A loss function that fuses positioning loss and classification loss is constructed and is expressed as follows: L total = a * L cls + b * L reg ; wherein the classification loss function L cls and the localization loss function L reg are represented as follows: L cls = -a(1 - p t ) γ log(p t ) wherein a and b represent weights, a is a weight for balancing positive and negative samples, p t is a probability value that the model prediction result belongs to the normal tower category, and g is a focusing factor, t i is the i-th bounding box parameter predicted by the model, and (x, y), w and h represent the center point coordinates, width and height of the bounding box, respectively, t i * is the true value of the i-th bounding box parameter, and b is a threshold value for controlling the division between the square loss and the linear loss; The original multi-branch feature YOLO model is trained in combination with each of the remote sensing image samples and the loss function to obtain the multi-branch feature YOLO model.

7. A method of collapse detection of a power transmission tower as claimed in claim 1, wherein, The comparison of the to-be-detected tower target recognition information and the reference tower target recognition information to output the toppling state recognition result of the power transmission tower corresponding to the to-be-detected remote sensing image comprises: Extracting first position information and a first aspect ratio of a tower target recognition box in the to-be-detected tower target recognition information, and extracting second position information and a second aspect ratio of a tower target recognition box in the reference tower target recognition information; In combination with a preset threshold, the first position information and the second position information, and the first aspect ratio and the second aspect ratio are compared to output the toppling state recognition result.

8. A system for detecting a collapse of a power transmission tower, the system comprising: It comprises: A data acquisition module, a model identification module and a result output module; The data acquisition module is configured to acquire a current to-be-detected remote sensing image of a power transmission tower and a corresponding reference remote sensing image. The model identification module is configured to input the to-be-detected remote sensing image into a preset multi-branch feature YOLO model, so that the multi-branch feature YOLO model performs multi-stage feature extraction on the to-be-detected remote sensing image, generates each multi-scale tower feature containing pixel-level details and low-level texture information, fuses each multi-scale tower feature, performs secondary feature extraction and deep fusion, and outputs diversified tower features containing semantic understanding, and according to the diversified tower features, outputs to-be-detected tower target identification information; the multi-branch feature YOLO model is trained according to remote sensing image samples of power transmission towers in a normal, non-toppling and non-damaged state. The result output module is configured to compare the to-be-detected tower target identification information and reference tower target identification information, and output a toppling state identification result of the power transmission tower corresponding to the to-be-detected remote sensing image; the reference tower target identification information is obtained by inputting the reference remote sensing image into the multi-branch feature YOLO model for processing.

9. A tower toppling detection apparatus for a power transmission tower, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein, The computer program, when loaded into the processor, implements the steps of the power transmission tower toppling detection method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program, when executed by the processor, implements the steps of the power transmission tower toppling detection method according to any one of claims 1-7.

Images