A method and device for detecting multiple fittings of a power transmission line

Abstract

The application is suitable for the technical field of power transmission line inspection, and provides a power transmission line multi-fixture detection method and device, which comprises the following steps: obtaining aerial image data set; constructing a fixture average relative distance prior matrix according to the aerial image data set; performing coarse detection on the fixture images in the aerial image data set to obtain coarse detection results; obtaining a structure scene sub-region with dense fixture distribution through a clustering algorithm according to the coarse detection results and the fixture average distance prior matrix; performing fine detection on the structure scene sub-region to obtain fine detection results; and combining the coarse detection results with the fine detection results to obtain final multi-fixture detection results. The application can reasonably utilize the information contained in the high-resolution fixture images, avoid the loss of small target information in the processing process, and effectively solve the dense occlusion problem in the aggregated structure of the fixtures.

Description

Technical Field

[0001] This invention relates to the field of power transmission line inspection technology, and in particular to a method and apparatus for detecting multiple hardware components in power transmission lines. Background Technology

[0002] The stable operation of transmission lines has a crucial impact on the power system. Hardware, as a vital component of transmission lines, plays a role in fixing, connecting, and protecting them. However, transmission lines are often located in harsh environments, sparsely populated areas, and complex terrain, making hardware prone to damage, corrosion, and misalignment. Failure to maintain these components promptly can lead to transmission line failures or even outages, resulting in significant economic losses. Therefore, regular inspection and maintenance of hardware is a crucial prerequisite for ensuring the stable operation of transmission lines.

[0003] Many deep learning-based methods exist for hardware detection in existing technologies. Data sets for power transmission line components primarily consist of high-resolution UAV aerial images. However, current advanced deep learning models are incompatible with high-resolution images, often requiring downsampling before subsequent learning. This process inevitably leads to information loss. For small hardware targets and combinations of hardware in power transmission lines, this information loss further impacts detection accuracy. Therefore, effectively addressing the issues of small targets and dense occlusion between multiple hardware components in aerial hardware images, and rationally utilizing the information contained in high-resolution aerial hardware images to improve detection accuracy, is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This invention provides a method and apparatus for detecting multiple hardware fittings in power transmission lines, in order to solve the problems of small targets and dense occlusion in aerial images of power transmission line hardware fittings.

[0005] In a first aspect, embodiments of the present invention provide a method for detecting multiple fittings in power transmission lines, including: acquiring an aerial image dataset;

[0006] Based on the aerial image dataset, construct the prior matrix of average relative distance of hardware;

[0007] Coarse detection is performed on the hardware images in the aerial image dataset to obtain coarse detection results;

[0008] Based on the coarse detection results and the prior matrix of the average distance between hardware fittings, a clustering algorithm is used to obtain the structural scene sub-regions with densely distributed hardware fittings.

[0009] The sub-regions of the structured scene are subjected to fine detection to obtain fine detection results;

[0010] The coarse inspection results are combined with the fine inspection results to obtain the final multi-fitting inspection results.

[0011] As another embodiment of this application, the step of constructing the average relative distance prior matrix of hardware based on the aerial image dataset includes:

[0012] Based on the aerial image dataset, obtain the hardware category set T and the size information of the images in the aerial image dataset;

[0013] Arbitrarily select the s-th type and t-th type of hardware from the hardware category set T, and determine the quantity of the s-th type and t-th type of hardware in any image in the aerial image dataset, as well as the coordinates of their respective real center points.

[0014] Based on the size information of the images in the aerial image dataset and the coordinates of the true center point, calculate the relative distance between the i-th type s hardware and the j-th type t hardware in any image in the aerial image dataset;

[0015] Based on the above method of calculating the relative distance between the i-th type s-fitting hardware and the j-th type t-fitting hardware in any image of the aerial image dataset, the relative distance between the s-th type s-fitting hardware and the t-th type t-fitting hardware in all images of the aerial image dataset is determined.

[0016] Based on the relative distances between type s and type t hardware in all images of the aerial image dataset, calculate the average relative distance between type s and type t hardware.

[0017] Based on the above method of calculating the average relative distance between type s hardware and type t hardware, calculate the average relative distance between all pairs of hardware of all types in the hardware category set T, and construct the hardware average relative distance prior matrix.

[0018] As another embodiment of this application, the step of performing coarse detection on the hardware images in the aerial image dataset to obtain coarse detection results includes:

[0019] The hardware images in the aerial image dataset are downsampled to obtain coarse detection regions;

[0020] Feature extraction is performed on the coarse detection region to obtain the basic features of the coarse detection region;

[0021] Based on the basic features of the coarse detection region, the corresponding coarse detection region proposal box is calculated through the region proposal network;

[0022] Based on the proposed coarse detection region bounding box, the corresponding coarse detection bounding box regression result and coarse detection classification result are calculated by the coarse detector.

[0023] The coarse detection bounding box regression results and the coarse detection classification results are processed to obtain the coarse detection results.

[0024] As another embodiment of this application, the step of obtaining a structural scene sub-region with densely distributed hardware based on the coarse detection results and the hardware average distance prior matrix through a clustering algorithm includes:

[0025] Determine the neighborhood density threshold of the clustering algorithm;

[0026] Based on the coarse detection classification results, the size information of the images in the aerial image dataset, and the prior matrix of the average relative distance of the hardware, the neighborhood radius of the clustering algorithm is determined.

[0027] Based on the coarse detection results, the sample set for the clustering algorithm is determined;

[0028] Based on the neighborhood density threshold, neighborhood radius, and sample set of the clustering algorithm, combined with the prior matrix of the average relative distance of the hardware, the structural scene sub-regions with densely distributed hardware that need to be precisely detected are obtained.

[0029] As another embodiment of this application, the step of performing fine detection on the sub-region of the structural scene to obtain fine detection results includes:

[0030] The hardware images in the aerial image dataset are cropped and downsampled to obtain a precise detection region;

[0031] Feature extraction is performed on the fine detection region to obtain the basic features of the fine detection region;

[0032] Based on the basic features of the fine detection region, the corresponding fine detection region proposal box is calculated and obtained through the region proposal network;

[0033] Within the structured scene sub-region, based on the proposed fine detection region bounding box, the corresponding fine detection bounding box regression result and fine detection classification result are calculated by the fine detector.

[0034] The regression results of the precise detection bounding boxes and the classification results of the precise detection are processed to obtain the precise detection results.

[0035] As another embodiment of this application, the step of extracting features from the coarse detection region to obtain the basic features of the coarse detection region includes:

[0036] According to F coarse =∏ CNN (X i |θ resnet101 The basic features of the coarse detection region are obtained;

[0037] Among them, F coarse Indicates from X i The basic features of the coarse detection region extracted from Xi ∏ represents the coarse detection region obtained by downsampling the hardware image in the i-th aerial image dataset. CNN (·|θ resnet101 ) represents the feature extraction function, θ resnet101 This represents the trainable parameters of the backbone network ResNet101 during the feature extraction process;

[0038] The step of extracting features from the fine detection region to obtain the basic features of the fine detection region includes:

[0039] according to The basic features of the precision detection region are obtained;

[0040] in, Indicates from A j The basic features of the fine detection region extracted from A j This represents the fine detection region obtained by cropping and downsampling the hardware images in the aerial image dataset, and j represents the number of structural scene sub-regions.

[0041] As another embodiment of this application, the step of calculating the corresponding coarse detection region proposal box through a region proposal network based on the basic features of the coarse detection region includes:

[0042] According to R coarse =∏ RPN1 (F coarse |θ RPN1 This yields a coarse detection region suggestion box;

[0043] Among them, R coarse Indicates from F coarse Extracted coarse detection region proposal, θ RPN1 This represents the trainable parameters in the coarse detection region proposal network, ∏ RPN1 (·|θ RPN1 () represents the function for extracting suggestion boxes for coarse detection regions;

[0044] The step of calculating the corresponding precise detection region proposal box through a region proposal network based on the basic features of the precise detection region includes:

[0045] according to Obtain the suggested bounding box for the precise detection area;

[0046] in, Indicates from Extracted precise detection region proposal, θ RPN2 This represents the trainable parameters in the fine detection region proposal network, ∏ RPN2 (·|θ RPN2 ) represents the function for extracting suggestion boxes for the fine detection region.

[0047] As another embodiment of this application, based on the coarse detection region proposal, the corresponding coarse detection bounding box regression result and coarse detection classification result are calculated by the coarse detector, including:

[0048] Based on the basic features of the coarse detection region and the proposed bounding box of the coarse detection region, the proposed features of the coarse detection region are obtained;

[0049] According to (B) coarse C coarse )=∏ Detector1 (P coarse |θ Detector1 The coarse detection bounding box regression results and coarse detection classification results are obtained.

[0050] Among them, B coarse C represents the coarse detection bounding box regression result. coarse P represents the coarse detection and classification result. coarse θ represents the suggested features for the coarse detection region. Detector1 ∏ represents the learnable parameters in the coarse detector. Detector1 (·|θ Detector1 ) is the coarse detector processing function.

[0051] As another embodiment of this application, the step of calculating the corresponding fine detection bounding box regression result and fine detection classification result by a fine detector based on the fine detection region proposal box in the structural scene sub-region includes:

[0052] Based on the basic features of the fine detection region and the suggestion box of the fine detection region, the suggested features of the fine detection region are obtained;

[0053] Under the aforementioned structural scene sub-region, according to We obtain the regression results of the precise detection bounding boxes and the classification results of the precise detection.

[0054] in, This indicates the regression results of the precise detected bounding boxes. This indicates the results of the precise detection and classification. θ represents the suggested features for the precise detection region. Detector2 ∏ represents the learnable parameters in the precision detector. Detector2 (·|θ Detector2 ) is the processing function for the precision detector.

[0055] Secondly, embodiments of the present invention provide a multi-fitting testing device for power transmission lines, comprising:

[0056] The acquisition module is used to acquire aerial image datasets;

[0057] The data processing module is used to construct a prior matrix of the average relative distance of hardware based on the aerial image dataset;

[0058] The coarse detection module is used to perform coarse detection on the hardware images in the aerial image dataset to obtain coarse detection results.

[0059] The structural scene sub-region selection module is used to obtain structural scene sub-regions with dense hardware distributions by using a clustering algorithm based on the coarse detection results and the hardware average distance prior matrix.

[0060] The fine detection module is used to perform fine detection on the sub-regions of the structure scene and obtain fine detection results;

[0061] The combination module is used to combine the coarse inspection results with the fine inspection results to obtain the final multi-fitting inspection results.

[0062] This invention provides a method and apparatus for detecting multiple hardware fittings on power transmission lines based on aerial images. By constructing a prior matrix of average relative distances between different types of hardware fittings based on the acquired aerial image dataset of hardware fittings, and by clustering the structural scene sub-regions with dense hardware fitting distribution based on the constructed prior matrix of average relative distances of hardware fittings and the coarse detection results, fine detection can be performed. This method can make reasonable use of the information contained in the high-resolution hardware fitting images, avoid the loss of small target information during the processing, and effectively solve the problem of dense occlusion in the hardware fitting aggregation structure. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the implementation of the multi-fitting detection method for power transmission lines provided in this embodiment of the invention.

[0065] Figure 2 This is a flowchart of constructing the prior matrix of average relative distance of hardware provided in an embodiment of the present invention;

[0066] Figure 3 This is a flowchart provided by an embodiment of the present invention for performing coarse detection on hardware images in an aerial image dataset to obtain coarse detection results;

[0067] Figure 4 This is a flowchart provided by an embodiment of the present invention, which obtains a structural scene sub-region with dense hardware distribution through a clustering algorithm based on coarse detection results and a priori matrix of average hardware distance;

[0068] Figure 5 This is a flowchart provided by an embodiment of the present invention for performing fine detection on a sub-region of a structural scene and obtaining fine detection results;

[0069] Figure 6 This is a schematic diagram of the transmission line multi-fitting detection device provided in an embodiment of the present invention;

[0070] Figure 7 A schematic diagram of the terminal device provided in an embodiment of the present invention. Detailed Implementation

[0071] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0072] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0073] Figure 1 A flowchart illustrating the implementation of a multi-fitting testing method for power transmission lines, as provided in this embodiment of the invention, is detailed below:

[0074] Step 101: Obtain the aerial image dataset.

[0075] Optionally, hardware fittings often maintain the stable operation of transmission lines through combined connections, thus they are often clustered together. Aerial images of the hardware fittings on transmission lines are obtained using drone aerial photography technology, forming an aerial image dataset.

[0076] Step 102: Construct the prior matrix of the average relative distance of the hardware based on the aerial image dataset.

[0077] In practice, hardware fittings are often used in combination to maintain the stable operation of transmission lines, so hardware fittings in aerial images often appear clustered together. Treating these clustered structures as structural scenarios, different structural scenarios are applied based on the actual needs of different parts of the transmission line. These structural scenarios share similarities, such as anti-galloping structural scenarios, suspension insulator structural scenarios, and equalizing shielding ring structural scenarios, each with its own unique hardware combination characteristics. Therefore, the prior matrix of average relative distances between different categories of hardware fittings obtained from aerial image datasets can encompass the relative distance characteristics between different categories of hardware fittings in various structural scenarios.

[0078] Optionally, in step 102, the prior matrix of the average relative distance of the hardware is constructed based on the aerial image dataset. This may include, for example: Figure 2 The steps are shown.

[0079] Step 201: Based on the aerial image dataset, obtain the hardware category set T and the size information of the images in the aerial image dataset.

[0080] Optionally, this step can obtain a hardware category set T based on the types of hardware contained in the images in the aerial image dataset. In this embodiment, the aerial image dataset contains 1838 images. The dataset is divided into a training set and a test set in an 8:2 ratio. The training set contains 1482 images and 19704 hardware targets, while the test set contains 356 images and 3632 hardware targets. Based on this aerial image dataset, the obtained hardware category set T includes 14 types of hardware targets: pre-twisted suspension clamps (PT), bag-type suspension clamps (BT), compression tension clamps (CT), wedge-type tension clamps (WT), hanging plates (HB), U-shaped hanging rings (UT), connecting plates (YP), parallel groove clamps (PG), vibration dampers (SH), spacers (SP), equalizing rings (GR), shielding rings (SR), counterweights (WE), and adjusting plates (AB). It should be noted that the categories of hardware included in the obtained hardware category set can be determined based on the analyzed aerial image dataset and specific circumstances. In this embodiment, the 14 categories of hardware targets included in the obtained hardware category set are merely an illustrative example.

[0081] Optionally, this step can obtain the size information of the images in the aerial image dataset based on the image information in the dataset. The size information may include the image width w. org and the height h of the image org .

[0082] Step 202: Randomly select the s-th type and t-th type of hardware from the hardware category set T, and determine the quantity of the s-th type and t-th type of hardware in any image in the aerial image dataset, as well as the coordinates of their respective real center points.

[0083] Optionally, this step may include determining the coordinates of the true center point of all s-th type hardware in any image of the aerial image dataset. The true center point coordinates corresponding to all t-th type hardware

[0084] Where, n s n represents the quantity of the s-th type of hardware in the image. t This represents the quantity of the t-th type of hardware in the image.

[0085] Optionally, in this embodiment, if an image contains type s hardware but not type t hardware, then the quantity of type t hardware in the image is 0, and the set of true center point coordinates corresponding to type t hardware in the image is empty. That is, if an image contains type s hardware but not type t hardware, then the quantity n of type t hardware in the image is... t =0, the coordinates of the true center point corresponding to the t-th type of hardware in the image are

[0086] Optionally, in this embodiment, if an image contains neither type s nor type t of hardware, the quantity n corresponding to the two types of hardware in the image is... s =n t =0, the corresponding true center point coordinates are

[0087] Step 203: Based on the size information of the images in the aerial image dataset and the coordinates of the true center point, calculate the relative distance between the i-th type s-fitting and the j-th type t-fitting in any image in the aerial image dataset.

[0088] Since the images in the aerial image dataset are mostly high-resolution images, but there are still some differences in resolution, it is not suitable to construct the prior matrix by calculating the absolute distance. Therefore, the prior matrix is ​​constructed by using the relative distance to represent the structural scene characteristics between the hardware.

[0089] Optional, this step can be based on Obtain the relative distance between the i-th s-type hardware and the j-th t-type hardware;

[0090] in, This represents the relative distance between the i-th type s hardware and the j-th type t hardware.

[0091] Optionally, in this embodiment, if an image does not simultaneously contain both type s and type t hardware, that is, if the set of true center point coordinates corresponding to either type s or type t hardware in the image is empty, then the relative distance dist(l) between type s and type t hardware in the image is... s ,l t ) = -1.

[0092] Step 204: Based on the method described above for calculating the relative distance between the i-th type s-fitting device and the j-th type t-fitting device in any image of the aerial image dataset, determine the relative distance between the s-th type s-fitting device and the t-th type t-fitting device in all images of the aerial image dataset.

[0093] Step 205: Calculate the average relative distance between type s hardware and type t hardware in all images of the aerial image dataset.

[0094] Optional, this step can be based on The average relative distance between the s-th type of hardware and the t-th type of hardware is obtained, where num is the number of images in the aerial image dataset.

[0095] Step 206: Based on the above method of calculating the average relative distance between type s hardware and type t hardware, calculate the average relative distance between all pairs of hardware of all types in the hardware category set T, and construct the hardware average relative distance prior matrix.

[0096] Optionally, the prior matrix of average relative distance between hardware components obtained in this step is a symmetric matrix. The value of the element in the s-th row and t-th column of this prior matrix represents the average relative distance between hardware components of type s and type t. If the value of the element in the s-th row and t-th column of the prior matrix is ​​equal to -1, it indicates that combinations of hardware components of types s and t will not appear in the aerial image dataset. This also suggests that the structural scenes of the hardware components are similar, and the probability of some hardware component combinations appearing simultaneously is low.

[0097] Step 103: Perform coarse detection on the hardware images in the aerial image dataset to obtain coarse detection results.

[0098] Optionally, in step 103, a coarse detection is performed on the hardware images in the aerial image dataset to obtain coarse detection results, which may include, for example: Figure 3 The steps are shown.

[0099] Step 301: Downsample the hardware images in the aerial image dataset to obtain coarse detection regions.

[0100] Optionally, in this embodiment, the hardware image with a resolution of 5760×3840 can be downsampled using the Faster R-CNN model to obtain a coarse detection area with a resolution of 900×600.

[0101] The Faster R-CNN model is an algorithm model for object detection. When extracting features from high-resolution images, it is necessary to first perform downsampling on the image to reduce the size of the original image, thereby filtering out redundant background information in the original image and retaining key feature information.

[0102] Step 302: Extract features from the coarse detection region to obtain the basic features of the coarse detection region.

[0103] Optionally, this step can be based on F coarse=∏ CNN (X i |θ resnet101 The basic features of the coarse detection region are obtained;

[0104] Among them, F coarse Indicates from X i The basic features of the coarse detection region extracted from X i ∏ represents the coarse detection region obtained by downsampling the hardware image in the i-th aerial image dataset. CNN (·|θ resnet101 ) represents the feature extraction function, θ resnet101 This represents the trainable parameters of the backbone network ResNet101 during the feature extraction process.

[0105] Step 303: Based on the basic features of the coarse detection region, calculate the corresponding coarse detection region proposal box through the region proposal network.

[0106] Optionally, this step can be based on R coarse =∏ RPN1 (F coarse |θ RPN1 This yields a coarse detection region suggestion box;

[0107] Among them, R coarse Indicates from F coarse Extracted coarse detection region proposal, θ RPN1 This represents the trainable parameters in the coarse detection region proposal network, ∏ RPN1 (·|θ RPN1 ) represents the function for extracting suggestion boxes for coarse detection regions.

[0108] Step 304: Based on the proposed coarse detection region bounding boxes, calculate the corresponding coarse detection bounding box regression results and coarse detection classification results using the coarse detector.

[0109] Optionally, this step may also include obtaining coarse detection region suggested features based on the basic features of the coarse detection region and the suggested bounding box of the coarse detection region.

[0110] Optionally, this step can be based on (B) coarse C coarse )=∏ Detector1 (P coarse |θ Detector1 The coarse detection bounding box regression results and coarse detection classification results are obtained.

[0111] Among them, B coarse C represents the coarse detection bounding box regression result. coarse P represents the coarse detection and classification result. coarse θ represents the suggested features for the coarse detection region. Detector1∏ represents the learnable parameters in the coarse detector. Detector1 (·|θ Detector1 ) is the coarse detector processing function.

[0112] Step 305: Process the coarse detection bounding box regression results and coarse detection classification results to obtain the coarse detection results.

[0113] Optionally, this step also includes transforming and projecting the coarse detection bounding box regression results, removing redundant boxes using NMS based on the coarse detection classification results, and then mapping the coarse detection prediction boxes onto the hardware images in the aerial image dataset to obtain the coarse detection results.

[0114] Step 104: Based on the coarse detection results and the prior matrix of the average distance between hardware and fittings, obtain the structural scene sub-regions with dense hardware distribution through clustering algorithm.

[0115] Optionally, in this embodiment, the structural scene sub-region with densely distributed hardware is obtained using the DBSCAN clustering algorithm. The DBSCAN clustering algorithm is a density-based clustering algorithm with two basic parameters: neighborhood radius and neighborhood density threshold. It can handle clusters of different sizes and shapes without requiring a pre-specified number of clusters, and thus cluster the structural scene sub-region with densely distributed hardware. It should be noted that the choice of clustering algorithm can be determined based on specific circumstances. In this embodiment, obtaining the structural scene sub-region with densely distributed hardware using the DBSCAN clustering algorithm is merely an illustrative example.

[0116] Optionally, in step 104, based on the coarse detection results and the prior matrix of the average distance between hardware and fixtures, a clustering algorithm is used to obtain structural scene sub-regions with dense hardware distribution. These sub-regions may include, for example: Figure 4 The steps are shown.

[0117] Step 401: Determine the neighborhood density threshold for the clustering algorithm.

[0118] Optionally, in this embodiment, considering the clustering characteristics of hardware images in the aerial image dataset, a hardware combination consisting of multiple hardwares (greater than or equal to 3) is defined as a structural scene. Therefore, the neighborhood density threshold of the DBSCAN clustering algorithm is determined to be 2, i.e., Minpts = 2.

[0119] Step 402: Determine the neighborhood radius of the clustering algorithm based on the coarse detection classification results, the size information of the images in the aerial image dataset, and the prior matrix of the average relative distance of the hardware.

[0120] Optionally, in this step, based on the coarse detection classification results, the image size information in the aerial image dataset, and the prior matrix of the average relative distance of hardware, the calculation method for the neighborhood radius of the DBSCAN clustering algorithm is as follows: Combine the hardware categories in the coarse detection classification results to obtain all hardware combination cases; find the prior matrix of the average relative distance of hardware to obtain the average relative distance of all hardware combination cases; obtain the average value of the average relative distance of all hardware combination cases as the proportion Q, and use the product of the proportion Q and the hypotenuse of the image in the aerial image dataset as the neighborhood radius R of the DBSCAN clustering algorithm.

[0121] Optionally, the calculation method described above in this step embodiment can be expressed as:

[0122]

[0123]

[0124] In this embodiment, the above calculation method fully considers the size information of the images in the aerial image dataset, making the obtained neighborhood radius more consistent with the detected image. By calculating the average relative distance for all hardware combinations, it avoids both situations where the neighborhood radius R is too small, resulting in incomplete scenes or too many fragmented regions in the clustering algorithm, and situations where the neighborhood radius R is too large, resulting in hardware structure scenes with excessively large average distances in the clustering algorithm. Furthermore, the above calculation method also fully considers the coarse detection classification results and combines them with the hardware average relative distance prior matrix, which can better adaptively determine the neighborhood radius R.

[0125] Step 403: Determine the sample set for the clustering algorithm based on the coarse detection results.

[0126] Optionally, this step may include obtaining the center point of the coarse detection prediction box on the hardware image mapped to the aerial image dataset.

[0127] Where N represents the number of hardware targets coarsely detected in the hardware images of the aerial image dataset. This represents the coordinates of the center point of the k-th coarse detection prediction box.

[0128] Step 404: Based on the neighborhood density threshold, neighborhood radius, and sample set of the clustering algorithm, and combined with the prior matrix of the average relative distance of the hardware, obtain the structural scene sub-regions where the hardware is densely distributed and requires precise detection.

[0129] Optionally, in this embodiment, based on the neighborhood density threshold, neighborhood radius, and sample set of the clustering algorithm, combined with the prior matrix of the average relative distance of the hardware, the calculation method for obtaining the structural scene sub-regions with dense hardware distribution that need to be precisely detected is as follows: determine whether a point in the sample set is a core point; after determining a certain point as a core point, find the largest set of all density-connected points starting from that point to form a cluster, then a cluster represents a structural scene sub-region; obtain the corresponding structural scene sub-region by selecting the minimum and maximum horizontal and vertical coordinates of the target coarse detector prediction box in the cluster, thereby realizing the selection function of structural scene sub-regions with dense hardware distribution.

[0130] Optionally, the calculation method described above in this embodiment can be expressed as follows:

[0131]

[0132]

[0133] Optionally, determining whether a point in the sample set is a core point in this step may include: in the sample set, if p i If the number of sample points contained in a neighborhood with a point as its center and a neighborhood radius as its radius is not less than the neighborhood density threshold, then p i The point is the core point.

[0134] Optionally, this step of finding the largest set of all density-connected points starting from this point to form a cluster may include: determining whether the current point is density-connected; in the sample set, if both point o and point q are density-connected to p... i If a point o and a neighborhood density threshold are reachable with respect to both the neighborhood radius and the neighborhood density threshold, then point o and point q are said to be density-connected.

[0135] Regarding the concept of density reachability, it is necessary to first clarify the concept of direct density reachability: in a sample set, p i There exists a point q in the neighborhood of point p, and p i If point q is the central point, then point q is said to originate from point p. i If a point is directly density-reachable from another point, then density-reachability can be defined as: in the sample set, there exists an object chain p1, p2, ..., p n If point p i+1 From p i A point is called a point p if it is directly reachable by density from a given point. i+1 From p i The density starting from point A can reach point B.

[0136] Step 105: Perform fine detection on the sub-regions of the structural scene to obtain fine detection results.

[0137] Optionally, in step 105, a fine-grained detection is performed on the sub-region of the structural scene to obtain the fine-grained detection results, which may include, for example: Figure 5 The steps are shown.

[0138] Step 501: Cropping and downsampling the hardware images in the aerial image dataset to obtain the precise detection region.

[0139] Optionally, in this embodiment, the hardware image with a resolution of 5760×3840 is cropped to a resolution of 1502×1071, and then downsampled to obtain a fine detection area with a resolution of 900×600. Specifically, the hardware target in the original hardware image with a resolution of 182×182 becomes 109×102 after cropping and downsampling.

[0140] As a comparative example, if a metal fitting image with a resolution of 5760×3840 is directly downsampled, for a metal fitting target with a resolution of 182×182, the resolution of the metal fitting target becomes 28×28 after direct downsampling, losing a lot of the information contained in the original image.

[0141] Step 502: Extract features from the fine detection region to obtain the basic features of the fine detection region.

[0142] Optional, this step can be based on Obtain the basic features of the precise detection region;

[0143] in, Indicates from A j The basic features of the fine detection region extracted from A j This represents the finely detected region obtained by cropping and downsampling the hardware images in the aerial image dataset, and j represents the number of sub-regions of the structured scene.

[0144] Step 503: Based on the basic features of the fine detection region, calculate the corresponding fine detection region proposal box through the region proposal network.

[0145] Optional, this step can be based on Obtain the suggested bounding box for the precise detection area;

[0146] in, Indicates from Extracted precise detection region proposal, θ RPN2 This represents the trainable parameters in the fine detection region proposal network, ∏ RPN2 (·|θ RPN2 ) represents the function for extracting precise detection region proposals. Step 504: Within the structured scene sub-region, based on the precise detection region proposals, the corresponding precise detection bounding box regression results and precise detection classification results are calculated using the precise detector.

[0147] Optionally, this step also includes obtaining suggested features for the coarse detection region based on the basic features of the fine detection region and the suggested bounding box of the coarse detection region;

[0148] Optional, this step can be based on The precise detection bounding box regression results and precise detection classification results are obtained in the structural scene sub-region;

[0149] in, This indicates the regression results of the precise detected bounding boxes. This indicates the results of the precise detection and classification. θ represents the suggested features for the coarse detection region. Detector2 ∏ represents the learnable parameters in the precision detector. Detector2 (·|θ Detector2 ) is the processing function for the precision detector.

[0150] Step 505: Process the regression results of the fine detection bounding boxes and the classification results of the fine detection to obtain the fine detection results.

[0151] Optionally, this step also includes summarizing the regression results and classification results of the fine detection bounding boxes of j structural scene sub-regions, transforming and projecting the regression results of the fine detection bounding boxes, removing redundant boxes using NMS based on the classification results of the fine detection, and then mapping the fine detection prediction boxes onto the hardware images in the aerial image dataset to obtain the fine detection results.

[0152] Step 106: Combine the coarse inspection results with the fine inspection results to obtain the final multi-fitting inspection results.

[0153] Optionally, in this step of the embodiment, the fine inspection results and the coarse inspection results can be combined by continuously using non-maximum values ​​to obtain the final multi-fitting inspection results.

[0154] This invention proposes a method for detecting multiple hardware fittings in power transmission lines. Based on an aerial image dataset, a prior matrix of average relative distances between different types of hardware fittings is constructed. Coarse detection is performed on the hardware fitting images, and the results are further combined with the prior matrix of average relative distances to improve the adaptive capability of the clustering algorithm. This clusters densely distributed structural scene sub-regions of hardware fittings, allowing for fine detection of the hardware structures within these sub-regions. Finally, the coarse and fine detection results are combined to obtain the final multi-hardware fitting detection result. Compared to existing technologies, this invention can effectively utilize the information contained in high-resolution hardware fitting images and, by constructing prior matrices of average relative distances for different types of hardware fittings, detect multiple hardware fittings and their aggregate structures. This effectively solves the problems of small targets and dense occlusion in aerial images of hardware fittings, improving the accuracy of target detection in multi-hardware fitting images of power transmission lines.

[0155] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0156] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0157] Figure 6 The diagram shows a structural schematic of a device for detecting multiple fittings in power transmission lines according to an embodiment of the present invention. For ease of explanation, only the parts relevant to the embodiment of the present invention are shown, and are described in detail below:

[0158] like Figure 6 As shown, the transmission line multi-fitting inspection device 6 may include: an acquisition module 601, a data processing module 602, a coarse inspection module 603, a structural scene sub-region selection module 604, a fine inspection module 605, and a combination module 606.

[0159] Acquisition module 601 is used to acquire aerial image datasets;

[0160] Data processing module 602 is used to construct a prior matrix of average relative distance of hardware based on aerial image dataset;

[0161] The coarse detection module is used to perform coarse detection on the hardware images in the aerial image dataset and obtain coarse detection result 603.

[0162] The structural scene sub-region selection module 604 is used to obtain structural scene sub-regions with dense hardware distribution based on coarse detection results and hardware average distance prior matrix through clustering algorithm;

[0163] The fine inspection module 605 is used to perform fine inspection on the sub-regions of the structural scene and obtain fine inspection results;

[0164] The combination module 606 is used to combine the coarse inspection results with the fine inspection results to obtain the final multi-fitting inspection results.

[0165] Optionally, when the data processing module 602 constructs the prior matrix of the average relative distance of the hardware based on the aerial image dataset, it can be used for:

[0166] Based on the size information of the images in the aerial image dataset and the coordinates of the true center point, calculate the relative distance between the i-th s-type hardware and the j-th t-type hardware in any image in the aerial image dataset;

[0167] Based on the above method of calculating the relative distance between the i-th type s-fitting device and the j-th type t-fitting device in any image of the aerial image dataset, the relative distance between the s-th type s-fitting device and the t-th type t-fitting device in all images of the aerial image dataset is determined.

[0168] Based on the relative distances between the s-th type of hardware and the t-th type of hardware in all images in the aerial image dataset, calculate the average relative distance between the s-th type of hardware and the t-th type of hardware.

[0169] Based on the above method of calculating the average relative distance between type s hardware and type t hardware, calculate the average relative distance between all pairs of hardware of all types in the hardware category set T, and construct the hardware average relative distance prior matrix.

[0170] Optionally, the coarse detection module 603 performs coarse detection on the hardware images in the aerial image dataset. When obtaining the coarse detection result, it can be used for:

[0171] Downsampling is performed on the hardware images in the aerial image dataset to obtain coarse detection regions;

[0172] Feature extraction is performed on the coarse detection region to obtain its basic features;

[0173] Based on the basic features of the coarse detection region, the corresponding coarse detection region proposal box is calculated through the region proposal network;

[0174] Based on the proposed bounding boxes of the coarse detection regions, the corresponding coarse detection bounding box regression results and coarse detection classification results are calculated by the coarse detector.

[0175] The coarse detection bounding box regression results and coarse detection classification results are processed to obtain the coarse detection results.

[0176] Optionally, when extracting features from the coarse detection region in the coarse detection module 603 to obtain the basic features of the coarse detection region, it can be used for:

[0177] According to F coarse =∏ CNN (X i |θ resnet101 The basic features of the coarse detection region are obtained;

[0178] Among them, F coarse Indicates from X i The basic features of the coarse detection region extracted from X i ∏ represents the coarse detection region obtained by downsampling the hardware image in the i-th aerial image dataset. CNN (·|θ resnet101 ) represents the feature extraction function, θ resnet101This represents the trainable parameters of the backbone network ResNet101 during the feature extraction process.

[0179] Optionally, when the coarse detection module 603 calculates the corresponding coarse detection region proposal box based on the basic features of the coarse detection region through the region proposal network, it can be used for:

[0180] According to R coarse =∏ RPN1 (F coarse |θ RPN1 This yields a coarse detection region suggestion box;

[0181] Among them, R coarse Indicates from F coarse Extracted coarse detection region proposal, θ RPN1 This represents the trainable parameters in the coarse detection region proposal network, ∏ RPN1 (·|θ RPN1 ) represents the function for extracting suggestion boxes for coarse detection regions.

[0182] Optionally, when the coarse detection module 603 calculates the corresponding coarse detection bounding box regression result and coarse detection classification result based on the coarse detection region proposal box through the coarse detector, it can be used for:

[0183] Based on the basic features of the coarse detection region and the proposed bounding box of the coarse detection region, the proposed features of the coarse detection region are obtained;

[0184] According to (B) coarse C coarse )=∏ Detector1 (P coarse |θ Detector1 The coarse detection bounding box regression results and coarse detection classification results are obtained.

[0185] Among them, B coarse C represents the coarse detection bounding box regression result. coarse P represents the coarse detection and classification result. coarse θ represents the suggested features for the coarse detection region. Detector1 ∏ represents the learnable parameters in the coarse detector. Detector1 (·|θ Detector1 ) is the coarse detector processing function.

[0186] Optionally, when the structural scene sub-region selection module 604 obtains structural scene sub-regions with dense hardware distribution through a clustering algorithm based on the coarse detection results and the hardware average distance prior matrix, it can be used for:

[0187] Determine the neighborhood density threshold for the clustering algorithm;

[0188] Based on the coarse detection classification results, the size information of the images in the aerial image dataset, and the prior matrix of the average relative distance of the hardware, the neighborhood radius of the clustering algorithm is determined.

[0189] Based on the coarse detection results, determine the sample set for the clustering algorithm;

[0190] Based on the neighborhood density threshold, neighborhood radius, and sample set of the clustering algorithm, combined with the prior matrix of the average relative distance of the hardware, the structural scene sub-regions with densely distributed hardware that need to be precisely detected are obtained.

[0191] Optionally, the fine detection module 605 performs fine detection on the structural scene sub-region, and when obtaining the fine detection result, it can be used for:

[0192] The hardware images in the aerial image dataset are cropped and then downsampled to obtain the precise detection region;

[0193] Feature extraction is performed on the fine detection region to obtain the basic features of the fine detection region;

[0194] Based on the basic features of the fine detection region, the corresponding fine detection region proposal box is calculated through the region proposal network;

[0195] Within the structural scene sub-region, based on the proposed boxes of the fine detection region, the corresponding fine detection bounding box regression results and fine detection classification results are calculated by the fine detector.

[0196] The regression results and classification results of the fine detection bounding boxes are processed to obtain the fine detection results.

[0197] Optionally, when performing feature extraction on the fine detection region in the fine detection module 605 to obtain the basic features of the fine detection region, it can be used for:

[0198] according to Obtain the basic features of the precise detection region;

[0199] in, Indicates from A j The basic features of the fine detection region extracted from A j This represents the fine detection region obtained by cropping and downsampling the hardware images in the aerial image dataset, and j represents the number of structural scene sub-regions.

[0200] Optionally, when the fine detection module 605 calculates the corresponding fine detection region proposal box based on the basic features of the fine detection region through the region proposal network, it can be used for:

[0201] according to Obtain the suggested bounding box for the precise detection area;

[0202] in, Indicates from Extracted precise detection region proposal, θ RPN2 This represents the trainable parameters in the fine detection region proposal network, ∏ RPN2 (·|θ RPN2 ) represents the function for extracting suggestion boxes for the fine detection region.

[0203] Optionally, in the fine detection module 606, when calculating the corresponding fine detection bounding box regression results and fine detection classification results based on the fine detection region proposal boxes within a structural scene sub-region, it can be used for:

[0204] Based on the basic features of the fine detection region and the suggested bounding box of the fine detection region, the suggested features of the fine detection region are obtained;

[0205] Within the structural scene sub-region, according to We obtain the regression results of the precise detection bounding boxes and the classification results of the precise detection.

[0206] in, This indicates the regression results of the precise detected bounding boxes. This indicates the results of the precise detection and classification. θ represents the suggested features for the precise detection region. Detector2 ∏ represents the learnable parameters in the precision detector. Detector2 (·|θ Detector2 ) is the processing function for the precision detector.

[0207] The aforementioned multi-fitting detection device for power transmission lines acquires aerial image datasets of fittings through an acquisition module, processes them through a data processing module to obtain a prior matrix of average relative distances between fittings, and obtains the average relative distances between different types of fittings. This provides a reference for the adaptive neighborhood radius selection of the clustering algorithm in the structural scene sub-region selection module. After the coarse detection module performs coarse detection on the aerial images of fittings to obtain coarse detection results, it combines the neighborhood radius selection obtained by the clustering algorithm to cluster structural scene sub-regions with densely distributed fittings through the structural scene sub-region selection module. Then, the fine detection module performs fine detection on the selected structural scene sub-regions to obtain fine detection results. Finally, the combination module combines the coarse detection results and the fine detection results to achieve the detection of multiple fittings on power transmission lines. This solves the problems of small targets and dense occlusion in multi-fitting detection, and improves the efficiency and accuracy of multi-fitting detection.

[0208] Compared to existing methods for detecting hardware based on deep learning networks, the multi-hardware detection method for transmission lines provided in this embodiment combines prior knowledge of the average relative distance between different types of hardware, providing a reference for better judging close contact and obstruction between hardware. By combining the coarse detection results with the prior matrix of the average relative distance of hardware, a clustering algorithm is used to cluster the structural scene sub-regions with dense hardware distribution for further fine detection, thereby improving the efficiency and accuracy of multi-hardware detection.

[0209] Figure 7 This is a schematic diagram of a terminal provided in an embodiment of the present invention. For example... Figure 7 As shown, the terminal device 700 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701, such as a transmission line multi-fitting detection program. When the processor 701 executes the computer program 703, it implements the steps in the above-described transmission line multi-fitting detection method embodiment, for example... Figure 1 Steps 101 to 106 shown, or Figure 2 , Figure 3 , Figure 4 as well as Figure 5 The steps shown indicate that when the processor 701 executes the computer program 703, it implements the functions of each module in the above-described device embodiments, for example... Figure 6 The functions of modules 601 to 606 are shown.

[0210] For example, the computer program 703 can be divided into one or more modules / units, which are stored in the memory 702 and executed by the processor 701 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 703 in the terminal device 700. For example, the computer program 703 can be divided into... Figure 6 Modules 601 to 606 are shown.

[0211] The terminal device 700 may include, but is not limited to, a processor 701 and a memory 702. Those skilled in the art will understand that... Figure 7 This is merely an example of terminal device 700 and does not constitute a limitation on terminal device 700. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0212] The processor 701 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0213] The memory 702 can be an internal storage unit of the terminal device 700, such as a hard disk or memory of the terminal device 700. The memory 702 can also be an external storage device of the terminal device 700, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 700. Furthermore, the memory 702 can include both internal and external storage units of the terminal device 700. The memory 702 is used to store the computer program and other programs and data required by the terminal. The memory 702 can also be used to temporarily store data that has been output or will be output.

[0214] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0215] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0216] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0217] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0218] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0219] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0220] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above embodiments of the multi-fitting detection method for transmission lines.

[0221] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0222] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for testing multiple fittings in power transmission lines, characterized in that, include: S1. Obtain the aerial image dataset; S2. Based on the aerial image dataset, construct the prior matrix of the average relative distance of the hardware. S3. Perform coarse detection on the hardware images in the aerial image dataset to obtain coarse detection results; S4. Based on the coarse detection results and the prior matrix of the average distance between hardware, obtain the structural scene sub-regions with dense hardware distribution through clustering algorithm; S5. Perform fine detection on the sub-region of the structure scene to obtain fine detection results; S6. Combine the coarse inspection results with the fine inspection results to obtain the final multi-fitting inspection results; Wherein, S3 includes: S31, downsample the hardware images in the aerial image dataset to obtain a coarse detection region; S32, Perform feature extraction on the coarse detection region to obtain the basic features of the coarse detection region; S33, Based on the basic features of the coarse detection region, the corresponding coarse detection region proposal box is calculated through the region proposal network; S34, Based on the proposed coarse detection region bounding box, the corresponding coarse detection bounding box regression result and coarse detection classification result are calculated by the coarse detector; S35, Process the coarse detection bounding box regression result and the coarse detection classification result to obtain the coarse detection result; Wherein, S32 includes: according to The basic features of the coarse detection region are obtained; in, Indicates from The basic features of the coarse detection region extracted from it. Indicates the first The coarse detection region is obtained by downsampling the hardware images in the Zhang aerial image dataset. Represents the feature extraction function. This represents the trainable parameters of the backbone network ResNet101 during the feature extraction process; S33 includes: according to Obtain a suggested bounding box for the coarse detection region; in, Indicates from Extracted coarse detection region suggestion boxes, This indicates the trainable parameters in the coarse detection region suggestion network. This represents the function for extracting suggestion boxes for the coarse detection region.

2. The method for testing multiple fittings in transmission lines according to claim 1, characterized in that, The step of constructing the average relative distance prior matrix for hardware based on the aerial image dataset includes: Based on the aerial image dataset, obtain the hardware category set T and the size information of the images in the aerial image dataset; Arbitrarily select the first from the set of hardware categories T Type of hardware and the first Metal fittings, determining the first element in any image of the aerial image dataset. Type of hardware and the first The quantity of each type of hardware and the corresponding coordinates of its actual center point; Based on the size information of the images in the aerial image dataset and the coordinates of the true center point, calculate the i-th value in any image of the aerial image dataset. indivual Type of hardware and the first indivual The relative distance between similar hardware components; Based on the above calculation, the first image in any image of the aerial image dataset... indivual Type of hardware and the first indivual The method of determining the relative distance between similar hardware components is used to identify the first image in the aerial image dataset. Type of hardware and the first The relative distance between hardware-like fittings; According to the aerial image dataset, the first image among all images Type of hardware and the first The relative distance of the hardware-like fixtures is used to calculate the first... Type of hardware and the first The average relative distance between similar hardware components; Based on the above calculation, the... Type of hardware and the first The method of calculating the average relative distance between hardware categories is to calculate the average relative distance between any two hardware categories in the hardware category set T, and then construct the hardware average relative distance prior matrix.

3. The method for testing multiple fittings in transmission lines according to claim 1, characterized in that, S4 includes: Determine the neighborhood density threshold of the clustering algorithm; Based on the coarse detection classification results, the size information of the images in the aerial image dataset, and the prior matrix of the average relative distance of the hardware, the neighborhood radius of the clustering algorithm is determined. Based on the coarse detection results, the sample set for the clustering algorithm is determined; Based on the neighborhood density threshold, neighborhood radius, and sample set of the clustering algorithm, combined with the prior matrix of the average relative distance of the hardware, the structural scene sub-regions with densely distributed hardware that need to be precisely detected are obtained.

4. The method for testing multiple fittings in transmission lines according to claim 3, characterized in that, S5 includes: S51. Crop and downsample the hardware images in the aerial image dataset to obtain a precise detection region; S52. Extract features from the fine detection region to obtain the basic features of the fine detection region; S53. Based on the basic features of the fine detection region, calculate the corresponding fine detection region proposal box through the region proposal network; S54. Under the structure scene sub-region, based on the fine detection region proposal box, the corresponding fine detection bounding box regression result and fine detection classification result are calculated by the fine detector; S55. Process the regression results of the fine detection bounding boxes and the classification results of the fine detection to obtain the fine detection results.

5. The method for testing multiple fittings in transmission lines according to claim 4, characterized in that, S52 includes: according to The basic features of the precision detection region are obtained; in, Indicates from The basic features of the fine detection region extracted from it. This represents the finely detected region obtained by cropping and downsampling the hardware images in the aerial image dataset. Indicates the number of sub-regions in the structure scene.

6. The method for testing multiple fittings in transmission lines according to claim 5, characterized in that, S53 includes: according to Obtain the suggested bounding box for the precise detection area; in, Indicates from Extracted precise detection region suggestion box, This represents the trainable parameters in the precision detection region suggestion network. This represents the function for extracting suggestion boxes for the precision detection region.

7. The method for testing multiple fittings in transmission lines according to claim 6, characterized in that, S34 includes: Based on the basic features of the coarse detection region and the proposed bounding box of the coarse detection region, the proposed features of the coarse detection region are obtained; according to The coarse detection bounding box regression results and coarse detection classification results are obtained; in, This indicates the regression results of the coarse detection bounding boxes. This indicates the results of the coarse detection and classification. This indicates suggested features for the coarse detection region. This represents the learnable parameters in the coarse detector. This is the coarse detector processing function.

8. The method for testing multiple fittings in transmission lines according to claim 4 or 7, characterized in that, In the structural scene sub-region, based on the precise detection region proposal, the corresponding precise detection bounding box regression result and precise detection classification result are calculated by the precise detector, including: Based on the basic features of the fine detection region and the suggestion box of the fine detection region, the suggested features of the fine detection region are obtained; Under the aforementioned structural scene sub-region, according to We obtain the regression results of the precise detection bounding boxes and the classification results of the precise detection. in, This indicates the regression results of the precise detected bounding boxes. This indicates the results of the precise detection and classification. Indicates the suggested features for the precise detection region. This represents the learnable parameters in the precision detector. This is the processing function for the precision detector.

9. A transmission line multi-fitting inspection device for performing the transmission line multi-fitting inspection method according to any one of claims 1 to 5, characterized in that, include: The acquisition module is used to acquire aerial image datasets; The data processing module is used to construct a prior matrix of the average relative distance of hardware based on the aerial image dataset; The coarse detection module is used to perform coarse detection on the hardware images in the aerial image dataset to obtain coarse detection results. The structural scene sub-region selection module is used to obtain structural scene sub-regions with dense hardware distributions by using a clustering algorithm based on the coarse detection results and the hardware average distance prior matrix. The fine detection module is used to perform fine detection on the sub-regions of the structure scene and obtain fine detection results; The combination module is used to combine the coarse inspection results with the fine inspection results to obtain the final multi-fitting inspection results.

Images