A power line fault visual detection method and system based on unmanned aerial vehicle inspection

Abstract

The application discloses a kind of power line fault visual detection method and system based on unmanned aerial vehicle inspection, it is related to electric power line intelligent inspection and computer vision field.The method comprises the following steps: collecting inspection image and environment information, calculating environment disturbance index and self-adaptive threshold updating, soft threshold and joint denoising pretreatment are carried out on the image;The pretreated image is input into a single-stage detection model to obtain a component candidate box, and a core component image is obtained by fine classification through a deep confidence network;The two-stage detection model is used to locate the fault area, the local feature is extracted, and the core fault feature is obtained by singular value decomposition;According to the component category and the environment index, select the template sub-library, determine the danger level by overall and local matching, and output the fault result, mark the abnormal time for review.The application improves the detection accuracy and environmental adaptability of components such as conductors and insulators through environment adaptive processing, component fine classification, low-rank feature extraction and hierarchical matching strategy.

Description

Technical Field

[0001] This invention relates to the fields of intelligent power line inspection, computer vision, and fault diagnosis technology, specifically to a visual detection method and system for power line faults based on unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] Traditional manual inspection methods suffer from high labor intensity, long inspection cycles, high safety risks, and low detection efficiency, making them unsuitable for the safe operation requirements of modern power grids. In recent years, drone inspection technology, with its advantages of mobility, wide coverage, and relatively low cost, has become an important means of acquiring image information of transmission lines, providing a new technical approach for power line fault detection.

[0003] Existing visual fault detection technologies for power lines mainly include manual review, monitoring with fixed video cameras, image processing methods based on template matching, and automatic detection methods based on deep learning. While manual review can accurately identify faults, it relies on the professional experience and subjective judgment of inspection personnel, resulting in a heavy workload and susceptibility to fatigue, making it unsuitable for large-scale inspections. Monitoring with fixed video cameras is limited by installation location and observation angle, resulting in limited coverage and making it difficult to effectively monitor the entire line. Template-matching-based image processing methods establish a standard fault template library and compare the similarity between inspection images and templates to determine the condition of components. While simple to implement, this method has significant limitations: First, different voltage levels, installation methods, and shooting angles can lead to significant differences in the appearance of similar components, making it difficult for fixed templates to cover all forms and easily confusing similar-looking components such as wires, clamps, and connectors. Second, high-definition inspection images generate massive amounts of data, requiring high computational complexity for pixel-by-pixel and template-by-template comparisons, making it difficult to meet real-time detection requirements. Third, complex environmental factors such as strong light, backlight, rain, fog, motion blur, and vegetation obstruction significantly reduce image quality, easily leading to missed detection of small-scale defects such as broken strands, insulator cracks, and localized joint anomalies. Fourth, existing systems lack clear verification and knowledge base update mechanisms for unknown faults and low-confidence results, making it difficult to balance online detection stability with subsequent iterative optimization capabilities.

[0004] In recent years, deep learning technology has been widely used in the field of power line fault detection. Detection methods based on deep learning models such as Convolutional Neural Networks (CNN), YOLO, and SSD have achieved good results in component identification and fault classification, but some shortcomings still exist: On the one hand, most existing deep learning methods adopt end-to-end classification output and lack a confidence leveling mechanism, making it difficult to achieve a balance between high detection accuracy and low false alarm rate; on the other hand, deep learning models usually require a large number of labeled samples for training, while power line fault samples are relatively scarce and the fault types are diverse, making it difficult to obtain sufficient training data; in addition, existing deep learning methods mainly focus on broad category identification, with insufficient research on fine classification of components such as conductors, insulators, clamps, joints, and tower connectors, which can easily lead to mismatches between components with similar appearances; at the same time, existing methods have limited adaptability to complex environments, mainly relying on data augmentation or fixed preprocessing procedures to cope with environmental changes, lacking a mechanism to dynamically adjust processing strategies according to real-time environmental conditions.

[0005] From a technological development perspective, UAV power line inspection technology is evolving from single-vision inspection to multi-modal inspection (visible light + infrared + ultraviolet), from manual review to automated and intelligent inspection, from simple defect identification to complex fault diagnosis, and from offline analysis to real-time online inspection. The application of deep learning in insulator inspection also shows trends of development from traditional two-stage inspection to single-stage inspection, from single-task inspection to multi-task inspection, from high-precision models to lightweight models, and from single-component inspection to multi-component integrated inspection. However, existing technologies still struggle to simultaneously address multiple challenges, including environmental adaptability, component confusion, small defect detection, false alarm control, and continuous knowledge accumulation.

[0006] Therefore, how to design a power line fault visual detection method that can adapt to complex environmental conditions, accurately identify various line components, effectively detect small-scale defects, reasonably control the false alarm rate, and has continuous learning capabilities has become an urgent technical problem to be solved in the field of intelligent power line inspection. Summary of the Invention

[0007] To address the problems of easy confusion in component identification, unclear features of small defects in complex environments, large computational load, and lack of closed-loop verification of low-confidence results in existing technologies, this invention provides a visual detection method and system for power line faults based on UAV inspection.

[0008] To achieve the above objectives, the first aspect of the present invention provides a visual detection method for power line faults based on unmanned aerial vehicle (UAV) inspection, comprising the following steps:

[0009] The process involves acquiring original images of power lines collected by drone inspection and corresponding environmental information, calculating environmental disturbance indices based on the environmental information, updating noise filtering thresholds based on the environmental disturbance indices, and performing denoising on the original images based on the current noise filtering thresholds to obtain preprocessed images.

[0010] The preprocessed image is input into a real-time target detection model to obtain candidate bounding boxes for line components. Based on the candidate bounding boxes, candidate component images are cropped and input into a deep belief network for fine classification to obtain images of core line components and component fine classification categories.

[0011] The image of the core component of the line is input into a two-stage target detection model to locate the fault candidate region. Local features are extracted from the fault candidate region to construct a feature matrix. The feature matrix is ​​then subjected to singular value decomposition and reconstructed to obtain the core fault feature matrix.

[0012] Based on the component sub-classification and the environmental disturbance index, a fault template sub-library is selected. The core fault feature matrix is ​​matched with the fault templates in the fault template sub-library. Based on the matching result, the normal state or fault type, location and confidence level are output.

[0013] Preferably, updating the noise filtering threshold according to the environmental disturbance index includes: updating the current noise filtering threshold according to the environmental disturbance index and the noise filtering threshold at the previous moment.

[0014] Preferably, the denoising process of the original image to obtain the preprocessed image includes: performing soft threshold denoising on the original image based on the current noise filtering threshold, and performing multi-algorithm joint denoising on the soft threshold denoised image to obtain the preprocessed image; and associating and saving the environmental disturbance index with the preprocessed image.

[0015] Preferably, the multi-algorithm joint denoising includes one or more of the following:

[0016] (1) Perform local thresholding and global thresholding based on grayscale, chroma, hue and position information;

[0017] (2) Use fuzzy membership mean clustering to perform soft segmentation of suspected line component regions and suspected noise regions;

[0018] (3) Perform median filtering on the impulse noise region;

[0019] (4) Perform background estimation and brightness compensation on regions with gradual brightness changes;

[0020] The edge preservation index within the candidate path region and the noise variance outside the candidate path region are calculated separately for each processing result. The normalized fusion weight is determined with the edge preservation index as the main factor and the noise variance as the secondary factor to obtain the preprocessed image.

[0021] Preferably, the current noise filtering threshold is calculated by weighting the previous noise filtering threshold, the initial noise filtering threshold, and the environmental disturbance index; a candidate path map is generated based on the linear structure and tower outline in the original image; the line structure filtering template used to characterize the conductor outline, insulator skirt, or clamp edge is convolved with the candidate path map to obtain template coefficients; template coefficients with absolute values ​​less than the current noise filtering threshold are set to zero; and soft thresholding is performed on the remaining template coefficients.

[0022] Preferably, the collected environmental information includes at least two of the following: average image brightness, brightness variance, contrast, image sharpness, and UAV attitude change; each of the collected environmental information items is normalized and then weighted and summed according to a preset environmental weight to obtain the environmental disturbance index.

[0023] Preferably, inputting the preprocessed image into the real-time target detection model to obtain candidate bounding boxes for line components includes: inputting the preprocessed image into a single-stage target detection model to obtain candidate bounding boxes for line components, component categories, and detection confidence scores; the real-time target detection model outputs at least one component category among conductors, insulators, clamps, joints, and tower connectors, and removes candidate bounding boxes with detection confidence scores lower than a preset detection threshold, and performs non-maximum suppression on overlapping candidate bounding boxes.

[0024] Preferably, the deep belief network comprises at least two restricted Boltzmann machines stacked together; the step of inputting the candidate component image into the deep belief network for fine classification includes:

[0025] The normalized candidate component image features are input into the first restricted Boltzmann machine to obtain the first hidden layer state;

[0026] The first hidden layer state is input into the second restricted Boltzmann machine to obtain the second hidden layer state;

[0027] The state of the second hidden layer is input into the classification output layer, and the probability of each component sub-class is calculated using the normalized exponential function. The category with the highest probability is taken as the component sub-class.

[0028] Preferably, the activation probability of the k-th hidden neuron in the L-th layer is calculated by the state of each neuron in the (L-1)-th layer, the corresponding connection weights, and the bias of the k-th hidden neuron in the L-th layer using a logical activation function, and Bernoulli sampling is performed based on the activation probability.

[0029] The activation probability of the k-th hidden neuron in the L-th layer is expressed as:

[0030]

[0031] Bernoulli sampling is performed based on the activation probability to obtain the hidden neuron state:

[0032]

[0033] Where L represents the hidden layer number of the deep belief network; k represents the neuron number of the Lth layer; j represents the neuron number of the previous layer; and m represents the total number of neurons in the previous layer. This represents the probability that the k-th neuron in layer L is activated for the current candidate circuit component. This indicates the feature state of the j-th component in the previous layer. This represents the connection weights learned from the training samples of the candidate circuit components. This indicates the bias of the current neuron. This represents the current neuron state after sampling; σ represents the logistic activation function, whose expression is: ; x represents the weighted input formed by candidate component features, connection weights, and biases, and e represents the natural constant; This represents a uniformly distributed random number generated independently for the current neuron within the interval [0,1].

[0034] Preferably, inputting the image of the core component of the line into a two-stage target detection model to locate the fault candidate region includes: inputting the image of the core component of the line into a two-stage target detection model with a region proposal network and a region classification regression branch; generating candidate regions by the region proposal network; determining the region of interest based on the fault probability and location regression results of the candidate regions; and mapping the region of interest to the fault candidate position in the original image; when there is no candidate region with a fault probability reaching a preset region threshold, the corresponding core component of the line is marked as a normal component.

[0035] Preferably, extracting local features from the candidate fault region to construct a feature matrix includes: extracting at least two of edge features, corner features, texture features, and color features from the candidate fault region, and constructing a suspected fault feature matrix according to the spatial location of the normalized features.

[0036] Preferably, the process of performing singular value decomposition and reconstructing the core fault feature matrix to obtain the core fault feature matrix includes: performing singular value decomposition on the suspected fault feature matrix, selecting the top k singular values ​​and corresponding singular vectors that make the cumulative energy contribution rate reach a preset contribution rate threshold, and reconstructing the core fault feature matrix with rank k.

[0037] Preferably, the cumulative energy contribution rate is the ratio of the sum of squares of the first k singular values ​​to the sum of squares of all singular values, where k is a positive integer that makes the cumulative energy contribution rate reach a preset contribution rate threshold for the first time; the suspected fault feature matrix is ​​reconstructed in low rank by the diagonal matrix formed by the first k columns of the left singular vector matrix, the first k singular values, and the first k rows of the right singular vector matrix, to obtain the core fault feature matrix with rank k.

[0038] Preferably, selecting the fault template sub-library based on the component sub-category and the environmental disturbance index includes: determining the target template sub-library from the fault template library organized according to component category, fault type and applicable environmental range based on the component sub-category and the environmental disturbance index.

[0039] The feature matching of the core fault feature matrix with the fault templates in the fault template sub-library includes: performing overall feature matching and local feature matching on the core fault feature matrix and the fault templates in the target template sub-library in sequence to obtain a local matching score;

[0040] Outputting the normal state or fault type, location, and confidence level based on the matching results includes: classifying and judging the local matching score according to the matching threshold corresponding to the fault hazard level marked by the candidate template, outputting the normal state, or outputting the fault type, fault candidate location, and result confidence level, or marking the detection result as an unknown fault or pending manual review.

[0041] Preferably, the step of sequentially performing overall feature matching and local feature matching between the core fault feature matrix and the fault templates in the target template sub-library includes: in the target template sub-library corresponding to the component sub-classification category and the environmental interval where the environmental disturbance index is located, vectorizing the core fault feature matrix and the overall feature matrix of each fault template in the same order, calculating the cosine similarity between the vectorization results, and screening candidate templates whose overall similarity reaches the overall matching threshold; calculating the similarity between the core fault features and each candidate template on at least two local features in edge, corner, texture and color respectively; setting feature weights for each local feature similarity according to the fault type corresponding to the candidate template and performing weighted summation to obtain the local matching score.

[0042] Preferably, when there are no candidate regions whose failure probability reaches a preset region threshold, a normal state is output;

[0043] When there is a candidate region with a fault probability that reaches the preset regional threshold, but there is no candidate template that reaches the overall matching threshold, or the local matching score is lower than the corresponding initial screening threshold, the detection result will be marked as an unknown fault.

[0044] When the local matching score reaches the initial screening threshold but is lower than the confirmation threshold, the detection result is marked as pending manual review;

[0045] When the local matching score reaches the confirmation threshold, the matched fault type is output.

[0046] Preferably, the local matching scores of each fault type are calibrated to the same value range by an independent validation set; the initial screening threshold for high-risk faults is lower than that for low-risk faults; for detection results that reach the initial screening threshold for high-risk faults but do not reach the confirmation threshold, an early warning is output and marked as pending manual review.

[0047] Preferably, the test results to be manually reviewed are reviewed offline, and the reviewed and confirmed samples are added to the candidate template library; regression testing is performed on the candidate template library, and a new version of the fault template library with version identification is formed after passing the regression test; the online detection terminal only loads the fault template library that has passed the regression test in read-only mode, and retains the previous version to support rollback.

[0048] A second aspect of the present invention provides a visual detection system for power line faults based on unmanned aerial vehicle (UAV) inspection, comprising:

[0049] The image acquisition and preprocessing module is used to acquire the original images of power lines and the acquisition environment information collected by the UAV inspection, and update the noise filtering threshold according to the acquisition environment information to obtain the preprocessed image.

[0050] The component detection and fine classification module is used to obtain candidate boxes of circuit components, component categories, and fine classification categories.

[0051] The fault region localization module is used to determine candidate fault regions and map them to the original image through a two-stage target detection model with a region proposal network and a region classification regression branch; the fault feature extraction module is used to construct a suspected fault feature matrix and obtain the low-rank reconstructed core fault feature matrix based on the singular value cumulative energy contribution rate.

[0052] The fault determination module is used to select the fault template sub-library based on the sub-category and environmental disturbance indicators, and perform hierarchical determination based on overall matching, local matching and fault hazard level, and output the normal status, confirmed fault, unknown fault or result pending manual review; the template library management module is used to perform regression testing and version release on manually reviewed samples in an offline environment.

[0053] A third aspect of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the aforementioned method.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method.

[0055] Compared with the prior art, the present invention has the following beneficial technical effects:

[0056] (1) This invention achieves intelligent adjustment of image quality under complex environmental conditions by dynamically calculating environmental disturbance indicators and adaptively updating noise filtering thresholds. The environmental disturbance indicators are not only used to update noise filtering thresholds, but also to screen fault templates that match the applicable environmental range, so that image preprocessing, template retrieval and alarm strategies form a processing link with clear data transmission relationship, which significantly improves the system's adaptability under different lighting, weather and shooting conditions.

[0057] (2) This invention adopts a hierarchical detection architecture, uses a real-time target detection model to complete component localization and major category screening, and then uses a deep belief network to perform fine classification of candidate components. The fine classification results are used to output component categories and to select the corresponding fault template sub-library, thereby reducing cross-component template comparisons and reducing mismatches between components with similar appearances, effectively solving the problem of easy confusion between components with similar appearances such as wires, clamps and connectors.

[0058] (3) This invention dynamically adjusts the noise filtering threshold based on environmental disturbance indicators and combines multiple algorithms such as soft threshold shrinkage, fuzzy membership mean clustering, median filtering, and brightness compensation to reduce the impact of complex lighting and impulse noise while retaining the edge information of minor faults such as cracks, broken strands, and wear. The low-rank reconstruction order is determined by the singular value cumulative energy contribution rate, which suppresses low-energy noise components while retaining the main feature energy, significantly improving the detection accuracy of small-scale defects.

[0059] (4) The present invention uses overall matching to screen candidate templates, uses weighted matching of multiple local features to confirm faults, and sets initial screening thresholds and confirmation thresholds for different risk levels. It can give priority to warnings for high-risk level faults, and transfer results with insufficient evidence to manual review or mark them as unknown faults, thus avoiding false alarms caused by forced classification and achieving a good balance between detection accuracy and false alarm rate.

[0060] (5) This invention separates template library updates from online detection. Manual review and regression testing are completed in an offline environment. The online terminal only loads template libraries that have passed verification and have version identifiers, and retains rollback versions. This design effectively reduces the risk of unverified templates affecting the stability of online detection while continuously expanding fault knowledge, providing a reliable mechanism for continuous system optimization and knowledge accumulation.

[0061] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the overall method of the present invention;

[0063] Figure 2 This is a comparison image of the original image of the strong light scene and the adaptive noise reduction result of the present invention;

[0064] Figure 3 This is a comparison image of the original image of the backlit scene and the adaptive noise reduction result of the present invention;

[0065] Figure 4 This is a schematic diagram of the component classification structure of the deep belief network formed by stacking two restricted Boltzmann machines in this invention;

[0066] Figure 5 This is a schematic diagram of the candidate regions and their region confidence levels for the linear component of the present invention;

[0067] Figure 6 This is a schematic diagram of the suspected abnormal region and its confidence level of the linear component of the present invention;

[0068] Figure 7 This is a schematic diagram showing the relative detection performance trends of the present invention compared to statistical methods and template matching methods. Detailed Implementation

[0069] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0070] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0071] like Figure 1 As shown, this embodiment provides a visual detection method for power line faults based on UAV inspection, including the following steps:

[0072] Step S1: Obtain the original image of the power line collected by the UAV inspection and the collection environment information corresponding to the original image. Calculate the environmental disturbance index based on the collection environment information and update the noise filtering threshold based on the environmental disturbance index. Perform noise reduction processing on the original image based on the current noise filtering threshold to obtain a preprocessed image.

[0073] UAV inspection images are easily affected by complex backgrounds, lighting changes, rain and fog, motion blur, conductor reflections, and changes in flight attitude. Step S1 is not only used to reduce background and noise, but also to adjust the denoising intensity according to the collected environmental information, so that the preprocessing results can be adapted to different inspection conditions, and to preserve the edge information of small-scale faults such as cracks, broken strands, and wear as much as possible.

[0074] This embodiment reads the original image and its acquisition time, and obtains at least two of the following as acquisition environment information: average image brightness, brightness variance, contrast, image sharpness, and UAV attitude change. Each environmental quantity is normalized to [0,1] according to the upper and lower limits determined by training data or historical inspection data. Let the q-th normalized environmental quantity be... The corresponding weight is The total environmental quantity is Q, with each weight being non-negative and the sum of the weights being 1; environmental disturbance index The formula for calculation is:

[0075]

[0076] Where t represents the acquisition time or frame number of the current inspection image; q represents the sequence number of the acquired environmental quantity; Q represents the total number of environmental quantities involved in the calculation; and normalized environmental quantities... This corresponds to one of the following: average brightness, brightness variance, contrast, image sharpness, or drone attitude change in the current image. This indicates the weight of the environmental quantity in relation to the degree of image disturbance. This represents the environmental disturbance index of the current image. Each environmental weight is determined by historical inspection data or an independent validation set, and their sum is 1. When the noise level increases, the noise filtering threshold is increased to enhance noise suppression; when When reducing noise, lower the noise filtering threshold to minimize loss of conductor profile, insulator skirts, clamp edges, and minor fault features.

[0077] In this embodiment, updating the noise filtering threshold based on the environmental disturbance index includes updating the current noise filtering threshold based on the environmental disturbance index and the noise filtering threshold at the previous moment.

[0078] The noise filtering threshold is updated according to the following formula:

[0079]

[0080] in, This indicates the noise filtering threshold used to shrink the line structure template coefficients in the current inspection image. This indicates the noise filtering threshold used at the previous inspection time. This represents the preset initial noise filtering threshold under standard imaging conditions. Indicates environmental disturbance indicators; This represents the threshold smoothing coefficient between adjacent inspection times. This indicates the proportion of the threshold update caused by the current environmental conditions. This represents the adjustment coefficient of the environmental disturbance index to the threshold increment. , and The noise in the preprocessed image is suppressed and the target edge retention rate of the conductor, insulator and clamp is not lower than the preset requirement, as determined by an independent validation set.

[0081] Line structure detection and coarse tower contour localization are performed on the original image to generate a candidate path map representing the possible distribution areas of line components. The candidate path map retains the original pixels within the candidate areas and sets the pixels outside the candidate areas to zero. The line component template is convolved with the candidate path map to obtain template coefficients. For absolute values ​​less than... The template coefficients are set to zero, and the remaining template coefficients are subjected to soft thresholding to suppress small noise and preserve significant structure. This process precedes step S2 and is independent of its detection results.

[0082]

[0083] Wherein, represents the sequence number of each template response position after the line structure template is convolved with the candidate path graph. This represents the coefficient of the i-th original template. This represents the result of soft-threshold shrinkage of the i-th coefficient. This represents the current noise filtering threshold; the sign function is used to preserve the positive and negative directions of the template coefficients, and the maximum value function is used to set the amplitude that is not greater than zero after shrinkage to zero. Used to reconstruct dynamically thresholded denoised images, the larger coefficients retained correspond to significant line structure responses such as conductor outlines, insulator skirts, or clamp edges.

[0084] In this embodiment, the step of denoising the original image to obtain a preprocessed image includes: performing soft threshold denoising on the original image based on the current noise filtering threshold, and performing multi-algorithm joint denoising on the soft threshold denoised image to obtain a preprocessed image; and associating and saving the environmental disturbance index with the preprocessed image.

[0085] In this embodiment, the method of joint noise reduction using multiple algorithms is as follows:

[0086] The multi-algorithm joint denoising includes the following processing: local and global threshold segmentation based on grayscale, chroma, hue, and location information to obtain suspected component regions and suspected noise regions; soft segmentation of the two types of regions using fuzzy membership mean clustering; median filtering for impulse noise such as strong light spots and dark spots; and background estimation and brightness compensation for brightness gradients caused by backlighting. For each processing result, the correlation coefficient of the gradient magnitude before and after processing within the candidate path region is calculated as an edge preservation index, and the residual noise variance outside the candidate path region is calculated; processing results with higher edge preservation indices and lower residual noise variance are assigned higher weights, and pixel-level weighted fusion is performed after normalizing the weights. Figure 2 and Figure 3 The processing results are shown for strong light and backlight scenes respectively.

[0087] Compared with existing technologies, the environmental adaptive denoising technology of this invention has significant advantages. Most existing technologies employ fixed preprocessing procedures or simple data augmentation, making them difficult to adapt to complex and changing environmental conditions. This invention achieves intelligent adjustment of image quality under complex environmental conditions through dynamic calculation of environmental disturbance indices and adaptive updating of noise filtering thresholds. The environmental disturbance indices are not only used to update the noise filtering thresholds but also to filter fault templates that match the applicable environmental range, forming a processing link with a clear data transmission relationship between image preprocessing, template retrieval, and alarm strategies. This end-to-end environmental adaptive mechanism significantly improves the system's adaptability to different lighting, weather, and shooting conditions, solving the problems of image quality degradation and easy miss of small defects in complex environments such as strong light, backlight, rain, fog, and motion blur in existing technologies.

[0088] Step S2: Input the preprocessed image into the real-time target detection model to obtain candidate bounding boxes of line components, extract candidate component images based on the candidate bounding boxes, and input the candidate component images into a deep belief network for fine classification to obtain images of core line components and component fine classification categories.

[0089] This embodiment employs a single-stage object detection model to obtain candidate bounding boxes, component categories, and detection confidence scores. Candidate bounding boxes below a preset detection threshold are eliminated, and non-maximum suppression is applied to overlapping candidate bounding boxes. Candidate component images are extracted from the retained candidate bounding boxes and then input into a deep confidence network for fine classification. By first locating and then finely classifying, the overall image fault analysis is transformed into fault analysis within the candidate component range; the fine classification categories are also used in step S5 to select a fault template sub-library.

[0090] In this embodiment, the step of inputting the preprocessed image into the real-time target detection model to obtain candidate bounding boxes for line components includes: inputting the preprocessed image into a single-stage target detection model to obtain candidate bounding boxes for line components, component categories, and detection confidence scores; the real-time target detection model outputs at least one component category among conductors, insulators, clamps, joints, and tower connectors, and removes candidate bounding boxes with detection confidence scores lower than a preset detection threshold, and performs non-maximum suppression on overlapping candidate bounding boxes.

[0091] The deep belief network in this embodiment is formed by stacking multiple restricted Boltzmann machines layer by layer. Each restricted Boltzmann machine includes a visible layer and a hidden layer, with fully connected layers between layers and no connections within layers. The output of the lower layer serves as the input of the upper layer, enabling the network to extract high-order features from texture, edges, and component structures layer by layer. It should be noted that the deep belief network belongs to the probabilistic generative model, and this embodiment does not confuse it with convolutional neural networks.

[0092] Compared with existing technologies, the deep belief network (DBN) component fine-classification technology employed in this invention has unique advantages. Existing technologies mainly focus on broad category identification, easily confusing similar-looking components such as conductors, clamps, and connectors. This invention introduces DBN for fine-classification, reducing cross-component mismatches. As a probabilistic generative model, DBN is suitable for learning component structural features with small sample sizes. Through layer-by-layer unsupervised pre-training and supervised fine-tuning, it improves the ability to distinguish between similar-looking components. This invention applies DBN to the fine-classification of power line components for the first time, achieving an innovative application in the technical field. The fine-classification results are also used for template sub-library selection, improving matching efficiency and forming a complete fine-classification-template selection closed loop.

[0093] In an embodiment, the deep belief network comprises at least two stacked restricted Boltzmann machines; the step of inputting the candidate component image into the deep belief network for fine classification includes: inputting the normalized candidate component image features into a first restricted Boltzmann machine to obtain a first hidden layer state; inputting the first hidden layer state into a second restricted Boltzmann machine to obtain a second hidden layer state; inputting the second hidden layer state into a classification output layer, calculating the probability of each component's fine classification category using a normalized exponential function, and using the category with the highest probability as the component's fine classification category.

[0094] In one implementation, the deep belief network includes a first restricted Boltzmann machine, a second restricted Boltzmann machine, and a classification output layer. The hidden state of the first restricted Boltzmann machine serves as the input to the visible layer of the second restricted Boltzmann machine, and the hidden state of the second restricted Boltzmann machine is output by the classification output layer as a fine-classification probability. Figure 4In this model, the input layer represents the normalized feature vector of the candidate components, the first hidden layer and the second hidden layer represent the hidden states of the first and second Restricted Boltzmann Machines, respectively, and the classification output layer outputs the probability of each component's sub-classification category using a normalized exponential function. The category corresponding to the highest category probability is taken as the sub-classification result of the component, and this probability is used as the sub-classification confidence. The number of neurons in each layer is determined based on the size of the training samples and the results of the validation set.

[0095] The forward propagation process of the deep belief network is as follows:

[0096] Step ST1: Denote the normalized candidate component image features or the component feature vector extracted by the real-time object detection model as v, and input it into the visible layer of the first restricted Boltzmann machine.

[0097] Step ST2: Calculate the activation state of the first hidden layer based on the probability model of the first restricted Boltzmann machine.

[0098] The activation probability of the j-th neuron in the first hidden layer is calculated by using the candidate component features, connection weights, and hidden layer biases via a logistic activation function:

[0099]

[0100] Where j represents the sequence number of the first hidden layer neuron, i represents the sequence number of the candidate circuit component feature, and n represents the total number of input features; The i-th normalized feature of the candidate part image is represented, wherein the feature is one of edge, texture, color, or shape features; This represents the connection weights learned from the candidate component training samples, used to measure the contribution of the i-th input feature to the j-th hidden neuron; This represents the bias of the j-th hidden neuron. This indicates the probability that the neuron will be activated for the current candidate component. (·) represents a logical activation function.

[0101] Bernoulli sampling is performed based on the activation probability of the first hidden layer to obtain the state of the first hidden layer:

[0102]

[0103] in, This represents the state vector generated by the first hidden layer for the current candidate line component. Let represent the state or activation probability of the j-th neuron in the reasoning stage, m represent the total number of neurons in the first hidden layer, the superscript (1) represents the first hidden layer, and the superscript T represents the arrangement of the states of each neuron into a column vector. This state vector is used to characterize the edge, texture, and structural combination features of the candidate component and serves as the input to the second restricted Boltzmann machine.

[0104] The Bernoulli sampling is represented as follows:

[0105]

[0106] in, This represents the activation probability of the j-th neuron in the first hidden layer. This represents a uniformly distributed random number generated independently for this neuron within the interval [0,1]. This represents the state of the neuron after sampling; a value of 1 indicates that the corresponding combination features of the current candidate component are activated, and a value of 0 indicates that it is not activated. Random sampling is used during the training phase, while it can be directly used during the inference phase. As a deterministic eigenvalue.

[0107] Step ST3: Using the activation state of the first restricted Boltzmann machine hidden layer as input, calculate the activation state of the second hidden layer according to the probability model of the second restricted Boltzmann machine.

[0108] The state of the first hidden layer is used as the input to the visible layer of the second restricted Boltzmann machine, and the activation probability and state of the k-th neuron in the second hidden layer are calculated in the same way as in step ST2.

[0109] The calculation process is as follows:

[0110]

[0111]

[0112] Where j represents the sequence number of the first hidden layer neuron, m represents the total number of neurons in the first hidden layer, and k represents the sequence number of the second hidden layer neuron. This represents the state of the candidate component combination features extracted from the first hidden layer. This indicates the connection weight of the state to the k-th neuron in the second hidden layer. This represents the bias of the k-th neuron. This represents the probability that the neuron is activated for the current candidate component, and σ(·) represents the logistic activation function; This represents a uniformly distributed random number generated independently for the k-th neuron within the interval [0,1]. This represents the state of the second hidden layer after sampling. A value of 1 indicates that the corresponding higher-order component structural features are activated, while a value of 0 indicates that they are not activated.

[0113] Step ST4: Calculate the fine classification results of candidate circuit components based on the activation state of the second restricted Boltzmann machine hidden layer.

[0114] The classification output layer calculates the original scores of the fine-grained categories of each component according to the second hidden layer state, and converts the original scores into category probabilities using a normalized exponential function:

[0115]

[0116]

[0117] wherein, o represents the sequence number of a specific fine-grained category of conductors, insulators, clamps, joints or tower connecting parts, k represents the sequence number of neurons in the second hidden layer, and K represents the total number of neurons in the second hidden layer; represents the k-th high-order component structure feature state, represents the connection weight of the feature for the original score of the o-th category, represents the output bias of the o-th category, represents the original score of the o-th category; the exponential function is used to convert the original scores into positive values, r represents the category sequence number in normalized summation, N represents the total number of fine-grained categories, represents the probability that the current candidate component belongs to the o-th category. Take the category with the maximum value as the fine-grained classification result of the component, and take the maximum probability as the fine-grained classification confidence.

[0118] Training of the deep belief network includes layer-wise unsupervised pre-training and full-network supervised fine-tuning:

[0119] P1: Adopt the candidate component image features in the training set, and train the connection weights and biases of the first restricted Boltzmann machine through the contrast divergence algorithm;

[0120] P2: Fix the parameters of the first restricted Boltzmann machine, take the output of the first hidden layer as the input of the second restricted Boltzmann machine, and train the second restricted Boltzmann machine through the contrast divergence algorithm;

[0121] P3: If more hidden layers are set, complete the unsupervised pre-training layer by layer in the same way;

[0122] P4: Connect the classification output layer, use labeled samples, take cross entropy as the loss function, and perform supervised fine-tuning on the entire network through back propagation; during the training process, an early stopping strategy is adopted according to the validation set loss, and the model parameters with the highest validation set accuracy are saved.

[0123] Layer-wise pre-training enables the network to learn component structure features even when labeled samples are limited, and supervised fine-tuning is used to strengthen the discrimination boundaries between different fine-grained categories. This can improve the distinguishing ability of components with similar appearances such as conductors and clamps, insulators and connecting parts.

[0124] Step S3: Input the image of the core component of the line into the two-stage target detection model to locate the fault candidate region, extract local features from the fault candidate region to construct a feature matrix, perform singular value decomposition on the feature matrix and reconstruct it to obtain the core fault feature matrix.

[0125] In a preferred embodiment of the present invention, the step of inputting the image of the core component of the line into a two-stage target detection model to locate the fault candidate region includes: inputting the image of the core component of the line into a two-stage target detection model having a region proposal network and a region classification regression branch; generating candidate regions by the region proposal network; determining the region of interest based on the fault probability and location regression results of the candidate regions; and mapping the region of interest to the fault candidate position in the original image; when there is no candidate region with a fault probability reaching a preset region threshold, the corresponding core component of the line is marked as a normal component.

[0126] The images of core components of the line are input into a two-stage target detection model with a region proposal network and a region classification and regression branch. The region proposal network generates candidate regions, which are then classified as faulty or non-faulty and their locations are regressed. Candidate regions with a fault probability reaching a preset region threshold are identified as regions of interest (ROIs). Based on the coordinate transformation relationship recorded during candidate box clipping, the ROI locations are back-mapped to the original inspection images to obtain the fault candidate locations. Step S3 outputs the ROI image, component sub-classification, fault candidate locations, and region confidence scores, such as... Figure 5 and Figure 6 As shown. Figure 5 and Figure 6 The value next to the middle frame is the region confidence level, ranging from [0,1], used for candidate region filtering, and does not represent the final failure probability; the region confidence level is denoted as C_2 in subsequent calculations. When there are no candidate regions with a failure probability reaching the preset region threshold, a normal state is output and the fault matching process for that component ends.

[0127] This invention utilizes a real-time target detection model to complete component localization and broad category selection, then employs a deep belief network for finer classification, and finally uses a two-stage target detection model to locate fault areas, forming a complete three-level architecture of "localization-fine classification-fault detection". This hierarchical architecture effectively resolves the contradiction between computational efficiency, component identification accuracy, and fault detection capability in existing technologies, achieving a balance between accuracy and efficiency.

[0128] In this embodiment, the step of extracting local features from the candidate fault region to construct a feature matrix includes: extracting at least two of edge features, corner features, texture features, and color features from the candidate fault region, and constructing a suspected fault feature matrix according to the spatial location of each normalized feature.

[0129] The process of performing singular value decomposition and reconstructing the core fault feature matrix to obtain the core fault feature matrix includes: performing singular value decomposition on the suspected fault feature matrix, selecting the top k singular values ​​and corresponding singular vectors that make the cumulative energy contribution rate reach a preset contribution rate threshold, and reconstructing the core fault feature matrix with rank k.

[0130] The image of the region of interest is scaled uniformly, and the luminance and color channels are normalized separately. On the luminance channel, a differential edge detection operator is used to obtain features such as edge amplitude, edge direction, edge length, and edge continuity. A multi-branch corner detection operator is used to obtain features such as corner position, corner response intensity, and number of branches. Texture features are obtained by statistically analyzing gray-level co-occurrence relationships or local binary patterns within the local neighborhood, and color features are obtained by statistically analyzing the partition histograms of the color channels. After normalizing at least two types of features, they are arranged according to their spatial positions in a unified grid to form a suspected fault feature matrix A. The template image uses the same scale, grid, and feature order.

[0131] Compared with existing technologies, the singular value decomposition (SVD) low-rank reconstruction fault feature extraction technology of this invention has unique advantages. Existing technologies directly use deep features or manual features, which are easily affected by noise interference, and small-scale defect features are not obvious. This invention retains the main feature energy and suppresses noise through SVD low-rank reconstruction. The order of low-rank reconstruction is determined by the cumulative energy contribution rate of singular values, which suppresses low-energy noise components while retaining the main fault features such as cracks, broken strands, and wear, making it suitable for handling fault feature extraction in small-scale defects and complex backgrounds. No identical technical solutions using SVD low-rank reconstruction for power line fault feature extraction were found in the search, demonstrating the innovation of this invention.

[0132] To reduce feature redundancy and improve subsequent matching efficiency, singular value decomposition is performed on feature matrix A:

[0133] The singular value decomposition formula is:

[0134]

[0135] Where A represents the suspected fault feature matrix composed of local fault features of the current region of interest, with each row corresponding to the spatial grid position in the region of interest and each column corresponding to the edge, corner, texture, or color feature dimension; U represents the left singular vector matrix reflecting the spatial distribution pattern of fault features; Σ represents the diagonal matrix with the singular values ​​of each fault feature component arranged in descending order; V represents the right singular vector matrix reflecting the combination direction of each feature dimension; and the superscript T indicates matrix transpose. The larger the singular value, the greater the contribution of the corresponding feature component to the current suspected fault region.

[0136] Let the suspected fault feature matrix A contain n spatial grid locations and d feature dimensions, i.e., A is an n×d matrix; n represents the total number of grid locations in the region of interest, d represents the total number of edge, corner, texture, and color feature dimensions actually used, r represents the rank of A, and i represents the singular value index. This represents the i-th singular value in descending order. The cumulative energy contribution rate of the first k singular values. Defined as:

[0137]

[0138] in, This represents the proportion of energy retained by the core fault feature compared to the original suspected fault feature, where k represents the number of fault feature components to be retained. Let represent the energy of the i-th fault characteristic component, with the numerator being the sum of the energies of the first k components and the denominator being the sum of the energies of all r effective components of the matrix. The choice is to make... The smallest positive integer k that first reaches the preset contribution rate threshold is used to construct the first k columns of U. Take the first k singular values ​​to form And take the first k columns of V to form Low-rank reconstruction of the suspected fault feature matrix:

[0139]

[0140] in, This represents the core fault feature matrix of the current region of interest. This represents the left singular vector corresponding to the first k main spatial distribution patterns. This represents the diagonal matrix formed by the first k singular values. Let T represent the right singular vector corresponding to the combination direction of the first k main features. The superscript T indicates matrix transpose, and k indicates the low-rank reconstruction order determined by the contribution rate threshold. Consistent with size A, it is used to preserve key failure characteristics such as cracks, broken strands, wear, or corrosion, and to suppress low-energy noise components.

[0141] In this embodiment, the cumulative energy contribution rate is the ratio of the sum of squares of the first k singular values ​​to the sum of squares of all singular values, where k is a positive integer that makes the cumulative energy contribution rate reach a preset contribution rate threshold for the first time. The suspected fault feature matrix is ​​reconstructed in a low-rank manner by the first k columns of the left singular vector matrix, the diagonal matrix formed by the first k singular values, and the first k rows of the right singular vector matrix, to obtain the core fault feature matrix with rank k.

[0142] Step S4: Select a fault template sub-library based on the component sub-classification and the environmental disturbance index, perform feature matching between the core fault feature matrix and the fault templates in the fault template sub-library, and output the normal state or fault type, location and confidence level based on the matching result.

[0143] In this embodiment, the step of selecting the fault template sub-library based on the component sub-category and the environmental disturbance index includes: determining the target template sub-library from the fault template library organized according to component category, fault type and applicable environmental range based on the component sub-category and the environmental disturbance index.

[0144] The step of matching the core fault feature matrix with the fault templates in the fault template sub-library includes: performing overall feature matching and local feature matching on the core fault feature matrix and the fault templates in the target template sub-library in sequence to obtain a local matching score;

[0145] The step of outputting the normal state or fault type, location and confidence level based on the matching result includes: classifying and judging the local matching score according to the matching threshold corresponding to the fault hazard level marked by the candidate template, outputting the normal state, or outputting the fault type, fault candidate location and result confidence level, or marking the detection result as an unknown fault or pending manual review.

[0146] The fault template library is indexed by component category (primary index) and by fault type (secondary index), and the applicable imaging conditions are recorded according to environmental disturbance index ranges. Each template includes at least the component category, fault type, overall features, local features, hazard level, sample source, applicable environmental range, and template version. The component sub-categories output in step S2 and the environmental disturbance indexes output in step S1 are used together to select the template sub-library, thereby avoiding cross-component comparisons and reducing mismatches caused by inconsistent conditions such as strong light, backlight, or motion blur. When the number of templates in the target environmental range is insufficient, the library reverts to the next higher-level template sub-library within the same component category, and the results are transferred for manual review.

[0147] Compared with existing technologies, the hierarchical matching and graded judgment fault identification strategy of this invention has significant advantages. Existing technologies mostly employ end-to-end classification, lacking a confidence level grading mechanism. This invention uses overall matching screening, local matching confirmation, and graded threshold judgment, balancing detection accuracy and false alarm rate. Different thresholds are set according to the hazard level, prioritizing warnings for high-hazard level faults, while results with insufficient evidence are transferred to manual review or marked as unknown faults, avoiding false alarms caused by forced classification, and supporting the marking of unknown faults and continuous knowledge accumulation.

[0148] Specifically, step S4 includes the following processes:

[0149] Step S41: Establish a fault template library. Collect historical fault images, maintenance records, and on-site verification results that have been manually verified. Label the samples with component categories, fault types, fault locations, and hazard levels. Perform preprocessing and feature extraction on the images in the same manner as online detection to generate template features.

[0150] Key features are defined for different fault types. For example, crack templates include edge length, width, direction, and continuity; broken strand templates include the degree of damage to parallel line structures and the number of breakpoints; wear or corrosion templates include texture roughness and color distribution; and deformation or loosening templates include changes in corner position and geometric offset.

[0151] Templates are indexed by component category as a primary index and by fault type as a secondary index. Template version, verification results, and applicable scenarios are recorded to facilitate quick retrieval, version rollback, and offline updates.

[0152] Step S42: Perform overall matching and local matching on the core fault features.

[0153] In a preferred embodiment of the present invention, the step of sequentially performing overall feature matching and local feature matching between the core fault feature matrix and the fault templates in the target template sub-library includes: in the target template sub-library corresponding to the component sub-classification category and the environmental interval where the environmental disturbance index is located, the core fault feature matrix and the overall feature matrix of each fault template are vectorized in the same order, the cosine similarity between the vectorization results is calculated, and candidate templates whose overall similarity reaches the overall matching threshold are selected; the similarity between the core fault feature and each candidate template in at least two local features in edge, corner, texture and color is calculated respectively; according to the fault type corresponding to the candidate template, feature weights are set for each local feature similarity and weighted summation is performed to obtain the local matching score.

[0154] During the overall matching phase, the core fault feature matrix is ​​only matched within the template sub-library corresponding to the component sub-categories and environmental disturbance indexes. and template overall feature matrix Vectorize in the same order and calculate the normalized cosine similarity. ;filter Templates that meet the overall matching threshold are used as candidate templates. If an abnormal region has been detected in step S3 but no candidate template exists, an unknown fault is output and the process proceeds to manual review, without forcibly classifying the sample into an existing fault type. The normalized cosine similarity calculation formula is:

[0155]

[0156] in, This indicates the overall feature similarity between the currently suspected faulty area and a candidate fault template; This represents the core feature matrix of the currently suspected faulty area. This represents the core feature matrix of the candidate template obtained using the same preprocessing, feature order, and low-rank reconstruction method. The vectorization operator represents expanding the matrix into column vectors in the same order. The superscript T indicates the vector transpose, and the double vertical bar and subscript 2 indicate the L2 norm. The numerator represents the inner product of two core feature vectors, and the denominator is used to eliminate differences in feature magnitude. The larger the value, the more similar the candidate region is to the overall structure of the fault template.

[0157] In the local matching stage, the similarity between the candidate sample and the candidate template is calculated on at least two local features in terms of edges, corners, texture, and color. Let D be the number of local features actually used, and the similarity of the d-th class of local features be... The corresponding non-negative weights are Each weight is determined and normalized on an independent validation set, and the local matching score is calculated. The formula for calculation is:

[0158]

[0159] in, This represents the local matching score between the current suspected fault area and the candidate fault template, where D represents the total number of local feature categories actually involved in the matching, and d represents the local feature category index. The similarity of local features of type d is represented by the local feature being one of edge, corner, texture, or color features. This represents the weight of the d-th type of feature determined on the independent validation set based on the fault type corresponding to the candidate template. All weights are non-negative and their sum is 1. The larger the value, the more consistent the current region is with the candidate template in terms of key local fault characteristics.

[0160] For each fault type, an initial screening threshold and a confirmation threshold that is not lower than the initial screening threshold are set. High-risk faults use relatively lower initial screening thresholds to reduce the risk of missed detections; low-risk faults use relatively higher confirmation thresholds to reduce the risk of false alarms. Each threshold should be determined using a validation set independent of the training set, based on precision, recall, and false alarm rate.

[0161] In a preferred embodiment of the present invention, when there is no candidate region with a fault probability reaching the preset region threshold in step S3, a normal state is output; when there is a candidate region in step S3, but no candidate template reaching the overall matching threshold, or when the local matching score is lower than the corresponding initial screening threshold, the detection result is marked as an unknown fault; when the local matching score reaches the initial screening threshold but is lower than the confirmation threshold, the detection result is marked as pending manual review; when the local matching score reaches the confirmation threshold, the matched fault type is output.

[0162] As an example, initial screening thresholds and confirmation thresholds can be set for high-risk faults on the verification data: when the local matching score is lower than the initial screening threshold, it is marked as an unknown fault; when it reaches the initial screening threshold but is lower than the confirmation threshold, an early warning is output and it is marked as awaiting manual review; when it reaches the confirmation threshold, a confirmed fault is output. The specific thresholds are determined by the actual verification results and are not limited to fixed values.

[0163] For defects with low risk levels, the confirmation threshold can be increased; when the local matching score is insufficient for confirmation, the candidate fault location and related features are retained and transferred to manual review, instead of directly generating a confirmation alarm.

[0164] In this embodiment, the local matching scores of each fault type are calibrated to the same value range by an independent validation set; the initial screening threshold for high-risk faults is lower than that for low-risk faults; for detection results that reach the initial screening threshold for high-risk faults but do not reach the confirmation threshold, an early warning is output and marked as pending manual review.

[0165] The final detection results should include at least the inspection image identifiers, component subcategories, normal status or fault type, unknown fault markers, fault candidate locations, result confidence level, hazard level, and whether manual verification is required. Let the single-stage target detection model provide a detection confidence level for the current line component candidate boxes. The two-stage target detection model gives the region confidence score for the current fault candidate region as follows: The local matching score between the current candidate fault features and the target template is: All three items are calibrated to [0,1]; the weights of the final result confidence C. , and Determined through an independent validation set, the calculation formula is:

[0166]

[0167] in, This indicates the final confidence level of the current line component inspection results. This represents the detection confidence level given by the single-stage target detection model for the candidate bounding boxes of the line component. This represents the regional confidence level given by the two-stage target detection model for the fault candidate region within this component. Indicates the score of a local match; , and represents the fusion weights of the three pieces of evidence: component detection, fault area localization, and local template matching, respectively, and 'a' represents the summation index of the three weights. Each input is first calibrated to [0,1], and each weight is determined on an independent validation set and sums to 1.

[0168] In this embodiment, the test results to be manually reviewed are reviewed offline, and the reviewed and confirmed samples are added to the candidate template library; regression testing is performed on the candidate template library, and a new version of the fault template library with a version identifier is formed after passing the regression test; the online detection terminal only loads the fault template library that has passed the regression test in read-only mode, and retains the previous version to support rollback.

[0169] Compared with existing technologies, the closed-loop knowledge update and version management mechanism of this invention has significant advantages. Existing technologies lack a continuous learning mechanism and struggle to adapt to newly emerging fault types and changing environmental conditions. This invention achieves knowledge accumulation through manual review, regression testing, and version management. Manual review and regression testing are performed offline; the online terminal only loads verified template libraries with version identifiers and retains rollback versions. This design effectively reduces the risk of unverified templates affecting online detection stability while continuously expanding fault knowledge, providing a reliable mechanism for continuous system optimization and knowledge accumulation.

[0170] Manual review is performed offline. Reviewers confirm the component category, fault type, hazard level, and applicable environment range of the samples to be reviewed, and add the confirmed samples to the candidate template library. Regression tests are performed on the candidate template library using historical confirmed samples and normal samples. A new version of the template library is only formed when the recall rate and false positive rate of the preset categories meet the release conditions. The new version of the template library has a unique version identifier, is loaded in read-only mode on the online terminal, and retains the previous version for rollback in case of abnormal detection indicators.

[0171] The above method can be implemented by an image acquisition and preprocessing module, a component detection and fine classification module, a fault area localization module, a fault feature extraction module, a fault determination module, and a template library management module. Each module can be stored as program instructions executed by the processor in a memory or a computer-readable storage medium; when the processor executes the program instructions, it completes image processing, fault determination, and result output according to steps S1 to S4.

[0172] Figure 7 The graph illustrates the relative detection performance trends of the statistical method, template matching method, and the method of this invention during continuous training or parameter updates. The horizontal axis represents the number of training or parameter updates: for the statistical method and template matching method, it corresponds to the number of updates to their decision parameters; for the method of this invention, it corresponds to the number of model training epochs. The vertical axis represents the normalized index used to reflect the relative detection performance of each method. The curves in the graph are used to illustrate the relative trends of each method with training or parameter updates, and do not constitute a limitation on the specific test sample size, scenario conditions, or performance values; the specific performance should be based on the results obtained based on the independent validation set in the actual application scenario, and the matching threshold is also determined by the independent validation set.

[0173] In summary, this invention forms a complete technical solution through adaptive image processing driven by environmental perturbation indicators, component fine-classification technology based on deep belief networks, fault feature extraction through low-rank reconstruction of singular value decomposition, fault identification strategies based on hierarchical matching and graded judgment, and a closed-loop knowledge update and version management mechanism. There are clear synergistic effects among the various technical means: environmental perturbation indicators are used not only for image preprocessing but also for template selection, forming a data transmission link; the fine-classification results of deep belief networks are used for template sub-library selection, improving matching efficiency; low-rank reconstruction preserves the energy of key features, providing high-quality features for subsequent matching; hierarchical matching and graded judgment balance accuracy and false alarm rate; and the closed-loop knowledge update mechanism ensures the system's continuous optimization capability. The combination of these technical means is not a simple superposition of existing technologies, but rather produces significant technical effects through a carefully designed synergistic mechanism, solving multiple problems that existing technologies struggle to address simultaneously.

[0174] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A visual detection method for power line faults based on unmanned aerial vehicle (UAV) inspection, characterized in that, Includes the following steps: The process involves acquiring original images of power lines collected by drone inspections and corresponding environmental information, calculating environmental disturbance indices based on the environmental information, updating noise filtering thresholds based on the environmental disturbance indices, and performing denoising on the original images based on the current noise filtering thresholds to obtain preprocessed images. The preprocessed image is input into a real-time target detection model to obtain candidate bounding boxes for line components. Based on the candidate bounding boxes, candidate component images are cropped and input into a deep belief network for fine classification to obtain images of core line components and component fine classification categories. The image of the core component of the line is input into a two-stage target detection model to locate the fault candidate region. Local features are extracted from the fault candidate region to construct a feature matrix. The feature matrix is ​​then subjected to singular value decomposition and reconstructed to obtain the core fault feature matrix. Based on the component sub-classification and the environmental disturbance index, a fault template sub-library is selected. The core fault feature matrix is ​​matched with the fault templates in the fault template sub-library. Based on the matching result, the normal state or fault type, location and confidence level are output.

2. The visual detection method for power line faults according to claim 1, characterized in that, Updating the noise filtering threshold based on the environmental disturbance index includes: updating the current noise filtering threshold based on the environmental disturbance index and the noise filtering threshold at the previous moment.

3. The visual detection method for power line faults according to claim 1, characterized in that, The process of denoising the original image to obtain a preprocessed image includes: performing soft threshold denoising on the original image based on the current noise filtering threshold, and performing multi-algorithm joint denoising on the soft threshold denoised image to obtain a preprocessed image; and associating and saving the environmental disturbance index with the preprocessed image.

4. The visual detection method for power line faults according to claim 3, characterized in that, The multi-algorithm joint denoising includes one or more of the following: (1) Perform local thresholding and global thresholding based on grayscale, chroma, hue and position information; (2) Use fuzzy membership mean clustering to perform soft segmentation of suspected line component regions and suspected noise regions; (3) Perform median filtering on the impulse noise region; (4) Perform background estimation and brightness compensation on regions with gradual brightness changes; The edge preservation index within the candidate path region and the noise variance outside the candidate path region are calculated separately for each processing result. The normalized fusion weight is determined with the edge preservation index as the main factor and the noise variance as the secondary factor to obtain the preprocessed image.

5. The visual detection method for power line faults according to any one of claims 1-3, characterized in that, The current noise filtering threshold is calculated by weighting the previous noise filtering threshold, the initial noise filtering threshold, and the environmental disturbance index. A candidate path map is generated based on the linear structure and tower outline in the original image. The line structure filtering template used to characterize the conductor outline, insulator skirt, or clamp edge is convolved with the candidate path map to obtain template coefficients. Template coefficients with absolute values ​​less than the current noise filtering threshold are set to zero, and soft thresholding is performed on the remaining template coefficients.

6. The visual detection method for power line faults according to claim 1, characterized in that, The collected environmental information includes at least two of the following: average image brightness, brightness variance, contrast, image sharpness, and UAV attitude change. Each piece of collected environmental information is normalized and then weighted and summed according to a preset environmental weight to obtain the environmental disturbance index.

7. The visual detection method for power line faults according to claim 1, characterized in that, The process of inputting the preprocessed image into a real-time target detection model to obtain candidate bounding boxes for line components includes: inputting the preprocessed image into a single-stage target detection model to obtain candidate bounding boxes for line components, component categories, and detection confidence scores; the real-time target detection model outputs at least one component category among conductors, insulators, clamps, joints, and tower connectors, and removes candidate bounding boxes with detection confidence scores lower than a preset detection threshold, and performs non-maximum suppression on overlapping candidate bounding boxes.

8. The visual detection method for power line faults according to claim 1, characterized in that, The deep belief network comprises at least two restricted Boltzmann machines stacked together; the step of inputting the candidate component images into the deep belief network for fine classification includes: The normalized candidate component image features are input into the first restricted Boltzmann machine to obtain the first hidden layer state; The first hidden layer state is input into the second restricted Boltzmann machine to obtain the second hidden layer state; The state of the second hidden layer is input into the classification output layer, and the probability of each component sub-class is calculated using the normalized exponential function. The category with the highest probability is taken as the component sub-class.

9. The visual detection method for power line faults according to claim 8, characterized in that, The activation probability of the k-th hidden neuron in layer L is calculated by the state of each neuron in layer L-1, the corresponding connection weights, and the bias of the k-th hidden neuron in layer L using a logical activation function, and Bernoulli sampling is performed based on the activation probability. The activation probability of the k-th hidden neuron in the L-th layer is expressed as: Bernoulli sampling is performed based on the activation probability to obtain the hidden neuron state: Where L represents the hidden layer number of the deep belief network; k represents the neuron number of the Lth layer; j represents the neuron number of the previous layer; and m represents the total number of neurons in the previous layer. This represents the probability that the k-th neuron in layer L is activated for the current candidate circuit component. This indicates the feature state of the j-th component in the previous layer. This represents the connection weights learned from training samples of candidate circuit components. This indicates the bias of the current neuron. This represents the current neuron state after sampling; σ represents the logistic activation function, whose expression is: ; x represents the weighted input formed by candidate component features, connection weights, and biases, and e represents the natural constant; This represents a uniformly distributed random number generated independently for the current neuron within the interval [0,1].

10. The visual detection method for power line faults according to claim 1, characterized in that, The process of inputting the image of the core components of the line into a two-stage target detection model to locate fault candidate regions includes: inputting the image of the core components of the line into a two-stage target detection model with a region proposal network and a region classification regression branch; generating candidate regions by the region proposal network; determining the region of interest based on the fault probability and location regression results of the candidate regions; and mapping the region of interest to the fault candidate locations in the original image; when there are no candidate regions with a fault probability reaching a preset region threshold, the corresponding core components of the line are marked as normal components.

11. The visual detection method for power line faults according to claim 1, characterized in that, Extracting local features from the candidate fault region to construct a feature matrix includes: extracting at least two of the following from the candidate fault region: edge features, corner features, texture features, and color features; and constructing a suspected fault feature matrix by arranging the normalized features according to their spatial location.

12. The visual detection method for power line faults according to claim 11, characterized in that, The process of performing singular value decomposition and reconstructing the core fault feature matrix to obtain the core fault feature matrix includes: performing singular value decomposition on the suspected fault feature matrix, selecting the top k singular values ​​and corresponding singular vectors that make the cumulative energy contribution rate reach a preset contribution rate threshold, and reconstructing the core fault feature matrix with rank k.

13. The visual detection method for power line faults according to claim 12, characterized in that, The cumulative energy contribution rate is the ratio of the sum of squares of the first k singular values ​​to the sum of squares of all singular values, where k is a positive integer that makes the cumulative energy contribution rate reach a preset contribution rate threshold for the first time. The suspected fault feature matrix is ​​reconstructed in a low-rank manner by the diagonal matrix formed by the first k columns of the left singular vector matrix, the first k singular values, and the first k rows of the right singular vector matrix, to obtain the core fault feature matrix with rank k.

14. The visual detection method for power line faults according to claim 1, characterized in that, Selecting a fault template sub-library based on the component sub-categories and the environmental disturbance indicators includes: determining a target template sub-library from a fault template library organized according to component category, fault type, and applicable environmental range based on the component sub-categories and the environmental disturbance indicators. The feature matching of the core fault feature matrix with the fault templates in the fault template sub-library includes: performing overall feature matching and local feature matching on the core fault feature matrix and the fault templates in the target template sub-library in sequence to obtain a local matching score; Outputting the normal state or fault type, location, and confidence level based on the matching results includes: classifying and judging the local matching score according to the matching threshold corresponding to the fault hazard level marked by the candidate template, outputting the normal state, or outputting the fault type, fault candidate location, and result confidence level, or marking the detection result as an unknown fault or pending manual review.

15. The visual detection method for power line faults according to claim 14, characterized in that, The step of sequentially performing overall feature matching and local feature matching between the core fault feature matrix and the fault templates in the target template sub-library includes: in the target template sub-library corresponding to the component sub-classification category and the environmental interval where the environmental disturbance index is located, the core fault feature matrix and the overall feature matrix of each fault template are vectorized in the same order, the cosine similarity between the vectorization results is calculated, and candidate templates whose overall similarity reaches the overall matching threshold are selected; the similarity between the core fault feature and each candidate template in at least two local features in edge, corner, texture and color is calculated respectively; according to the fault type corresponding to the candidate template, feature weights are set for each local feature similarity and weighted summation is performed to obtain the local matching score.

16. The visual detection method for power line faults according to claim 15, characterized in that, When there are no candidate regions whose failure probability reaches the preset region threshold, the output is a normal state. When there is a candidate region with a fault probability that reaches the preset regional threshold, but there is no candidate template that reaches the overall matching threshold, or the local matching score is lower than the corresponding initial screening threshold, the detection result will be marked as an unknown fault. When the local matching score reaches the initial screening threshold but is lower than the confirmation threshold, the detection result is marked as pending manual review; When the local matching score reaches the confirmation threshold, the matched fault type is output.

17. The visual detection method for power line faults according to claim 16, characterized in that, The local matching scores for each fault type are calibrated to the same value range using an independent validation set; the initial screening threshold for high-risk faults is lower than that for low-risk faults; for detection results that reach the initial screening threshold for high-risk faults but do not reach the confirmation threshold, an early warning is output and marked as pending manual review.

18. The visual detection method for power line faults according to claim 17, characterized in that, Offline review is conducted on the test results that have been manually verified, and the confirmed samples are added to the candidate template library. Regression testing is performed on the candidate template library, and a new version of the fault template library with a version identifier is formed after passing the regression test. The online detection terminal only loads the fault template library that has passed the regression test in read-only mode, and retains the previous version to support rollback.

19. A visual inspection system for power line faults based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: The image acquisition and preprocessing module is used to acquire the original images of power lines and the acquisition environment information collected by the UAV inspection, and update the noise filtering threshold according to the acquisition environment information to obtain the preprocessed image. The component detection and fine classification module is used to obtain candidate boxes of circuit components, component categories, and fine classification categories. The fault region localization module is used to determine candidate fault regions and map them to the original image through a two-stage target detection model with a region proposal network and a region classification regression branch. The fault feature extraction module is used to construct a suspected fault feature matrix and obtain the core fault feature matrix for low-rank reconstruction based on the singular value cumulative energy contribution rate. The fault determination module is used to select the fault template sub-library based on the sub-category and environmental disturbance indicators, and perform hierarchical determination based on overall matching, local matching and fault hazard level, and output the results of normal status, confirmed fault, unknown fault or pending manual review. The template library management module is used to perform regression testing and version release on manually reviewed samples in an offline environment.

20. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the method described in any one of claims 1-18.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-18.

Images