A power transmission equipment zero-sample defect detection method
By constructing a hierarchical text prompt lexicon for power transmission equipment and an adaptive alignment module for visual features, the problems of sample scarcity and feature drift for rare defects in power transmission equipment inspection were solved, achieving high-precision and low-cost defect detection for power transmission equipment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SKILL TRAINING CENT STATE GRID JIBEI ELECTRONICS POWER COMPANY
- Filing Date
- 2026-04-14
- Publication Date
- 2026-07-10
AI Technical Summary
Existing methods for detecting defects in power transmission equipment suffer from low accuracy and difficulty in meeting industry standards due to scarce samples, high annotation costs, and drift of general model features when faced with rare defects.
By constructing a hierarchical text prompt vocabulary and semantic feature space for power transmission equipment, a component perception and localization model based on YOLO is built, and a visual feature adaptive alignment module (Visual Adapter) is embedded. The model is trained using a contrastive alignment loss function to achieve alignment between visual features and text descriptions, thus solving the problem of visual feature misalignment.
It enables high-precision, low-cost inspection of power transmission equipment in complex environments, can identify rare defects, reduce system maintenance costs, and improve inspection accuracy and robustness.
Smart Images
Figure CN122368433A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a zero-sample defect detection method for power transmission equipment. Background Technology
[0002] With the widespread adoption of smart grids and drone inspections, a massive amount of images of power transmission lines have been collected. However, the fault patterns of power transmission equipment are complex, and some serious faults occur very infrequently, making it difficult to collect sufficient negative samples for training. Mainstream defect detection algorithms fall into two categories: one is fully supervised deep learning algorithms, which require pre-defined categories and extensive labeling, but struggle to handle rare defects, and lack sufficient negative samples for training when adding new categories, making them time-consuming and labor-intensive; the other is zero-shot detection algorithms based on general multimodal large models, which match images through text descriptions, but these models are trained on natural images from the internet, while power inspection images have unique perspectives, complex backgrounds, and varying distributions, making it difficult for the model to understand specialized features, resulting in a very high false alarm rate.
[0003] Zero-shot detection currently faces three major challenges: First, samples are scarce, defects are distributed in a "long tail" pattern, rare samples are insufficient, annotation relies on experts, which is costly and has a slow response time; second, features are misaligned, general models have difficulty distinguishing between normal and defective images, visual and semantic spaces are misaligned, and accuracy is insufficient; third, fine-tuning is difficult, full fine-tuning requires high computing power and is prone to overfitting, while terminal computing power is limited and difficult to adapt to actual needs.
[0004] Zero-shot learning based on vision-language pre-trained models guides recognition through a unified semantic space, eliminating reliance on labeled samples. However, in the power industry, domain drift and long-tail effects are significant, equipment features are subtle and specialized, and general-purpose model pre-training data, derived from natural images, lacks sufficient understanding of industrial scenarios. Even with strong model generalization capabilities, direct application still suffers from feature misalignment due to background interference and semantic differences, failing to achieve industrial-grade detection accuracy.
[0005] Existing fully supervised methods, such as the YOLO series, which are based on fully supervised learning, perform reasonably well on common defects, but they heavily rely on large-scale annotations, making it difficult to handle rare defects with scarce samples. Furthermore, adding new types requires retraining, resulting in high maintenance costs and slow response times. Meanwhile, directly applying general multimodal models like CLIP to attempt zero-shot identification using text descriptions, while theoretically supporting zero-shot identification, suffers from extremely low detection accuracy due to the significant differences in feature distribution between power scene and natural image, making it difficult for visual features to accurately match defect text and failing to meet industrial standards.
[0006] To address the aforementioned issues, two main technical approaches exist: one involves introducing fine-tuning techniques on top of a pre-trained model; the other involves introducing few-shot learning or pseudo-labeling techniques on top of a pre-trained model. Full fine-tuning involves retraining all parameters of a large model using domain-specific data, ensuring the model weights are fully adapted to the new dataset. For example, some existing technologies disclose a power equipment classification method based on the CLIP model, which iteratively updates the image encoder and text encoder of the CLIP model with all parameters using collected insulator image data. However, due to the massive number of parameters in deep learning models, the introduction of adjustments incurs high computational and storage costs. To address these issues, the academic community has published some methods based on cue word learning or few-shot adaptation. For instance, defect detection methods based on learnable cue words keep the model parameters frozen, adapting to the new task only by optimizing the input text cue vector. However, while cue word learning does not significantly increase computational and storage costs, it only optimizes on the text side, ignoring the feature distribution bias inherent in the visual side, and lacks an evaluation mechanism for the quality of generated virtual samples.
[0007] Therefore, directly matching misaligned image features with background noise with text will seriously threaten the accuracy of the detection results, leading to a high false alarm rate in complex backgrounds. Summary of the Invention
[0008] The purpose of this invention is to provide a zero-shot defect detection method for power transmission equipment. By separating different types of components from a complex background, the subsequent zero-shot model can focus on the fine-grained feature analysis of a single object, thereby solving the technical problem that general large models are prone to distraction and difficulty in distinguishing defects of different equipment in multi-object scenarios.
[0009] To achieve the above objectives, the present invention provides a zero-sample defect detection method for power transmission equipment, comprising the following steps: S1. Construct a hierarchical text prompt lexicon and semantic feature space for power transmission equipment; S2. Construct a component perception and localization model based on YOLO; S3. Build the Visual Adapter module for embedded visual feature adaptive alignment; S4. The contrastive alignment loss function is used as the training optimization objective to train the Visual Adapter module for adaptive alignment of visual features built in S3. S5. Collect the inspection image to be detected. Based on the category information output by the YOLO component perception and localization model, input the inspection image to be detected into the model to obtain sub-images with detection boxes that are separated from each other. S6. Input the cropped or labeled detection area into the CLIP model of VisualAdapter, which integrates visual feature adaptive alignment module, for recognition, and obtain the error category of the sample.
[0010] Preferably, S1 is as follows: We construct a hierarchical text description library that includes equipment-level and defect-level descriptions using prompt word engineering. For each target to be detected, we construct positive sample descriptions and negative sample descriptions respectively. We use the text encoder in the pre-trained multimodal large model to transform the positive sample descriptions and negative sample descriptions into text feature vectors. We construct a semantically rich and stable benchmark feature space to provide a semantic benchmark for zero-shot image-text matching.
[0011] Preferably, S2 is as follows: A pre-processor component-aware model is trained using the YOLO lightweight object detection network. The input image is I, and the model outputs a set of predicted bounding boxes for key components. During training, classification loss is used. and bounding box regression loss The weighted sum is used as the total loss function The network is optimized using the following formula: ; in, For classification loss weight parameters, These are the weighting coefficients for the bounding box regression loss; Among them, bounding box regression loss The CIoU loss is calculated using the following formula: ; In the formula, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Indicates the center point of the prediction box Center point of the real frame Euclidean distance; It is the diagonal distance of the smallest closure region that simultaneously contains both the predicted bounding box and the ground truth bounding box; For balancing parameters, Used to measure the consistency of aspect ratio.
[0012] Preferably, in S3, the visual feature adaptive alignment module Visual Adapter is embedded at the output of the pre-trained CLIP visual encoder. Let the original image features output by the visual encoder be... In the initial training phase, the alignment features are processed by the Adapter. The calculation process is as follows: ; in, For the dimension-reduced projection matrix, For the up-dimensional projection matrix; It is a non-linear activation function. A learnable scaling factor. This is an attention mask generated from the feature map; adding spatial attention enables the model to focus on the pixel region where the defect is located when aligning features.
[0013] Preferably, S4 is as follows: During training, the parameters of the original CLIP model's visual encoder and text encoder are frozen, and the dimensionality reduction projection matrix and dimensionality increase projection matrix in the Adapter module are updated. A contrastive alignment loss function is used as the optimization objective to force the part image features to match the correct text description. The loss function is calculated as follows: ; in, This represents the calculated alignment loss value; This represents the total number of samples in the current training batch; This represents the visual alignment feature vector after dimensionality reduction and dimensionality increase projection matrix processing by the Adapter module; This represents the text feature vector of the positive sample that matches the current component image; This represents the text feature vectors of all candidates in the current batch, including both positive and negative samples; The similarity measurement function is represented, preferably cosine similarity calculation; This represents the preset temperature hyperparameter, used to control the smoothness of the similarity distribution.
[0014] Preferably, S6 includes the following steps: S61. Based on the category information output by the YOLO-based component perception and localization model, the cropped sub-images are routed to the corresponding feature extraction channels. S62. After inputting these sub-images into the CLIP model with the Adapter module trained in S4, the corrected visual feature vectors are extracted. ; S63. Calculate the feature vector and the corresponding category of defective text feature vector. cosine similarity score The formula is as follows: ; in, Cosine similarity.
[0015] Preferably, the following steps are included after S63: S64. Threshold discrimination based on similarity score: If the similarity between the component sub-image and the defect text is the highest and exceeds a preset threshold... If so, the component is determined to have this type of defect; If the similarity score is less than If the condition is met, the target is either considered normal or ignored, and the current reasoning process ends. The S65 and CLIP models will identify images with defects exceeding a threshold and fuse the identified defect category information with the bounding box location information output by the YOLO model to generate a detection result image containing defect category, confidence level, and precise coordinates.
[0016] Therefore, the present invention employs the above-mentioned zero-sample defect detection method for power transmission equipment, and the beneficial effects are as follows: (1) To address the semantic interference problem caused by the coexistence of multiple targets such as insulators, vibration dampers, and tower materials in complex backgrounds of power inspection images, this invention introduces a component perception and localization model based on YOLO as a spatial decoupling and hard attention mechanism. Its purpose is not only to solve the problem that key components account for a very small proportion in high-resolution images, but also to decouple the complex full-image multi-label detection task into multiple high signal-to-noise ratio local sub-image single classification tasks.
[0017] (2) By separating different types of components from a complex background, the present invention enables the subsequent zero-shot model to focus on the fine-grained feature analysis of a single object, thereby solving the technical problem of general large models having scattered attention and difficulty in distinguishing defects of different devices in multi-object scenarios.
[0018] (3) In view of the problems of scarce rare defect samples, high cost of manual annotation and feature drift of general large models in the power industry, this invention introduces a visual feature adaptive alignment unit in the multimodal diagnostic module. This unit aims to map the visual features of specific power equipment to the general semantic space through self-supervised or weakly supervised learning without full fine-tuning and without destroying the generalization ability of the pre-trained model. In this way, it can directly realize zero-sample recognition of unknown and long-tail defects by using natural language description, completely solve the pain point of not being able to train the detection model due to lack of negative samples, and greatly reduce the maintenance cost and implementation threshold of the system.
[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0020] Figure 1 This is an overall flowchart of an embodiment of the zero-sample defect detection method for power transmission equipment according to the present invention; Figure 2 This is a schematic diagram of model training for an embodiment of the zero-sample defect detection method for power transmission equipment according to the present invention; Figure 3 This is a schematic diagram of the structure of the Visual Adapter, a visual feature adaptive alignment module, according to an embodiment of the zero-sample defect detection method for power transmission equipment of the present invention. Figure 4 This is a schematic diagram of a sample from an embodiment of the zero-sample defect detection method for power transmission equipment according to the present invention. Detailed Implementation
[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0022] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0023] like Figure 1 As shown, a zero-sample defect detection method for power transmission equipment includes the following steps: S1. Construct a hierarchical text prompt lexicon and semantic feature space for power transmission equipment, specifically: First, considering the diverse equipment types and complex fault modes in power transmission scenarios, a hierarchical text description library containing equipment-level and defect-level descriptions is constructed using prompt word engineering. For each target to be detected, positive and negative sample descriptions are constructed separately. Then, using the text encoder in a pre-trained multimodal large model, the positive and negative sample descriptions are transformed into high-dimensional text feature vectors. A semantically rich and stable benchmark feature space is constructed to provide a semantic benchmark for subsequent zero-shot image-text matching.
[0024] S2. Construct a YOLO-based component perception and localization model. This step aims to address the issue of excessively small proportions of relevant components in high-resolution inspection images. Specifically: A pre-processor component awareness model is trained using the YOLO lightweight object detection network. Input image I, model output a set of predicted bounding boxes for key components. During training, classification loss is used. and bounding box regression loss The weighted sum is used as the total loss function The network is optimized using the following formula: ; in, For classification loss weight parameters, These are the weighting coefficients for the bounding box regression loss; Among them, bounding box regression loss To improve positioning accuracy, CIoU (Complete IoU) loss is used, and the calculation formula is as follows: ; In the formula, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Indicates the center point of the prediction box Center point of the real frame Euclidean distance; It is the diagonal distance of the smallest closure region that simultaneously contains both the predicted bounding box and the ground truth bounding box; For balancing parameters, Used to measure the consistency of aspect ratio.
[0025] Through this model inference, the system outputs not only the bounding boxes of key components but also their category labels, which do not include defects. Based on this information, the system performs spatial decoupling, cropping component sub-image sequences without redundant backgrounds from the original image and constructing a dynamic sub-image queue. For components of different categories, such as insulators and vibration dampers, that may coexist in the same inspection image, the system marks them according to their category labels and crops them into images so that they can be routed to different defect diagnosis processes in subsequent steps.
[0026] S3, such as Figure 3 As shown, a Visual Adapter module with embedded visual features adaptive alignment is built.
[0027] To address the issues of "feature drift" in general large-scale models within the power industry context and the fact that defects in power equipment often occur locally in components, this invention designs a lightweight visual feature adaptive alignment module, VisualAdapter. This module is embedded at the output of a pre-trained CLIP visual encoder. Let the original image features output by the visual encoder be... In the initial stage of training, the scaling factor If initialized to a value close to 0, the alignment feature after processing by the Adapter will be... The calculation process is as follows: ; in, For the dimension-reduced projection matrix, For the up-dimensional projection matrix; It is a non-linear activation function. A learnable scaling factor. This is an attention mask generated from the feature map; adding spatial attention allows the model to focus on the pixel region where the defect is located when aligning features. The residual connection structure in this formula ensures that the model can retain its original general visual capabilities in the initial training phase, and then gradually learns power-specific features through training.
[0028] S4, such as Figure 2 As shown, the adaptive module collaborative training based on contrast alignment loss is as follows: The contrast alignment loss function is used as the training optimization objective. A small number of normal-state power transmission equipment samples or samples containing fundamental defects are used as the support set to train the visual feature adaptive alignment module Visual Adapter built with S3. Specifically: During training, the parameters of the original CLIP model's visual encoder and text encoder are frozen, and the dimensionality reduction projection matrix and dimensionality increase projection matrix in the Adapter module are updated. A contrastive alignment loss function is used as the optimization objective to force the part image features to match the correct text description. The loss function is calculated as follows: ; in, This represents the calculated alignment loss value; This represents the total number of samples in the current training batch; This represents the visual alignment feature vector after dimensionality reduction and dimensionality increase projection matrix processing by the Adapter module; The text feature vector represents a positive sample (correct description) that matches the current part image; This represents the text feature vectors of all candidates in the current batch, including both positive and negative samples; The similarity measurement function is represented, preferably cosine similarity calculation; This represents the preset temperature hyperparameter, used to control the smoothness of the similarity distribution.
[0029] S5. Collect the inspection image to be detected. Based on the category information output by the YOLO component perception and localization model, input the inspection image to be detected into the model to obtain sub-images with detection boxes that are separated from each other.
[0030] S6. Input the cropped or labeled detection region into the CLIP model, which incorporates the visual feature adaptive alignment module VisualAdapter, for identification, and derive the error category of the sample, including the following steps: S61. Multi-target parallel distribution: Based on the category information output by the YOLO component perception and localization model, the system routes the cropped sub-images to the corresponding feature extraction channels. For example, the "insulator sub-image" is sent to the channel matching "insulator defect prompt words", and the "vibration hammer image" is sent to the channel matching "vibration hammer defect prompt words".
[0031] S62. Feature Extraction and Diagnosis: After inputting these sub-images into the CLIP model with embedded modules trained in S4, the corrected visual feature vectors are extracted. .
[0032] S63. Result Calculation: Calculate the feature vector and the corresponding category of defective text feature vector. cosine similarity score This allows the system to independently determine the defect status of multiple heterogeneous devices in the image simultaneously during a single reasoning process, without interference between them. (Cosine similarity score) The calculation formula is as follows: ; in, Cosine similarity.
[0033] S64. The system performs threshold discrimination based on similarity scores: If a component sub-image has the highest similarity to a certain defect text and exceeds a preset threshold... If so, the component is determined to have this type of defect.
[0034] If the similarity score is less than If the condition is met, the target is either considered normal or ignored, and the current reasoning process ends.
[0035] The S65 and CLIP models will identify images with defects exceeding a threshold and fuse the identified defect category information with the bounding box location information output by the YOLO model to generate a detection result image containing defect category, confidence level, and precise coordinates.
[0036] Example 1: Step 1: Input the high-resolution original image of the drone inspection.
[0037] Image data captured during the inspection of power transmission lines is obtained. Considering that the actual inspection environment includes interference such as mountains, forests, and buildings, and that the image resolution is usually high and the image size is usually large, it is difficult to directly detect small defects. Therefore, it is used as the initial input of the system and will be processed in multiple stages.
[0038] Step 2: The YOLO component perception model is used for localization.
[0039] A pre-trained lightweight YOLO object detection network is used as a pre-localizer to perform inference on the input high-resolution original image. The training objective of the YOLO model is not to directly identify specific defects in power equipment, but to identify and locate key components of related precision equipment. This step outputs the bounding box coordinates of the key components in the original image.
[0040] Step 3: Crop the component sub-images based on the bounding box.
[0041] Based on the bounding box coordinates output in step 2, a sub-image sequence containing key components is cropped from the high-resolution original image. This step effectively filters out a large amount of irrelevant background noise, focusing subsequent diagnostic attention on the core component region and significantly improving the signal-to-noise ratio.
[0042] Step 4: CLIP visual encoder extracts raw features.
[0043] The cropped component sub-images are input into the pre-trained CLIP visual encoder. In this embodiment, the parameters of the CLIP visual encoder are frozen and do not participate in backpropagation updates, so as to preserve the generalized visual features learned on large-scale general datasets. Let the input sub-image be... After processing by the visual encoder, the original high-dimensional visual feature vector is output. .
[0044] Step 5: Visual Adapter module feature alignment.
[0045] This is the core step of the invention, which involves using the original visual features obtained in step 4. Input is fed into the lightweight visual adaptive alignment module. The Visual Adapter module's feature alignment employs a bottleneck residual architecture, the internal structure of which is shown in the attached figure. Figure 3 As shown, this enables the transfer of general visual features to the power industry feature space with minimal parameter cost.
[0046] Step 6: Calculate cosine similarity with text features.
[0047] Hierarchical text prompts are constructed, such as "device with surface cracks" or "normal device," and then input into a CLIP text encoder with frozen parameters to obtain text feature vectors. Calculate the aligned visual features. With text feature vectors The cosine similarity between them is calculated using the following formula: ; This similarity score represents the degree of semantic matching between the current image and the defect description.
[0048] Step 7: Threshold determination and result output.
[0049] Set a discrimination threshold The similarity score calculated in step 6 is compared with the threshold. Comparison: If the maximum similarity score is greater than If the similarity score is less than a certain value, the system determines that the component has a defect of the corresponding category, and outputs the final detection result, including the defect category and its corresponding location coordinates; if the similarity score is less than a certain value... If the result is not found, the target is either considered normal or ignored, and the current reasoning process ends. This process is repeated until the diagnosis of all component sub-images is completed.
[0050] To demonstrate the advantages of the zero-sample defect detection system for power transmission equipment based on component perception and visual feature alignment proposed in this invention, the experimental analysis setup is as follows: 1. Construct the dataset: like Figure 4 As shown, the power transmission inspection dataset used in this invention was collected from actual UAV inspection missions and includes key components such as insulators, vibration dampers, and grading rings. The dataset contains a total of 2500 images. To verify zero-shot capability, the dataset was divided into "visible class" and "unseen class". The test set contains various defect samples.
[0051] 2. Evaluation index formulation and model performance evaluation: This invention uses mean average precision (mAP), frames per second (FPS), and zero-shot recall as evaluation metrics. mAP@0.5 measures detection accuracy, and FPS measures inference speed. Furthermore, a zero-shot recall metric is introduced to specifically evaluate the model's ability to discover unseen defect categories.
[0052] The system proposed in this invention is compared with current mainstream detection methods, including: (1) the traditional fully supervised method YOLOv8 (requiring full annotation); (2) the original CLIP model (without an adapter, direct inference); and (3) the method of this application (YOLO+Adapter+CLIP). The experimental results are shown in Table 1.
[0053] Table 1 Comparison of detection performance of different models
[0054] The results are shown in Table 1. The original CLIP model suffers from severe feature drift and an inability to focus on small targets, resulting in an mAP of only 42.3%, which is insufficient for practical applications. In contrast, this invention introduces a YOLO component awareness and VisualAdapter alignment module: the mAP reaches 85.7%, approaching that of a fully supervised YOLOv8 model, demonstrating that the Adapter module effectively solves the problem of difficulty in identifying power equipment. Even without having seen certain defective samples, the zero-sample recall rate reaches 82.4%, proving the system's strong generalization ability.
[0055] Furthermore, thanks to the "component perception-spatial decoupling" architecture proposed in this invention, the system can simultaneously and accurately locate and identify various types of defects such as insulator damage and anti-vibration hammer slippage in a high-resolution inspection map containing complex line structures, thus achieving an efficient inspection mode.
[0056] Therefore, the present invention adopts the above-mentioned zero-sample defect detection method for power transmission equipment, which significantly reduces the cost of data annotation and achieves high-precision and robust detection of rare defects in power transmission equipment, solving the core pain points of the scarcity of long-tail samples and the incompatibility of general models in the prior art.
[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for zero-sample defect detection of power transmission equipment, characterized in that, Includes the following steps: S1. Construct a hierarchical text prompt lexicon and semantic feature space for power transmission equipment; S2. Construct a component perception and localization model based on YOLO; S3. Build the Visual Adapter module for embedded visual feature adaptive alignment; S4. The contrastive alignment loss function is used as the training optimization objective to train the Visual Adapter module for adaptive alignment of visual features built in S3. S5. Collect the inspection image to be detected. Based on the category information output by the YOLO component perception and localization model, input the inspection image to be detected into the model to obtain sub-images with detection boxes that are separated from each other. S6. Input the cropped or labeled detection area into the CLIP model of VisualAdapter, which integrates visual feature adaptive alignment module, for recognition, and obtain the error category of the sample.
2. The method for zero-sample defect detection of power transmission equipment according to claim 1, characterized in that, S1 specifically refers to: We construct a hierarchical text description library that includes equipment-level and defect-level descriptions using prompt word engineering. For each target to be detected, we construct positive sample descriptions and negative sample descriptions respectively. We use the text encoder in the pre-trained multimodal large model to transform the positive sample descriptions and negative sample descriptions into text feature vectors. We construct a semantically rich and stable benchmark feature space to provide a semantic benchmark for zero-shot image-text matching.
3. The method for zero-sample defect detection of power transmission equipment according to claim 2, characterized in that, S2 specifically refers to: A pre-processor component-aware model is trained using the YOLO lightweight object detection network. The input image is I, and the model outputs a set of predicted bounding boxes for key components. During training, classification loss is used. and bounding box regression loss The weighted sum is used as the total loss function. The network is optimized using the following formula: ； in, For classification loss weight parameters, These are the weighting coefficients for the bounding box regression loss; Among them, bounding box regression loss The CIoU loss is calculated using the following formula: ； In the formula, This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. Indicates the center point of the prediction box Center point of the real frame Euclidean distance; It is the diagonal distance of the smallest closure region that simultaneously contains both the predicted bounding box and the ground truth bounding box; For balancing parameters, Used to measure the consistency of aspect ratio.
4. The method for zero-sample defect detection of power transmission equipment according to claim 3, characterized in that, In S3, the Visual Adapter, a visual feature adaptive alignment module, is embedded at the output of the pre-trained CLIP visual encoder. Let the original image features output by the visual encoder be... In the initial training phase, the alignment features are processed by the Adapter. The calculation process is as follows: ； in, For the dimension-reduced projection matrix, For the up-dimensional projection matrix; It is a non-linear activation function. A learnable scaling factor. This is an attention mask generated from the feature map; adding spatial attention enables the model to focus on the pixel region where the defect is located when aligning features.
5. The method for zero-sample defect detection of power transmission equipment according to claim 4, characterized in that, S4 specifically refers to: During training, the parameters of the original CLIP model's visual encoder and text encoder are frozen, and the dimensionality reduction projection matrix and dimensionality increase projection matrix in the Adapter module are updated. A contrastive alignment loss function is used as the optimization objective to force the part image features to match the correct text description. The loss function is calculated as follows: ； in, This represents the calculated alignment loss value; This represents the total number of samples in the current training batch; This represents the visual alignment feature vector after dimensionality reduction and dimensionality increase projection matrix processing by the Adapter module; This represents the text feature vector of the positive sample that matches the current component image; This represents the text feature vectors of all candidates in the current batch, including both positive and negative samples; The similarity measurement function is represented, preferably cosine similarity calculation; This represents the preset temperature hyperparameter, used to control the smoothness of the similarity distribution.
6. The method for zero-sample defect detection of power transmission equipment according to claim 5, characterized in that, S6 includes the following steps: S61. Based on the category information output by the YOLO-based component perception and localization model, the cropped sub-images are routed to the corresponding feature extraction channels. S62. After inputting these sub-images into the CLIP model with the Adapter module trained in S4, the corrected visual feature vectors are extracted. ; S63. Calculate the feature vector and the corresponding category of defective text feature vector. cosine similarity score The formula is as follows: ； in, Cosine similarity.
7. The method for zero-sample defect detection of power transmission equipment according to claim 6, characterized in that, S63 is followed by the following steps: S64. Threshold discrimination based on similarity score: If the similarity between the component sub-image and the defect text is the highest and exceeds a preset threshold... If so, the component is determined to have this type of defect; If the similarity score is less than If the condition is met, the target is either considered normal or ignored, and the current reasoning process ends. The S65 and CLIP models will identify images with defects exceeding a threshold and fuse the identified defect category information with the bounding box location information output by the YOLO model to generate a detection result image containing defect category, confidence level, and precise coordinates.