Power transmission line small target detection method based on cross-modal feature enhancement under small sample condition

Abstract

The application provides a power transmission line small target detection method based on cross-modal feature enhancement under a small sample condition, a power transmission line small sample target data set is constructed by collecting power transmission line aerial images and labeling; a target detection model based on Florence-2 and cross-modal feature enhancement is constructed, including a feature enhancement module, a visual feature enhancement module and a large-scale visual language model Florence-2; a dynamic boundary regression loss L DBR The boundary box regression weight is adaptively adjusted according to the target positioning difficulty and the category rarity, a mixed loss function is designed, and a final small sample target detection model is output after fine-tuning; the detection result is obtained by inputting the power transmission line aerial image to be detected into the final small sample target detection model; the method can realize accurate identification of small targets, i.e., key power devices, in the power transmission line, and realize inter-class distinguishability of the target and accurate positioning of the small target under the small sample condition.

Description

Technical Field

[0001] This invention relates to a method for detecting small targets in power transmission lines based on cross-modal feature enhancement under small sample conditions, belonging to the field of power system target detection technology. Background Technology

[0002] The power system is a core infrastructure supporting the stable operation of modern society. With the ever-expanding scale of transmission lines, the importance of their safe and stable operation is becoming increasingly prominent. Traditional transmission line inspection is inefficient, costly, and unsafe. In recent years, intelligent inspection technology using UAVs (Unmanned Aerial Vehicles) has been widely applied. Among these technologies, target detection technology is the core technology for quickly and accurately identifying key power components such as insulators, vibration dampers, and bolts.

[0003] Current research on target detection for key components in power transmission lines mainly falls into two categories: methods based on traditional convolutional neural networks (CNNs) and methods based on visual language models. Methods based on traditional CNNs (such as improved Faster R-CNN and YOLO series) focus on feature extraction and fusion across visual modalities. They enhance multi-scale feature extraction capabilities by introducing attention mechanisms (such as global attention mechanisms) and optimizing the backbone network or feature pyramid network structure. Methods based on visual language models leverage the powerful general visual representation capabilities of large-scale pre-trained models, treating target detection as a cross-modal sequence-to-sequence (Seq2Seq) generation task. This involves inputting an image and task prompts, directly generating structured text containing target categories and precise coordinates, thus achieving a unified output of the detection results and their bounding boxes. Existing methods currently suffer from the following problems:

[0004] 1. Traditional convolutional neural network methods typically require a large amount of labeled data for training to achieve good performance. In scenarios where data is scarce or labeling costs are high, the model's generalization ability is limited. Traditional methods focus on optimizing and adjusting the structure of the backbone network or feature pyramid network. When the number of target samples is too small, the model struggles to extract robust and discriminative target features from a limited amount of data, and is easily identified as common categories with a large number of samples.

[0005] 2. Large-scale visual language models have powerful general feature extraction capabilities, but when faced with severe data imbalance scenarios such as power transmission lines, existing visual language model-based methods have not yet found an optimization approach, and the accuracy of target recognition and positioning precision still need to be improved.

[0006] 3. In the inspection of power transmission lines, the targets such as bolts, anti-vibration hammers, and insulators have few pixels and their features are difficult to extract when captured in long-distance, large-scene photography. This makes it difficult to accurately regress the target bounding box to its true boundary. Existing methods still need to be improved in terms of accurate positioning of small target boundaries. Summary of the Invention

[0007] The purpose of this invention is to provide a method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions, which solves the problems of insufficient feature learning and inaccurate boundary localization of small targets under small sample conditions in the existing technology, and the need to improve the accuracy of target recognition and localization.

[0008] The technical solution of this invention is:

[0009] A method for detecting small targets on transmission lines based on cross-modal feature enhancement under small sample conditions includes the following steps:

[0010] Step 1: Collect aerial images of power transmission lines and label the target categories to construct a small sample target dataset of power transmission lines;

[0011] Step 2: Construct an object detection model based on Florence-2 and cross-modal feature enhancement. This model includes a feature enhancement module, a visual feature enhancement module, and the large-scale visual language model Florence-2. The feature enhancement module calculates the cosine similarity between multi-sample category images and the input few-sample category text, selects the multi-sample category images most semantically similar to the few-sample category text, and transforms them into a knowledge vector V. knowledge The large-scale visual language model Florence-2 includes a first visual encoder, a text encoder, and a multimodal encoder-decoder. The first visual encoder extracts the raw visual features F from the input aerial images of power transmission lines. raw By using a hook mechanism registered in the output layer of the first visual encoder, the knowledge vector V is transferred by the visual feature enhancement module. knowledge Injected and fused into the original visual feature F raw In the process, enhanced visual features F are generated. enhanced Simultaneously, the task instruction text is transformed into text features F by a text encoder. text Next, the enhanced visual feature F enhanced and text features F text The multimodal features are spliced ​​and fused together to form multimodal features. These multimodal features, together with the generated time-stamped sequence, are input into the multimodal encoder-decoder. Through cross-modal interaction and autoregressive prediction, the next target label is generated, and finally a structured text sequence is output to achieve target detection and localization.

[0012] Step 3: Introduce dynamic boundary regression loss L DBR Based on the difficulty of target localization and the rarity of the category, the bounding box regression weights are adaptively adjusted, and a hybrid loss function is designed.

[0013] Step 4: Use the low-rank adaptation method LoRA and a hybrid loss function to fine-tune the target detection model based on Florence-2 and cross-modal feature enhancement, and then output the final small-sample target detection model.

[0014] Step 5: Input the aerial image of the transmission line to be detected into the final small sample target detection model to obtain the detection results.

[0015] Furthermore, in step 2, "few-sample class text" refers to the target class whose sample number in the small-sample target dataset of transmission lines is less than a set number; "multi-sample class image" refers to the original image of the target class whose sample number in the small-sample target dataset of transmission lines is greater than a set value.

[0016] Furthermore, in the target detection model based on Florence-2 and cross-modal feature enhancement, the feature enhancement module calculates the cosine similarity between the multi-sample class image and the input few-sample class text, selects the multi-sample class image that is semantically most similar to the few-sample class text, and transforms it into a knowledge vector V. knowledge Specifically,

[0017] Step 21: Extract multi-sample image features from multi-sample category images using the contrastive language-image pre-trained model CLIP;

[0018] Step 22: Extract the text features of the category labels of the few-sample class texts by using the contrastive language-image pre-trained model CLIP to obtain the few-sample text features;

[0019] Step 23: Calculate the cosine similarity between the few-sample text features and the many-sample image features, and select the top N many-sample images that are most semantically similar to the few-sample text features based on the similarity scores as the selected many-sample images. Then, use the second visual encoder to extract visual image features from the selected many-sample images.

[0020] Step 24: Normalize the top N similarity scores using Softmax, introduce a temperature parameter to control the distribution, obtain weights, and then perform a weighted summation of the visual image features to generate a knowledge vector V. knowledge .

[0021] Furthermore, in the target detection model based on Florence-2 and cross-modal feature enhancement, the visual feature enhancement module will integrate the knowledge vector V... knowledge Injected and fused into the original visual feature F raw In the process, enhanced visual features F are generated. enhanced Specifically, ,in, For weight fusion.

[0022] Furthermore, in step 3, the hybrid loss function Including basic losses and dynamic boundary regression loss : ,in, This is the loss balance weighting coefficient.

[0023] Furthermore, the basic loss adopts the cross-entropy loss function, and the basic loss... :

[0024]

[0025] in, Network parameters representing a target detection model based on Florence-2 and cross-modal feature enhancement; It is the total length of the structured text sequence; This indicates that the target detection model based on Florence-2 and cross-modal feature enhancement is based on multimodal features x and the generated target label sequence. Predict the next target marker The conditional probability is given by the multimodal input x, which is a combination of image features and task instructions.

[0026] Furthermore, in step 3, the dynamic boundary regression loss L DBR :

[0027]

[0028] in, This represents the number of bounding box pairs matched in the current batch. Let i be the IoU loss for the i-th matching pair. The dynamic weights for the i-th matching pair are: The aim is to adaptively adjust the contribution of the bounding box loss based on the difficulty of target localization and the rarity of the category.

[0029] Furthermore, dynamic weights Among them, the positioning difficulty weight Rarity weight ,in, To focus on hyperparameters, used to adjust the intensity of attention to difficult-to-locate targets; This represents the CIoU loss value of the i-th target to be detected in the current input image. , The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. It is a constant. Let be the number of instances in the category to which the i-th sample belongs. γ represents the total number of instances across all categories, and γ is a rarity weight hyperparameter used to adjust the intensity of attention given to smaller sample categories.

[0030] The beneficial effects of this invention are: this method for detecting small targets on transmission lines based on cross-modal feature enhancement under small sample conditions, by using the large-scale visual language model Florence-2 as the basic framework, introduces the feature enhancement module FEM and the dynamic boundary regression loss L... DBR It can accurately identify small targets, i.e. key power devices, in power transmission lines, and achieve inter-class differentiation and precise positioning of small targets under small sample conditions. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions according to an embodiment of the present invention.

[0032] Figure 2 This is an illustrative diagram illustrating the target detection model based on Florence-2 and cross-modal feature enhancement in the embodiment;

[0033] Figure 3 This is a schematic diagram illustrating the feature enhancement module in the embodiment;

[0034] Figure 4 This is a schematic diagram of the small target detection method for transmission lines based on cross-modal feature enhancement under small sample conditions, the original model, and the detection results of different combinations.

[0035] Figure 5 This is a schematic diagram comparing the detection results of the small target detection method for transmission lines based on cross-modal feature enhancement under small sample conditions with the Mask-RCNN method, Faster-RCNN method, and Florence2 method. Detailed Implementation

[0036] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] Traditional target detection methods for power transmission lines, when dealing with critical power components, are often limited to visual modality feature enhancement frameworks, making it difficult to extract robust and discriminative target features from small sample data. Furthermore, existing methods offer limited improvement in boundary localization accuracy for small targets such as bolts and vibration dampers, failing to meet high-standard inspection requirements. This invention designs and implements a small-sample target detection method for power transmission lines based on cross-modal feature enhancement under small sample conditions. This method uses the large-scale visual language model Florence-2 as its foundation framework, designing a feature enhancement module and a dynamic boundary regression loss function. The feature enhancement module utilizes cross-modal knowledge to selectively supplement the visual features of small samples, improving inter-class discriminability and robustness; the dynamic boundary regression loss function effectively enhances the localization accuracy of small-sized targets by optimizing the bounding box regression process. Ultimately, this invention achieves a synergistic improvement in the inter-class discriminability of small-sample target features and the accuracy of small target boundary localization, providing high reliability and practical value for intelligent inspection of power transmission lines and advancing technological development in the field of power transmission line safety inspection.

[0038] The embodiment provides a method for detecting small targets on transmission lines based on cross-modal feature enhancement under small sample conditions, such as... Figure 1 This includes the following steps:

[0039] Step 1: Collect aerial images of power transmission lines and annotate them to construct a small sample target dataset of power transmission lines.

[0040] Step 2: Construct an object detection model based on Florence-2 and cross-modal feature enhancement. This model includes a feature enhancement module, a visual feature enhancement module, and the large-scale visual language model Florence-2. The feature enhancement module calculates the cosine similarity between multi-sample category images and the input few-sample category text, selects the multi-sample category images most semantically similar to the few-sample category text, and transforms them into a knowledge vector V. knowledge The large-scale visual language model Florence-2 includes a first visual encoder, a text encoder, and a multimodal encoder-decoder. The first visual encoder extracts the raw visual features F from the input aerial images of power transmission lines. raw By using a hook mechanism registered in the output layer of the first visual encoder, the knowledge vector V is transferred by the visual feature enhancement module. knowledge Injected and fused into the original visual feature F raw In the process, enhanced visual features F are generated. enhanced Simultaneously, the task instruction text is transformed into text features F by a text encoder. text Next, the enhanced visual feature F enhanced and text features F textThe multimodal features are spliced ​​and fused together to form multimodal features. These multimodal features, together with the generated time-stamped sequence, are input into a multimodal encoder-decoder. Through cross-modal interaction and autoregressive prediction, the next target label is generated, and finally a structured text sequence is output to achieve target detection and localization.

[0041] In step 2, "few-sample class text" refers to the target class whose sample number in the small-sample target dataset of transmission lines is less than a set number; "multi-sample class image" refers to the original image of the target class whose sample number in the small-sample target dataset of transmission lines is greater than a set value.

[0042] The feature enhancement module aims to address the imbalanced sample problem by extracting semantic and visual knowledge from multi-sample categories and injecting it into the features of the few-sample category through feature transfer. Specifically, the module calculates the cosine similarity between the multi-sample category images and the input few-sample category text, selects the multi-sample category images that are semantically most similar to the few-sample category text, and transforms them into a knowledge vector V. knowledge Specifically, such as Figure 3 :

[0043] Step 21: Extract multi-sample image features from multi-sample category images using the contrastive language-image pre-trained model CLIP;

[0044] Step 22: Extract the text features of the category labels of the few-sample class texts by using the contrastive language-image pre-trained model CLIP to obtain the few-sample text features;

[0045] Step 23: Calculate the cosine similarity between the few-sample text features and the many-sample image features, and select the top N many-sample images that are most semantically similar to the few-sample text features based on the similarity scores as the selected many-sample images. Then, use the second visual encoder to extract visual image features from the selected many-sample images.

[0046] Step 24: Normalize the top N similarity scores using Softmax, introduce a temperature parameter to control the distribution, obtain weights, and then perform a weighted summation of the visual image features to generate a knowledge vector V. knowledge .

[0047] All generated knowledge vectors will be stored in an enhanced feature dictionary for subsequent training phases. This cross-modal retrieval and weighting process leverages the semantic alignment capabilities of the contrastive language-image pre-trained model CLIP, ensuring that the multi-sample class features fused in the knowledge vectors are highly semantically correlated with the few-sample class features. The formula for generating knowledge vectors is shown below:

[0048]

[0049] During the forward propagation of model training, knowledge is injected into the feature stream through a hook mechanism. We first register an 'Enhanced Feature Hook' at the output layer of the DaViT visual encoder. When the visual encoder outputs raw features, the hook mechanism intercepts those features and looks up the corresponding knowledge vector from the dictionary based on the image name of the current batch.

[0050] The visual feature enhancement module uses a weighted summation method to fuse the data, controlling the intensity of knowledge injection, and generating enhanced visual features F. enhanced Through this feature-level knowledge transfer and injection, the few-sample class acquires semantic priors and high-quality visual features provided by the many-sample class during training, significantly enhancing its perceptual capabilities. This strategy effectively improves the detection performance of few samples without altering the original data distribution, thus strongly demonstrating its targeted adaptability to imbalanced scenarios. Fusion formula: , where α is the fusion weight.

[0051] In step 2, a basic object detection network is built based on Florence-2, and a feature enhancement module is designed. Prior knowledge from the contrastive language-image pre-trained model CLIP is used to generate knowledge vectors that are injected into the visual feature stream. This invention uses the large-scale visual language model Florence-2 as the basic network architecture and integrates a feature enhancement module on top of the Florence-2 basic model to achieve end-to-end detection from image to structured text output.

[0052] Step 3: Introduce dynamic boundary regression loss The bounding box regression weights are adaptively adjusted based on the difficulty of target localization and the rarity of the category, and a hybrid loss function is designed.

[0053] In step 3, the hybrid loss function Including basic losses and dynamic boundary regression loss : ,in, This is the loss balance weighting coefficient.

[0054] The base loss employs the cross-entropy loss function, which is fundamentally derived from the sequence-to-sequence training paradigm of large-scale visual language models. It aims to maximize the model's predicted probability for each token in the target output sequence, thereby training the model's text generation capabilities. Base Loss :

[0055]

[0056] in, Network parameters representing a target detection model based on Florence-2 and cross-modal feature enhancement; It is the total length of the structured text sequence; This indicates that the target detection model based on Florence-2 and cross-modal feature enhancement is based on multimodal features x and the generated target label sequence. Predict the next target marker The conditional probability is given by the multimodal input x, which is a combination of image features and task instructions.

[0057] This loss function can effectively optimize the model's category prediction accuracy and the consistency of text sequence generation, providing a basic guarantee for sequence generation capabilities for object detection tasks.

[0058] Dynamic boundary regression loss To address the imbalanced sample problem, a dynamic weighting mechanism is introduced, which automatically tilts the model towards classes with smaller sample sizes and hard-to-locate targets during training, thereby improving the detection accuracy of these key targets. Dynamic boundary regression loss is used. :

[0059]

[0060] in, This represents the number of bounding box pairs matched in the current batch. Let i be the IoU loss for the i-th matching pair. The dynamic weights for the i-th matching pair are: The aim is to adaptively adjust the contribution of the bounding box loss based on the difficulty of target localization and the rarity of the category.

[0061] Dynamic weights Among them, the positioning difficulty weight Rarity weight ,in, To focus on hyperparameters, used to adjust the intensity of attention to difficult-to-locate targets; This represents the CIoU loss value of the i-th target to be detected in the current input image. , The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. It is a constant, taking extremely small values, used to avoid when... When = 0, the model exhibits numerical instability or gradient calculation errors during backpropagation. Let be the number of instances in the category to which the i-th sample belongs. γ represents the total number of instances across all categories, and γ is a rarity weight hyperparameter used to adjust the intensity of attention given to smaller sample categories. It is based on CIoU loss calculation and is used to measure the difficulty of target localization; the higher the localization difficulty, the greater the weight. (Sparse weight) Used to solve the problem of imbalanced samples. Due to sparse weighting. It relies solely on the global class instance count of the dataset, which is a fixed value that remains unchanged during training, and assigns a higher loss contribution to smaller class samples.

[0062] In step 3, to address the problem of insufficient adaptation of traditional loss functions to imbalanced sample scenarios, this invention designs a hybrid loss function system consisting of basic loss and dynamic boundary regression loss, which enhances training for small sample categories and difficult-to-locate targets through dynamic weight allocation.

[0063] Step 4: Use the low-rank adaptation method LoRA and a hybrid loss function to fine-tune the target detection model based on Florence-2 and cross-modal feature enhancement, and then output the final small-sample target detection model.

[0064] In step 4, to maintain the powerful generalization ability of the Florence-2 large-scale visual language model while efficiently and quickly adapting it to the specific scenario of power transmission line target detection, this invention employs the low-rank adaptation technique from the parameter-efficient fine-tuning method for model fine-tuning. The core idea of ​​the low-rank adaptation technique is that when a large pre-trained model adapts to a new task, its weight matrix... Update volume It inherently possesses a low intrinsic rank. Therefore, compared to its fully trained... Instead, we can constrain it to a low-rank factorization form: Among them, the low-rank matrix and low-rank matrix , while the rank r is much smaller Where d is the output dimension of the weight matrix update ΔW, and k is the input dimension of the weight matrix update ΔW. During fine-tuning, the model's weight matrix... Keep the system frozen and optimize and update only the parameters in the two low-rank matrices A and B. Using this method, what would normally require training... A matrix with parameters, now only training is needed. This reduces the number of trainable parameters significantly.

[0065] Step 5: Input the aerial image of the transmission line to be detected into the final small sample target detection model to obtain the detection results.

[0066] This method for detecting small targets on transmission lines under small sample conditions, based on cross-modal feature enhancement, introduces a Feature Enhancement Module (FEM) and a Dynamic Boundary Regression Loss (DBRS) based on the large-scale visual language model Florence-2. It can accurately identify small targets, i.e. key power devices, in power transmission lines, and achieve inter-class differentiation and precise positioning of small targets under small sample conditions.

[0067] The experimental verification of the small target detection method for transmission lines based on cross-modal feature enhancement under the small sample condition in this embodiment is as follows:

[0068] (1) Self-built dataset and experimental parameter settings

[0069] We collected actual aerial inspection data and publicly available power datasets containing aerial images of power transmission lines to construct a high-resolution, multi-angle, multi-distance, and multi-weather target detection dataset for power transmission lines. The dataset has high environmental complexity, covering scene types such as urban areas, rural areas, mountains, forests, and extreme weather, to ensure that the model has robust learning capabilities for complex inspection scenarios.

[0070] The dataset consists of 1680 images, with 1480 images in the training set and 208 images in the test set. The small sample targets include three categories: insulators, dampers, and pins. The number of instances in each category is shown in Table 1.

[0071] Table 1. Number of examples for each target category

[0072]

[0073] The experimental environment consisted of Python 3.10, PyTotch 2.6.0, and CUDA 12.4, with an NVIDIA RTX 4090 GPU used for training and testing. The core training parameters were configured as follows: initial learning rate of 5×10⁻⁶, AdamW optimizer, weight decay of 0.01, linear learning rate scheduler, 20 total iterations, batch size of 4, LoRA Rank of 8, LoRA Alpha of 16, and LoRA Dropout of 0.05.

[0074] Evaluation metrics: ① mAP@0.5 refers to the average precision AP calculated for all classes when the IoU threshold is set to 0.5, and then the AP of all classes is averaged. ② Precision represents the proportion of correctly predicted positive samples out of all predicted positive samples. Higher precision means fewer false positives. ③ Recall represents the proportion of correctly predicted positive samples out of all actual positive samples. Higher recall means fewer missed detections. ④ F1 score is the harmonic mean of precision and recall, which measures the overall performance of the model.

[0075] (2) Ablation experiment verification of the present invention

[0076] Ablation experiments were conducted to fully verify the independent effectiveness and synergistic effect of the two core innovations proposed in this invention—the feature enhancement module and the dynamic boundary regression loss function—in the small sample target detection task of transmission lines.

[0077] The original Florence-2 visual language model was used as the baseline model, and validation was performed on a small sample dataset of transmission lines. The experiment was designed with the following three progressive combinations:

[0078] Combination 1: Introduce the cross-modal feature enhancement module only in the baseline model.

[0079] Combination 2: Introduce dynamic boundary regression loss only in the baseline model.

[0080] Combination 3: Integrating the feature enhancement module and dynamic boundary regression loss into the baseline model simultaneously (i.e., the final technical solution of this invention).

[0081] The ablation test results are shown in Table 2.

[0082] Table 2 Ablation Experiment Results

[0083]

[0084] The experimental results in Table 2 show that Combination 1 achieves a significant improvement in overall performance compared to the baseline model, with mAP@0.5 increasing to 0.7651, a relative improvement of 2.57%. This result proves the effectiveness of the feature enhancement module, which successfully utilizes the textual semantic information of CLIP through cross-modal knowledge injection, compensating for insufficient feature learning caused by data sparsity in small samples, thereby significantly enhancing the inter-class discriminability and robustness of small sample features. The results of Combination 2 show that the model achieves significant optimization in boundary localization accuracy, with mAP@0.5 jumping to 0.7809, an improvement of 4.15% relative to the baseline. This proves the effectiveness of the dynamic boundary regression loss, which effectively directs training attention to small targets that are difficult to locate by introducing a dynamic mechanism of localization difficulty weight and class sparseness weight, successfully improving the accuracy and stability of bounding box regression. Combination 3 of the implementation method achieves the best performance, with the final model's mAP@0.5 reaching 0.8042, achieving an overall performance improvement of 8.76% compared to the baseline model. This optimal result strongly demonstrates the positive synergistic effect between FEM and the dynamic boundary regression loss function. Specifically, the feature enhancement module provides robust features with high discriminative power, while the dynamic boundary regression loss function supervises the high-precision localization of these features. Together, they achieve joint optimization of robust learning and high-precision localization for small-sample features.

[0085] (3) Comparison experiment on the advancement of the invention

[0086] The proposed method is compared with mainstream algorithms in the fields of power transmission line inspection and general target detection. The comparison algorithms include classic two-stage detectors such as Mask R-CNN and Faster R-CNN, as well as the original Florence-2 benchmark model. All methods are fairly tested on a constructed small-sample dataset of power transmission lines. The experimental results are shown in Table 3.

[0087] Table 3 Comparison of results from different algorithms

[0088]

[0089] The comparative experimental results in Table 3 fully demonstrate the significant technical advantages of the method of this invention. In terms of overall performance, the method of this invention achieves an mAP@0.5 of 0.8042, significantly outperforming Mask R-CNN (mAP@0.5=0.7358) and Faster R-CNN (mAP@0.5=0.6627), proving the superiority of the visual language model-based architecture and cross-modal augmentation. Most importantly, the method of this invention achieves a key breakthrough in solving the problem of small sample detection: for the category of vibration dampers with the fewest samples in the dataset, its AP value is improved by 6.09% compared to the baseline model, and its F1 score reaches as high as 0.9636. Furthermore, for small target bolts, which are extremely difficult to detect, the method of this invention also achieves high-precision positioning, with its AP value improved by 16.27% compared to the original Florence-2. In summary, this invention demonstrates significant advantages and higher reliability in overall performance and detection accuracy for rare, small targets on transmission lines, reaching an industry-leading level and possessing great practical value. Comparative experiments have verified the advanced nature of this invention.

[0090] (4) Display of detection results for different combinations

[0091] Different combination detection results, such as Figure 4 As shown in the second row, the original model and the combined model 2 misdetect the debris in front of the house as a vibration damper. However, the combined models 1 and 3, which incorporate the feature enhancement module, successfully eliminated this misdetection, demonstrating that the feature enhancement module improves target discrimination. In the third row, the original model and the combined model 1 misdetect a single vibration damper as two. However, the combined models 1 and 3, which incorporate the dynamic boundary regression loss, can accurately identify a single vibration damper, indicating that the dynamic boundary regression loss contributes to the stability and accuracy of bounding box regression. In the fourth row, only the combined model 3, which integrates the feature enhancement module and the dynamic boundary regression loss, detects the small target vibration damper; the other models fail to detect it. This strongly demonstrates the crucial role of the synergistic optimization of the feature enhancement module and the dynamic boundary regression loss in solving the problem of detecting small-sample, difficult-to-localize targets.

[0092] (5) Display of test results

[0093] Test results as follows Figure 5 As shown, Figure 5The first and second rows illustrate typical scenarios where small-sample targets (such as vibration dampers) are partially obscured by poles or trees due to aerial viewing limitations. Under these extreme conditions of weakened features, traditional models often fail to establish reliable feature representations, leading to missed target detections. In the results shown in the second row, only the algorithm proposed in this invention successfully detected the vibration damper target obscured by poles. This result strongly demonstrates that cross-modal knowledge injection effectively enhances the feature robustness of small-sample categories, enabling them to overcome information loss caused by target occlusion and achieve accurate identification.

[0094] Figure 5 The third to sixth lines of the diagram illustrate how, in complex environments such as farmland, the target features of vibration dampers and insulators are easily confused with the background (such as crops and road surfaces), severely impacting recognition accuracy. Through comparison... Figure 4 The relevant detection results show that the algorithm proposed in this invention exhibits the highest robustness in dealing with background interference, successfully identifying vibration dampers and insulators affected by complex backgrounds. In contrast, Mask R-CNN is most severely affected by background interference and fails to identify vibration dampers and insulators at all. Although Faster R-CNN and the original Florence-2 model can detect some vibration dampers, there are still significant missed detections, and the completeness and stability of the detection are poor. This further confirms that the cross-modal knowledge injection strategy can effectively improve the inter-class discrimination of small sample features, reduce background confusion, and ensure the accuracy and high recall of detection.

[0095] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting small targets on transmission lines based on cross-modal feature enhancement under small sample conditions, characterized in that: Includes the following steps, Step 1: Collect aerial images of power transmission lines and label the target categories to construct a small sample target dataset for power transmission lines; Step 2: Construct an object detection model based on Florence-2 and cross-modal feature enhancement. This model includes a feature enhancement module, a visual feature enhancement module, and the large-scale visual language model Florence-2. The feature enhancement module calculates the cosine similarity between multi-sample category images and the input few-sample category text, selects the multi-sample category images most semantically similar to the few-sample category text, and transforms them into a knowledge vector V. knowledge The large-scale visual language model Florence-2 includes a first visual encoder, a text encoder, and a multimodal encoder-decoder. The first visual encoder extracts the raw visual features F from the input aerial images of power transmission lines. raw By using a hook mechanism registered at the output layer of the first visual encoder, the knowledge vector V is transferred by the visual feature enhancement module. knowledge Injected and fused into the original visual feature F raw In the process, enhanced visual features F are generated. enhanced Simultaneously, the task instruction text is transformed into text features F by a text encoder. text Next, the enhanced visual feature F enhanced and text features F text The multimodal features are spliced ​​and fused together to form multimodal features. These multimodal features, together with the generated time-stamped sequence, are input into the multimodal encoder-decoder. Through cross-modal interaction and autoregressive prediction, the next target label is generated, and finally a structured text sequence is output to achieve target detection and localization. Step 3: Introduce dynamic boundary regression loss L DBR Based on the difficulty of target localization and the rarity of the category, the bounding box regression weights are adaptively adjusted, and a hybrid loss function is designed. Step 4: Use the low-rank adaptation method LoRA and a hybrid loss function to fine-tune the target detection model based on Florence-2 and cross-modal feature enhancement, and then output the final small-sample target detection model. Step 5: Input the aerial image of the transmission line to be detected into the final small sample target detection model to obtain the detection results.

2. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in claim 1, characterized in that: In step 2, "few-sample category text" refers to the target category whose sample size in the small-sample target dataset for transmission lines is less than a set number. Multi-sample category images refer to the original images of the target category in a small sample target dataset of transmission lines where the number of samples is greater than a set value.

3. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in claim 1, characterized in that: In the target detection model based on Florence-2 and cross-modal feature enhancement, the feature enhancement module calculates the cosine similarity between multi-class images and input few-class text, selects the multi-class images that are semantically most similar to the few-class text, and transforms them into a knowledge vector V. knowledge Specifically, Step 21: Extract multi-sample image features from multi-sample category images using the contrastive language-image pre-trained model CLIP; Step 22: Extract the text features of the category labels of the few-sample class texts by using the contrastive language-image pre-trained model CLIP to obtain the few-sample text features; Step 23: Calculate the cosine similarity between the few-sample text features and the many-sample image features, and select the top N many-sample images that are most semantically similar to the few-sample text features based on the similarity scores as the selected many-sample images. Then, use the second visual encoder to extract visual image features from the selected many-sample images. Step 24: Normalize the top N similarity scores using Softmax, introduce a temperature parameter to control the distribution, obtain weights, and then perform a weighted summation of the visual image features to generate a knowledge vector V. knowledge .

4. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in claim 1, characterized in that: In the target detection model based on Florence-2 and cross-modal feature enhancement, the visual feature enhancement module integrates the knowledge vector V knowledge Injected and fused into the original visual feature F raw In the process, enhanced visual features F are generated. enhanced Specifically, ,in, For weight fusion.

5. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in any one of claims 1-4, characterized in that: In step 3, the hybrid loss function Including basic losses and dynamic boundary regression loss L DBR : ,in, This is the loss balance weighting coefficient.

6. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in claim 5, characterized in that: The basic loss uses the cross-entropy loss function. : ， in, Network parameters representing a target detection model based on Florence-2 and cross-modal feature enhancement; It is the total length of the structured text sequence; This indicates that the target detection model based on Florence-2 and cross-modal feature enhancement is based on multimodal features x and pre-generated target labels. Predict the next target marker The conditional probability.

7. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in any one of claims 1-4, characterized in that: In step 3, the dynamic boundary regression loss : ， in, This represents the number of bounding box pairs matched in the current batch. Let i be the IoU loss for the i-th matching pair. The dynamic weights for the i-th matching pair are: The aim is to adaptively adjust the contribution of the bounding box loss based on the difficulty of target localization and the rarity of the category.

8. The method for detecting small targets in transmission lines based on cross-modal feature enhancement under small sample conditions as described in claim 7, characterized in that: Dynamic weights Among them, the positioning difficulty weight Rarity weight ,in, To focus on hyperparameters, used to adjust the intensity of attention to difficult-to-locate targets; This represents the CIoU loss value of the i-th target to be detected in the current input image. , The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. It is a constant. Let be the number of instances in the category to which the i-th sample belongs. γ represents the total number of instances across all categories, and γ is a rarity weight hyperparameter used to adjust the intensity of attention given to smaller sample categories.

Images