An adaptive learning method and system for power transmission line inspection

Abstract

The invention discloses an adaptive learning method and system for power transmission line inspection, belonging to the technical field of image processing, and solving the problem of how to enhance the recognition ability and domain generalization ability of a visual language pre-training model for open vocabulary. The invention initializes a teacher model and a student model through a CLIP encoder, realizes knowledge distillation by constructing a sample-level contrast distillation loss and an inter-sample relationship distillation loss, and possesses the open vocabulary target detection ability; uses the teacher model to guide the student model, updates the student model parameters through gradient backpropagation, improves the accuracy of the student model for target domain recognition, enables the student model to approach or even reach the level of the teacher model in terms of accuracy and generalization performance, and uses the student model parameters to update the teacher model parameters, ensures the stability of the teacher model, and effectively enhances the domain generalization ability of the target detection model.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and relates to an adaptive learning method and system for power transmission line inspection. Background Art

[0002] In recent years, visual language pre-training models represented by the CLIP model have demonstrated strong processing capabilities in visual tasks by pre-training on a large amount of visual-text pairing data, and have good zero-sample recognition capabilities for conventional natural image processing. However, for specific downstream tasks, such as processing power line inspection tasks based on image or video input, model fine-tuning is required to obtain better recognition effects, but model fine-tuning will cause the model's original open vocabulary recognition ability to decrease, and it will overfit to a specific style of power line scenes, reducing the domain generalization ability of the visual language pre-training model. Therefore, how to maintain the open vocabulary recognition ability of the pre-trained model and achieve good recognition effects of the visual language pre-training model for new scenes has become the key to model fine-tuning.

[0003] The invention patent with publication number CN118038299A discloses a multimodal open word set target detection method in drone scenarios, designs an open word set target detection framework in drone scenarios, improves real-time detection performance by utilizing the DETR model and position prior information, and realizes zero-sample detection of unknown categories by utilizing the multimodal CLIP model and designing scene-related language prompts, so that the model has a certain open vocabulary detection capability.

[0004] The invention patent with publication number CN117789219A discloses a full-view segmentation method based on open vocabulary under multimodality. It uses multimodal models (such as CLIP) to provide image-text features in a shared embedding space, which can effectively bridge the gap between closed vocabulary and open vocabulary recognition. It uses a shared frozen convolution CLIP backbone to build all content into a single-stage framework, simplifying the current two-stage pipeline.

[0005] Existing technologies focus on adapting the CLIP model to a single downstream task and using its pre-trained knowledge to achieve open vocabulary recognition capabilities. However, due to its lack of domain generalization capabilities, the diverse target domains will lead to a decline in model recognition performance. For example, in the task of power line inspection, the detection model trained on sunny days using existing technologies will not perform well on cloudy or rainy days. Therefore, it is particularly important to enhance the open vocabulary recognition and domain generalization capabilities of the visual language pre-training model. Summary of the invention

[0006] The technical problem to be solved by the present invention is how to enhance the recognition ability and domain generalization ability of the visual language pre-training model for open vocabulary.

[0007] The present invention solves the above technical problems through the following technical solutions:

[0008] An adaptive learning method for power transmission line inspection comprises the following steps:

[0009] S1. Acquire source domain data and target domain data for power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, and construct a training set and a test set;

[0010] S2. Establish a target detection model, wherein the target detection model includes a CLIP encoder and an RPN network, and construct an average teacher framework based on the target detection model, wherein the average teacher framework includes a teacher model and a student model;

[0011] S3, input the source domain image into the student model, use the source domain annotation to construct the fully supervised object detection loss, and update the parameters of the student model through gradient backpropagation;

[0012] S4, after enhancing the target domain image, input it into the teacher model and the student model respectively, the target detection result output by the teacher model is used as the supervision information of the student model, and the target domain pseudo-label supervision loss of the student model is constructed;

[0013] S5. Input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model to construct sample-level contrast distillation and inter-sample relationship distillation.

[0014] S6. For each batch of source domain and target domain data, the student model is updated by backpropagation of the loss function gradient, and the parameters of the teacher model are updated using the parameters of the student model through exponential sliding average until the maximum number of iterations.

[0015] Furthermore, the following logic is used in S3 to construct the fully supervised target detection loss:

[0016]

[0017] In the formula, represents the fully supervised object detection loss, represents the classification loss, represents the regression loss, represents the number of source domain images, represents the i-th source domain image, represents the i-th source domain annotation, the semantic category of the source domain annotation comes from , Represents the set of semantic categories of source domain data.

[0018] Furthermore, the following logic is used in S4 to construct the target domain pseudo-label supervision loss of the student model:

[0019]

[0020] In the formula, represents the target domain pseudo-label supervision loss, represents the number of target domain images, represents the pseudo-label generated by the teacher model. The pseudo-label contains category labels and position labels. The semantic categories annotated in the target domain come from , Represents the semantic category of the target domain data.

[0021] Further, the S5 comprises the following steps:

[0022] S51, obtaining a historical teacher model, enhancing the target domain image, generating a candidate box using the historical teacher model, respectively calculating the category prediction probability of the candidate box obtained by the historical teacher model and the current student model, and calculating the sample-level contrast distillation loss;

[0023] S52. Obtain the weighted semantic features of the current student model and the historical teacher model respectively, calculate the conditional probability of the sample relationship between the current student model and the historical teacher model respectively, and calculate the distillation loss of the relationship between samples.

[0024] Furthermore, the following logic is used in S51 to calculate the historical teacher model to obtain the category prediction probability of the candidate box:

[0025]

[0026] In the formula, represents the probability that the historical teacher model predicts that the candidate box belongs to category c, represents exponential operation, Representation of calculation and The cosine similarity between Representation calculation representation and The cosine similarity between represents the text features of category c, Indicates the category The text features of , represents the number of categories, Indicates that the teacher model outputs the image features of the candidate box, and traverses the calculation history teacher model to predict that the candidate box belongs to The probability of all categories in , , represents the set of real numbers, represents the number of candidate boxes in the target domain image, Indicates the number of semantic categories in the target domain;

[0027] The following logic is used to calculate the category prediction probability of the candidate box obtained by the current student model:

[0028]

[0029] In the formula, It indicates the probability that the current student model predicts that the candidate box belongs to category c. Representation of calculation and The cosine similarity between Representation calculation representation and The cosine similarity between Indicates that the current student model outputs the image features of the candidate box, and traverses and calculates the current student model to predict that the candidate box belongs to The probability of all categories in , .

[0030] Furthermore, the following logic is used in S51 to calculate the sample-level contrast distillation loss:

[0031]

[0032] In the formula, represents the sample-level contrastive distillation loss, represents the query sample features, , represents the positive sample feature, , is a negative sample, coming from the memory bank, Represents the number of samples stored in the memory bank, represents the temperature coefficient, and Respectively indicate that the candidate box belongs to the category and reliability.

[0033] Furthermore, the following logic is used in S52 to respectively obtain the weighted semantic features of the historical teacher model and the weighted semantic features of the current student model:

[0034]

[0035]

[0036] In the formula, Indicates that the current student model obtains the The weighted semantic features of candidate boxes, , represents the feature dimension, Indicates that the current student model obtains the The category prediction probability of candidate boxes, , Indicates that the history teacher model obtains The weighted semantic features of candidate boxes, , Indicates that the history teacher model obtains The category prediction probability of candidate boxes, .

[0037] Furthermore, the conditional probability of the current student model sample relationship is calculated using the following logic in S52:

[0038]

[0039] In the formula, represents the conditional probability of the current student model sample relationship, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the current student model in the same batch obtains the candidate boxes and The weighted semantic features of candidate boxes, Indicates the number of weighted semantic feature sets of all candidate boxes obtained by the current student model in the same batch. express The current student model gets the Weighted semantic features of candidate boxes;

[0040] The conditional probability of the historical teacher model sample relationship is calculated using the following logic:

[0041]

[0042] In the formula, represents the conditional probability of the sample relationship of the historical teacher model, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the historical teacher model obtains the first candidate boxes and The weighted semantic features of candidate boxes, It indicates the number of weighted semantic feature sets of all candidate boxes obtained by the historical teacher model in the same batch. express The history teacher model obtains the The weighted semantic features of the candidate boxes.

[0043] Furthermore, the following logic is used in S52 to calculate the distillation loss of the relationship between samples:

[0044]

[0045] In the formula, represents the distillation loss of the relationship between samples, Represents the number of weighted semantic feature sets of all candidate boxes in the same batch, represents KL loss, Indicates the use of measure difference.

[0046] An adaptive learning system for power transmission line inspection, comprising a data acquisition module, a detection model building module, a classification loss module, a regression loss module, a knowledge distillation module and a model updating module;

[0047] The data acquisition module is used to acquire source domain data and target domain data for power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, and construct a training set and a test set;

[0048] The detection model building module is used to build a target detection model, the target detection model includes a CLIP encoder and an RPN network, and an average teacher framework is constructed based on the target detection model, the average teacher framework includes a teacher model and a student model;

[0049] The classification loss module is used to input the source domain image into the student model, construct the fully supervised object detection loss using the source domain annotation, and update the parameters of the student model through gradient back propagation;

[0050] The regression loss module is used to input the enhanced target domain image into the teacher model and the student model respectively, and the target detection result output by the teacher model is used as the supervision information of the student model to construct the target domain pseudo label supervision loss of the student model;

[0051] The knowledge distillation module is used to input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model, and construct sample-level contrast distillation and inter-sample relationship distillation;

[0052] The model update module is used to update the student model for each batch of source domain and target domain data by back-propagation of the loss function gradient, and to update the parameters of the teacher model using the parameters of the student model through exponential sliding average until the maximum number of iterations.

[0053] The advantages of the present invention are:

[0054] The present invention provides an object detection model based on an average teacher framework, wherein a teacher model and a student model are initialized through a CLIP encoder, and knowledge distillation is realized by constructing a sample-level contrast distillation loss and an inter-sample relationship distillation loss between a historical teacher model and a current student model, so that the object detection model has an open vocabulary object detection capability and an unsupervised domain adaptation capability, so that the model can adapt to category differences and domain style differences in new scenes, and the generalization capability of the object detection model for new scenes is greatly improved; at the same time, the present invention uses a teacher model to guide a student model, updates the student model parameters through gradient backpropagation, improves the accuracy of the student model for target domain recognition, enables the student model to approach or even reach the level of the teacher model in terms of accuracy and generalization performance, uses the parameters of the student model to update the teacher model parameters, ensures the stability of the teacher model, and effectively enhances the domain generalization capability of the object detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of an adaptive learning method for power transmission line inspection according to the first embodiment of the present invention;

[0056] Figure 2 1 is a schematic diagram of the overall model structure of the target detection model of the first embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of the overall framework structure of the average teacher framework of the first embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0060] Embodiment 1

[0061] like Figure 1Specifically, an adaptive learning method for power transmission line inspection is disclosed, comprising the following steps:

[0062] S1. Acquire source domain data and target domain data for power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, randomly crop the source domain images and target domain images, scale all image sizes to a uniform size, and then perform numerical normalization processing to construct a training set and a test set, wherein the training set includes the source domain data and the target domain data, and the test set includes the target domain data;

[0063] In this embodiment, the transmission line inspection process is divided into multiple stages. The training set and test set data of the transmission line inspection will be supplemented batch by batch. The single-stage transmission line inspection data is used as an example to illustrate. The source domain images and target domain images are randomly cropped, and all image sizes are scaled to 512×1024. Then, the numerical normalization is performed to construct the training set and test set. The semantic category set of the source domain data is , the semantic category set of the target domain data is , for It means that the target domain data has semantic categories that have not been seen in the source domain data. In the test phase, all categories of the target domain data need to be identified.

[0064] S2. Establish a target detection model, wherein the target detection model includes a CLIP encoder and an RPN network, construct an average teacher framework based on the target detection model, wherein the average teacher framework includes a teacher model and a student model, and initialize model parameters of the teacher model and the student model using the CLIP encoder;

[0065] like Figure 2 As shown, this embodiment establishes a target detection model based on the Pytorch deep learning framework, and the target detection model includes a CLIP encoder and an RPN network.

[0066] In this embodiment, the CLIP encoder includes a visual branch and text branch , CLIP encoder uses the CLIP encoder based on ResNet50 as the backbone network of the Faster R-CNN detector, and adopts the category-independent candidate box generation network (RPN network) to propose image regions and use the CLIP visual encoder to extract visual embedding. For each candidate box output by the RPN network, RoIAlign is used on the 4th feature map res4 of ResNet50 to obtain the image features of the candidate box. The candidate box features are passed through the last ResNet block and the attention pool mechanism of CLIP to obtain the representation of the candidate box for classification. .

[0067] The text branch of the CLIP encoder It is often used to accept text input containing only categories but no style, such as the unstyled text "A photo of a [class].". The text description is represented as an image containing a specific semantic category class, which is a set of semantic categories from the source domain data. Or the semantic category of the target domain data For the text prompt words of the background category, a fixed all-zero representation is used.

[0068] In this embodiment, the mean teacher framework includes a teacher model and a student model; this embodiment constructs a mean teacher framework based on the target detection model, including a teacher model for pseudo-label generation and a student model that enhances the teacher model through knowledge updating. Both the teacher model and the student model are initialized using the CLIP encoder at the beginning of the training process to provide good recognition capabilities for the two models. The student model parameters are updated by gradient descent, and the teacher model parameters are updated using an exponential sliding average strategy derived from the student model.

[0069] S3, input the source domain image into the student model, use the source domain annotation to construct the fully supervised object detection loss, and update the parameters of the student model through gradient backpropagation;

[0070] In this embodiment, during each iteration of model training, supervised learning is performed by inputting annotated source domain images into the student model, and the following logical representation is used to construct the fully supervised object detection loss:

[0071]

[0072] In the formula, represents the fully supervised object detection loss, represents the classification loss, represents the regression loss, represents the number of source domain images, represents the i-th source domain image, represents the i-th source domain annotation, the semantic category of the source domain annotation comes from .

[0073] S4, after enhancing the target domain image, input it into the teacher model and the student model respectively, the target detection result output by the teacher model is used as the supervision information of the student model, the target domain pseudo-label supervision loss of the student model is constructed, and the parameters of the student model are updated through gradient back propagation;

[0074] like Figure 3As shown, in this embodiment, since the target domain data is unlabeled, the target domain image is weakly enhanced (such as horizontal flipping, random cropping) and then input into the teacher model to generate pseudo labels. The target detection results output by the teacher model are used as the supervision information of the student model; the target domain image is strongly enhanced by the RandAug library and then input into the student model. The following logic is used to express the target domain pseudo label supervision loss of the student model:

[0075]

[0076] In the formula, represents the target domain pseudo-label supervision loss, represents the number of target domain images, represents the pseudo-label generated by the teacher model. The pseudo-label contains the category label and the position label. The semantic category of the target domain image comes from .

[0077] S5. Input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model, construct sample-level contrast distillation and inter-sample relationship distillation, and update the parameters of the student model through gradient backpropagation;

[0078] like Figure 3 As shown, in this embodiment, S5 includes the following steps:

[0079] S51, obtaining a historical teacher model, enhancing the target domain image, generating a candidate box using the historical teacher model, respectively calculating the category prediction probability of the candidate box obtained by the historical teacher model and the current student model, and calculating the sample-level contrast distillation loss;

[0080] In this embodiment, the teacher model that has been iterated T times before the current stage is obtained as the historical teacher model, and the student model of the current stage is used as the current student model. Compared with the teacher model of the current stage, the historical teacher model has more knowledge of the original CLIP encoder and has stronger feature generalization ability. This example realizes knowledge distillation by constructing a sample-level contrast distillation loss, and uses the teacher model to guide the student model, so that the student model can approach or even reach the level of the teacher model in terms of accuracy and generalization performance.

[0081] In this embodiment, the target domain image is enhanced by the RandAug library, and the candidate box is generated by the historical teacher model. The following logic is used to calculate the category prediction probability of the candidate box by the historical teacher model: :

[0082]

[0083] In the formula, represents the probability that the historical teacher model predicts that the candidate box belongs to category c, represents exponential operation, Representation calculation representation and The cosine similarity between Representation calculation representation and The cosine similarity between represents the text features of category c, Indicates the category The text features of , represents the number of categories, Represents the image features of the candidate box output by the teacher model, wherein the traversal calculation history teacher model predicts that the candidate box belongs to The probability of all categories in , , represents the set of real numbers, represents the number of candidate boxes in the target domain image, Represents the number of target domain semantic categories, including background categories.

[0084] The following logic is used to calculate the current student model to obtain the category prediction probability of the candidate box :

[0085]

[0086] In the formula, It indicates the probability that the current student model predicts that the candidate box belongs to category c. Representation calculation representation and The cosine similarity between Representation calculation representation and The cosine similarity between Indicates that the current student model outputs the image features of the candidate box, wherein the current student model predicts that the candidate box belongs to The probability of all categories in , .

[0087] In this embodiment, the sample-level contrast distillation loss is mainly used to select positive samples and negative samples. In this embodiment, the student model obtains the category prediction probability of the candidate box. As a query sample, the historical teacher model obtains the category prediction probability of the same candidate box As positive samples, negative samples use memory banks Used to store the prediction results of the historical teacher model, where Representation Memory The number of samples, the memory bank obtains the category prediction probability of other candidate boxes through the historical teacher model. This embodiment uses the following logic to calculate the sample-level contrast distillation loss:

[0088]

[0089] In the formula, represents the sample-level contrastive distillation loss, represents the query sample features, , represents the positive sample feature, , is a negative sample, coming from the memory bank, Represents the number of samples stored in the memory bank, represents the temperature coefficient, and Respectively indicate that the candidate box belongs to the category and Reliability, take ,When the predicted probability is larger, it means that the sample is more reliable and has a greater contribution to the sample-level contrast distillation loss.

[0090] S52, respectively obtain the weighted semantic features of the current student model and the historical teacher model, respectively calculate the conditional probability of the sample relationship between the current student model and the historical teacher model, and calculate the distillation loss of the relationship between the samples;

[0091] In this embodiment, sample-level comparative distillation is performed based on the historical model, focusing only on the knowledge transfer of a single instance. This embodiment further constructs inter-sample relationship distillation based on the semantic space mapping relationship.

[0092] In this embodiment, in order to fuse the text branch of the CLIP encoder The knowledge contained in the semantic space is used to transform the text branch of the CLIP encoder Output text features As a set of coordinate systems, , ,in, Represents feature dimensions; Use the historical teacher model and the current student model to obtain the category prediction probability of the candidate box for the text feature Weighted, use the following logic to obtain the weighted semantic features of the historical teacher model and the weighted semantic features of the current student model:

[0093]

[0094]

[0095] In the formula, Indicates that the current student model obtains the The weighted semantic features of candidate boxes, , Indicates that the current student model obtains the The category prediction probability of candidate boxes, , Indicates that the history teacher model obtains The weighted semantic features of candidate boxes, , Indicates that the history teacher model obtains The category prediction probability of candidate boxes, .

[0096] In this embodiment, the relationship between samples is considered, and the conditional probability of the relationship between the current student model samples is calculated using the following logic:

[0097]

[0098] In the formula, represents the conditional probability of the current student model sample relationship, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the current student model in the same batch obtains the candidate boxes and The weighted semantic features of candidate boxes, Indicates the number of weighted semantic feature sets of all candidate boxes obtained by the current student model in the same batch. express The current student model gets the The weighted semantic features of the candidate boxes.

[0099] In this embodiment, the conditional probability of the historical teacher model sample relationship is calculated using the following logic:

[0100]

[0101] In the formula, represents the conditional probability of the sample relationship of the historical teacher model, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the historical teacher model obtains the first candidate boxes and The weighted semantic features of candidate boxes, It indicates the number of weighted semantic feature sets of all candidate boxes obtained by the historical teacher model in the same batch. express The history teacher model obtains the The weighted semantic features of the candidate boxes.

[0102] In this embodiment, the following logic is used to calculate the distillation loss of the relationship between samples:

[0103]

[0104] In the formula, represents the distillation loss of the relationship between samples, Represents the number of weighted semantic feature sets of all candidate boxes in the same batch, represents the KL loss, Indicates the use of measure The difference, among which, .

[0105] S6. For each batch of source domain and target domain data, the student model is updated by backpropagation of the loss function gradient. The parameters of the teacher model are updated by using the parameters of the student model through exponential sliding average until the maximum number of iterations is reached. The test set is input and the accuracy of the target detection model is calculated.

[0106] In this embodiment, for each batch of source domain and target domain data, the current student model is updated by backpropagation of the loss function gradient. The student model can inherit deeper knowledge from the teacher model, help the current student model improve the accuracy of target domain recognition, and improve the generalization performance of the current student model. By using the exponential sliding average, the parameters of the historical teacher model are updated with the parameters of the current student model, which can maintain the stability of the teacher model, so that the teacher model will not change too quickly due to fluctuations in a single training iteration, and can more stably transfer useful knowledge to the student model.

[0107] Embodiment 2

[0108] The present invention also discloses an adaptive learning system for power transmission line inspection, comprising a data acquisition module, a detection model building module, a classification loss module, a regression loss module, a knowledge distillation module and a model updating module;

[0109] The data acquisition module is used to acquire source domain data and target domain data of power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, randomly crop the source domain images and target domain images, scale all image sizes to a uniform size, perform numerical normalization processing, and construct a training set and a test set;

[0110] In this embodiment, the transmission line inspection process is divided into multiple stages. The training set and test set data of the transmission line inspection will be supplemented batch by batch. The single-stage transmission line inspection data is used as an example to illustrate. The source domain images and target domain images are randomly cropped, and all image sizes are scaled to 512×1024. Then, the numerical normalization is performed to construct the training set and test set. The semantic category set of the source domain data is , the semantic category set of the target domain data is , for It means that the target domain data has semantic categories that have not been seen in the source domain data. In the test phase, all categories of the target domain data need to be identified.

[0111] The detection model building module is used to build a target detection model, the target detection model includes a CLIP encoder and an RPN network, an average teacher framework is constructed based on the target detection model, the average teacher framework includes a teacher model and a student model, and the model parameters of the teacher model and the student model are initialized using the CLIP encoder;

[0112] This embodiment establishes a target detection model based on the Pytorch deep learning framework, and the target detection model includes a CLIP encoder and an RPN network.

[0113] In this embodiment, the CLIP encoder includes a visual branch and text branch , CLIP encoder uses the CLIP encoder based on ResNet50 as the backbone network of the Faster R-CNN detector, and adopts the category-independent candidate box generation network (RPN network) to propose image regions and use the CLIP visual encoder to extract visual embedding. For each candidate box output by the RPN network, RoIAlign is used on the 4th feature map res4 of ResNet50 to obtain the image features of the candidate box. The candidate box features are passed through the last ResNet block and the attention pool mechanism of CLIP to obtain the representation of the candidate box for classification. .

[0114] The text branch of the CLIP encoder It is often used to accept text input containing only categories but no style, such as the unstyled text "A photo of a [class].". The text description is represented as an image containing a specific semantic category class, which is a set of semantic categories from the source domain data. Or the semantic category of the target domain data For the text prompt words of the background category, a fixed all-zero representation is used.

[0115] In this embodiment, the mean teacher framework includes a teacher model and a student model; this embodiment constructs a mean teacher framework based on the target detection model, including a teacher model for pseudo-label generation and a student model that enhances the teacher model through knowledge updating. Both the teacher model and the student model are initialized using the CLIP encoder at the beginning of the training process to provide good recognition capabilities for the two models. The student model parameters are updated by gradient descent, and the teacher model parameters are updated using an exponential sliding average strategy derived from the student model.

[0116] The classification loss module is used to input the source domain image into the student model, construct the fully supervised object detection loss using the source domain annotation, and update the parameters of the student model through gradient back propagation;

[0117] In this embodiment, during each iteration of model training, supervised learning is performed by inputting annotated source domain images into the student model, and the following logical representation is used to construct the fully supervised object detection loss:

[0118]

[0119] In the formula, represents the fully supervised object detection loss, represents the classification loss, represents the regression loss, represents the number of source domain images, represents the i-th source domain image, represents the i-th source domain annotation, the semantic category of the source domain annotation comes from .

[0120] The regression loss module is used to enhance the target domain image and input it into the teacher model and the student model respectively, the target detection result output by the teacher model is used as the supervision information of the student model, the target domain pseudo-label supervision loss of the student model is constructed, and the parameters of the student model are updated through gradient back propagation;

[0121] In this embodiment, since the target domain data is unlabeled, the target domain image is weakly enhanced (such as horizontal flipping, random cropping) and then input into the teacher model to generate pseudo labels. The target detection results output by the teacher model are used as the supervision information of the student model. The target domain image is strongly enhanced by the RandAug library and then input into the student model. The following logic is used to express the target domain pseudo label supervision loss of the student model:

[0122]

[0123] In the formula, represents the target domain pseudo-label supervision loss, represents the number of target domain images, represents the pseudo-label generated by the teacher model. The pseudo-label contains the category label and the position label. The semantic category of the target domain image comes from .

[0124] The knowledge distillation module is used to input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model, construct sample-level contrast distillation and inter-sample relationship distillation, and update the parameters of the student model through gradient back propagation;

[0125] The knowledge distillation module includes a contrast distillation loss unit and a relational distillation loss unit;

[0126] The contrastive distillation loss unit is used to obtain a historical teacher model, enhance the target domain image, generate a candidate box using the historical teacher model, calculate the category prediction probability of the candidate box obtained by the historical teacher model and the current student model respectively, and calculate the sample-level contrastive distillation loss;

[0127] In this embodiment, the teacher model that has been iterated T times before the current stage is obtained as the historical teacher model, and the student model of the current stage is used as the current student model. Compared with the teacher model of the current stage, the historical teacher model has more knowledge of the original CLIP encoder and has stronger feature generalization ability. This example realizes knowledge distillation by constructing a sample-level contrast distillation loss, and uses the teacher model to guide the student model, so that the student model can approach or even reach the level of the teacher model in terms of accuracy and generalization performance.

[0128] In this embodiment, the target domain image is enhanced by the RandAug library, and the candidate box is generated by the historical teacher model. The following logic is used to calculate the category prediction probability of the candidate box by the historical teacher model: :

[0129]

[0130] In the formula, represents the probability that the historical teacher model predicts that the candidate box belongs to category c, represents exponential operation, Representation calculation representation and The cosine similarity between Representation calculation representation and The cosine similarity between represents the text features of category c, Indicates the category The text features of , represents the number of categories, Represents the image features of the candidate box output by the teacher model, wherein the traversal calculation history teacher model predicts that the candidate box belongs to The probability of all categories in , , represents the set of real numbers, represents the number of candidate boxes in the target domain image, Represents the number of target domain semantic categories, including background categories.

[0131] The following logic is used to calculate the current student model to obtain the category prediction probability of the candidate box :

[0132]

[0133] In the formula, It indicates the probability that the current student model predicts that the candidate box belongs to category c. Representation calculation representation and The cosine similarity between Representation calculation representation and The cosine similarity between Indicates that the current student model outputs the image features of the candidate box, wherein the current student model predicts that the candidate box belongs to The probability of all categories in , .

[0134] In this embodiment, the sample-level contrast distillation loss is mainly used to select positive samples and negative samples. In this embodiment, the student model obtains the category prediction probability of the candidate box. As a query sample, the historical teacher model obtains the category prediction probability of the same candidate box As positive samples, negative samples use memory banks Used to store the prediction results of the historical teacher model, where Representation Memory The number of samples, the memory bank obtains the category prediction probability of other candidate boxes through the historical teacher model. This embodiment uses the following logic to calculate the sample-level contrast distillation loss:

[0135]

[0136] In the formula, represents the sample-level contrastive distillation loss, represents the query sample features, , represents the positive sample feature, , is a negative sample, coming from the memory bank, Represents the number of samples stored in the memory bank, represents the temperature coefficient, and Respectively indicate that the candidate box belongs to the category and Reliability, take ,When the predicted probability is larger, it means that the sample is more reliable and has a greater contribution to the sample-level contrast distillation loss.

[0137] The relation distillation loss unit is used to obtain weighted semantic features of the current student model and the historical teacher model, respectively, calculate the conditional probability of the relationship between the current student model and the historical teacher model samples, and calculate the relationship distillation loss between the samples;

[0138] In this embodiment, sample-level comparative distillation is performed based on the historical model, focusing only on the knowledge transfer of a single instance. This embodiment further constructs inter-sample relationship distillation based on the semantic space mapping relationship.

[0139] In this embodiment, in order to fuse the text branch of the CLIP encoder The knowledge contained in the semantic space is used to transform the text branch of the CLIP encoder Output text features As a set of coordinate systems, , ,in, Represents feature dimensions; Use the historical teacher model and the current student model to obtain the category prediction probability of the candidate box for the text feature Weighted, use the following logic to obtain the weighted semantic features of the historical teacher model and the weighted semantic features of the current student model:

[0140]

[0141]

[0142] In the formula, Indicates that the current student model obtains the The weighted semantic features of candidate boxes, , Indicates that the current student model obtains the The category prediction probability of candidate boxes, , Indicates that the history teacher model obtains The weighted semantic features of candidate boxes, , Indicates that the history teacher model obtains The category prediction probability of candidate boxes, .

[0143] In this embodiment, the relationship between samples is considered, and the conditional probability of the relationship between the current student model samples is calculated using the following logic:

[0144]

[0145] In the formula, represents the conditional probability of the current student model sample relationship, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the current student model in the same batch obtains the candidate boxes and The weighted semantic features of candidate boxes, Indicates the number of weighted semantic feature sets of all candidate boxes obtained by the current student model in the same batch. express The current student model gets the The weighted semantic features of the candidate boxes.

[0146] In this embodiment, the conditional probability of the historical teacher model sample relationship is calculated using the following logic:

[0147]

[0148] In the formula, represents the conditional probability of the sample relationship of the historical teacher model, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the historical teacher model obtains the first candidate boxes and The weighted semantic features of candidate boxes, It indicates the number of weighted semantic feature sets of all candidate boxes obtained by the historical teacher model in the same batch. express The history teacher model obtains the The weighted semantic features of the candidate boxes.

[0149] In this embodiment, the following logic is used to calculate the distillation loss of the relationship between samples:

[0150]

[0151] In the formula, represents the distillation loss of the relationship between samples, Represents the number of weighted semantic feature sets of all candidate boxes in the same batch, represents the KL loss, Indicates the use of measure The difference, among which, .

[0152] The model update module is used to update the student model for each batch of source domain and target domain data by back-propagation of the loss function gradient, and to update the parameters of the teacher model using the parameters of the student model through exponential sliding average until the maximum number of iterations is reached. The test set is input to calculate the accuracy of the target detection model.

[0153] In this embodiment, for each batch of source domain and target domain data, the current student model is updated by backpropagation of the loss function gradient. The student model can inherit deeper knowledge from the teacher model, help the current student model improve the accuracy of target domain recognition, and improve the generalization performance of the current student model. By using the exponential sliding average, the parameters of the historical teacher model are updated with the parameters of the current student model, which can maintain the stability of the teacher model, so that the teacher model will not change too quickly due to fluctuations in a single training iteration, and can more stably transfer useful knowledge to the student model.

[0154] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive learning method for power transmission line inspection, characterized in that: The following steps are involved: S1. Acquire source domain data and target domain data for power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, and construct a training set and a test set; S2. Establish a target detection model, wherein the target detection model includes a CLIP encoder and an RPN network, and construct an average teacher framework based on the target detection model, wherein the average teacher framework includes a teacher model and a student model; S3, input the source domain image into the student model, use the source domain annotation to construct the fully supervised object detection loss, and update the parameters of the student model through gradient backpropagation; S4, after enhancing the target domain image, input it into the teacher model and the student model respectively, the target detection result output by the teacher model is used as the supervision information of the student model, and the target domain pseudo-label supervision loss of the student model is constructed; S5. Input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model to construct sample-level contrast distillation and inter-sample relationship distillation. S6. For each batch of source domain and target domain data, the student model is updated by backpropagation of the loss function gradient, and the parameters of the teacher model are updated using the parameters of the student model through exponential sliding average until the maximum number of iterations.

2. The adaptive learning method for power transmission line inspection according to claim 1, characterized in that: The following logic is used in S3 to construct the fully supervised target detection loss: In the formula, represents the fully supervised object detection loss, represents the classification loss, represents the regression loss, represents the number of source domain images, represents the i-th source domain image, represents the i-th source domain annotation, the semantic category of the source domain annotation comes from , Represents the set of semantic categories of source domain data.

3. The adaptive learning method for power transmission line inspection according to claim 2, characterized in that: The following logic is used in S4 to construct the target domain pseudo-label supervision loss of the student model: In the formula, represents the target domain pseudo-label supervision loss, represents the number of target domain images, represents the pseudo-label generated by the teacher model. The pseudo-label contains category labels and position labels. The semantic categories annotated in the target domain come from , Represents the semantic category of the target domain data.

4. The adaptive learning method for power transmission line inspection according to claim 3, characterized in that: The S5 comprises the following steps: S51, obtaining a historical teacher model, enhancing the target domain image, generating a candidate box using the historical teacher model, respectively calculating the category prediction probability of the candidate box obtained by the historical teacher model and the current student model, and calculating the sample-level contrast distillation loss; S52. Obtain the weighted semantic features of the current student model and the historical teacher model respectively, calculate the conditional probability of the sample relationship between the current student model and the historical teacher model respectively, and calculate the distillation loss of the relationship between samples.

5. The adaptive learning method for power transmission line inspection according to claim 4, characterized in that: In S51, the historical teacher model is calculated using the following logic to obtain the category prediction probability of the candidate box: In the formula, represents the probability that the historical teacher model predicts that the candidate box belongs to category c, represents exponential operation, Representation of calculation and The cosine similarity between Representation of calculation and The cosine similarity between represents the text features of category c, Indicates the category The text features of , represents the number of categories, Indicates that the teacher model outputs the image features of the candidate box, and traverses the calculation history teacher model to predict that the candidate box belongs to The probability of all categories in , , represents the set of real numbers, represents the number of candidate boxes in the target domain image, Indicates the number of semantic categories in the target domain; The following logic is used to calculate the category prediction probability of the candidate box obtained by the current student model: In the formula, It indicates the probability that the current student model predicts that the candidate box belongs to category c. Representation of calculation and The cosine similarity between Representation of calculation and The cosine similarity between Indicates that the current student model outputs the image features of the candidate box, and traverses and calculates the current student model to predict that the candidate box belongs to The probability of all categories in , .

6. The adaptive learning method for power transmission line inspection according to claim 5, characterized in that: In S51, the sample-level contrast distillation loss is calculated using the following logic: In the formula, represents the sample-level contrastive distillation loss, represents the query sample features, , represents the positive sample feature, , is a negative sample, coming from the memory bank, Represents the number of samples stored in the memory bank, represents the temperature coefficient, and Respectively indicate that the candidate box belongs to the category and reliability.

7. The adaptive learning method for power transmission line inspection according to claim 6, characterized in that: In S52, the following logic is used to respectively obtain the weighted semantic features of the historical teacher model and the weighted semantic features of the current student model: In the formula, Indicates that the current student model obtains the The weighted semantic features of candidate boxes, , represents the feature dimension, Indicates that the current student model obtains the The category prediction probability of candidate boxes, , Indicates that the history teacher model obtains The weighted semantic features of candidate boxes, , Indicates that the history teacher model obtains The category prediction probability of candidate boxes, .

8. The adaptive learning method for power transmission line inspection according to claim 7, characterized in that: In S52, the conditional probability of the current student model sample relationship is calculated using the following logic: In the formula, represents the conditional probability of the current student model sample relationship, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the current student model in the same batch obtains the candidate boxes and The weighted semantic features of candidate boxes, Indicates the number of weighted semantic feature sets of all candidate boxes obtained by the current student model in the same batch. express The current student model gets the Weighted semantic features of candidate boxes; The conditional probability of the historical teacher model sample relationship is calculated using the following logic: In the formula, represents the conditional probability of the sample relationship of the historical teacher model, Representation calculation and The cosine similarity between Representation calculation and The cosine similarity between and Indicates that the historical teacher model obtains the first candidate boxes and The weighted semantic features of candidate boxes, It indicates the number of weighted semantic feature sets of all candidate boxes obtained by the historical teacher model in the same batch. express The history teacher model obtains the The weighted semantic features of the candidate boxes.

9. The adaptive learning method for power transmission line inspection according to claim 8, characterized in that: In S52, the following logic is used to calculate the distillation loss of the relationship between samples: In the formula, represents the distillation loss of the relationship between samples, Represents the number of weighted semantic feature sets of all candidate boxes in the same batch, represents the KL loss, Indicates the use of measure difference.

10. An adaptive learning system for power transmission line inspection, characterized in that: It includes data acquisition module, detection model building module, classification loss module, regression loss module, knowledge distillation module and model update module; The data acquisition module is used to acquire source domain data and target domain data for power transmission line inspection, wherein the source domain data includes source domain images and source domain annotations, and the target domain data includes target domain images, and construct a training set and a test set; The detection model building module is used to build a target detection model, the target detection model includes a CLIP encoder and an RPN network, and an average teacher framework is constructed based on the target detection model, the average teacher framework includes a teacher model and a student model; The classification loss module is used to input the source domain image into the student model, construct the fully supervised object detection loss using the source domain annotation, and update the parameters of the student model through gradient backpropagation; The regression loss module is used to input the enhanced target domain image into the teacher model and the student model respectively, and the target detection result output by the teacher model is used as the supervision information of the student model to construct the target domain pseudo label supervision loss of the student model; The knowledge distillation module is used to input the candidate boxes output by the candidate box generation network of the teacher model into the teacher model and the student model, and construct sample-level contrast distillation and inter-sample relationship distillation; The model update module is used to update the student model for each batch of source domain and target domain data by back-propagation of the loss function gradient, and to update the parameters of the teacher model using the parameters of the student model through exponential sliding average until the maximum number of iterations.

Images