A photovoltaic power station detection method based on feature decoupling domain adaptation

Abstract

This invention discloses a photovoltaic power plant detection method based on feature decoupling domain adaptation, comprising: constructing a multi-source domain new energy power plant hidden danger dataset; establishing a feature decoupling domain adaptation model, which is based on the DETR detection framework and includes a backbone network for extracting image features, a Transformer encoder / decoder, and a prediction head; setting a domain prompt generation module between the backbone network and the encoder; and connecting the encoder output to the decoder via a feature aligner, query filtering, and channel separation module; pre-training the DETR detection framework using the multi-source domain new energy power plant hidden danger dataset to obtain a pre-trained DETR detection framework; and freezing the core parameters of the DETR detection framework for the target domain data, optimizing only the lightweight domain adaptation module composed of the domain prompt generation module and the channel separation module to obtain an adaptation model for photovoltaic power plant detection. This method maintains detection robustness even under data scarcity conditions.

Description

Technical Field

[0001] This invention belongs to the field of industrial defect detection and target detection technology, specifically a new energy power plant hidden danger detection method based on feature decoupling domain adaptation, which is particularly suitable for cross-domain robust detection in scenarios such as hot spots and battery defects in photovoltaic power plants. Background Technology

[0002] In the automated quality inspection of new energy power plants (such as photovoltaic power plants), the visual model often suffers from image feature distribution shifts due to changes in production batches, differences in acquisition equipment, or environmental changes, which in turn leads to a decline in detection performance. Traditional solutions include fine-tuning the model or using single-step domain adaptation methods, but they have the following limitations: (1) Fine-tuning requires a large amount of labeled data, while defect samples of new energy power plants are scarce, and it is difficult to obtain enough target domain samples in the short term; (2) Existing domain adaptation methods (such as domain adversarial training) only focus on global feature alignment and ignore instance-level feature decoupling, which makes the detection head unable to adapt to the defect features of the target domain; (3) Although the end-to-end model based on the detection transformer (DETR) simplifies the detection process, it is more sensitive to cross-domain feature shifts and requires additional supervised learning to generalize biases. Summary of the Invention

[0003] To address the performance degradation of models in new energy power plant hazard detection caused by production line changes, equipment upgrades, or batch differences, this invention proposes a photovoltaic power plant detection method based on feature decoupling and domain adaptation. By decoupling domain-invariant features from domain-specific features and adapting to target domain characteristics in stages, the method significantly improves detection stability in cross-domain scenarios. In the first stage, adversarial learning and feature alignment loss are used to decouple image features into domain-specific and domain-invariant features. In the second stage, the backbone network parameters are frozen, and only a lightweight domain adaptation module is updated, achieving efficient feature distribution alignment. This method maintains robustness even under data scarcity conditions, significantly outperforming existing domain adaptation techniques.

[0004] The technical solution adopted by the present invention to solve the aforementioned technical problem is as follows:

[0005] This invention provides a photovoltaic power plant detection method based on feature decoupling domain adaptation, the method comprising the following:

[0006] Construct a dataset of potential hazards in multi-source domain new energy power plants;

[0007] A feature decoupling domain adaptation model is established. The feature decoupling domain adaptation model is based on the DETR detection framework and includes a backbone network for extracting image features, a Transformer encoder and a Transformer decoder for global context modeling and object query interaction, and a prediction head for outputting the detection result set. A domain prompt generation module TEPG is set between the backbone network and the Transformer encoder. The output of the Transformer encoder is connected to the Transformer decoder through a feature aligner, query filtering, and channel separation module TDCS.

[0008] The domain cue generation module TEPG includes 1×1 Conv, 3×3 DWConv, 1×1 Conv, 3×3 DWConv, cue filtering operation, and spatial pooling operation, and the target domain image features extracted by the backbone network. After 1×1 Conv processing, channel-adjusted features are obtained. These channel-adjusted features are then processed by 3×3 DWConv, 1×1 Conv, and 3×3 DWConv. The output of this process is added to the channel-adjusted features, and the resulting feature enters the prompt selection operation. In the prompt selection operation, the features are first activated by calculating importance weights using a sigmoid function. Then, a Top-K selection mechanism is used to select the few most critical feature elements. Finally, these selected information are used to obtain the target domain prompt through spatial pooling. ;

[0009] The channel separation module TDCS includes a multilayer perceptron FFN-MLP, which queries domain-invariant objects obtained through query filtering. and domain-specific queries The target domain prompt obtained by the domain prompt generation module TEPG and domain-specific queries After vector concatenation, the target domain specific channels are adjusted through feedforward neural network-multilayer perceptron-MLP processing, and then queried with domain-invariant object queries. Concatenate the vectors to obtain the reconstructed query vector. ;

[0010] Reconstruct the query vector Querying specific domains The concatenated data is used as input to the Transformer decoder, and the detection result is output after processing by the prediction head.

[0011] Two-stage training:

[0012] Phase 1: The DETR detection framework, consisting of a backbone network, a Transformer encoder, a Transformer decoder, and a prediction head, is pre-trained using a multi-source domain new energy power plant hidden danger dataset. Image features are decoupled into domain-specific features and domain-invariant features through adversarial learning and feature alignment loss. During training, detection loss and decoupling loss are set to obtain the pre-trained DETR detection framework.

[0013] Phase 2: For the target domain data, freeze the core parameters of the DETR detection framework and optimize only the lightweight domain adaptation module composed of the domain hint generation module TEPG and the channel separation module TDCS to obtain the adaptation model;

[0014] The adapted model is embedded into the intelligent inspection system for new energy power plants for photovoltaic power plant inspection.

[0015] The present invention also protects a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, can implement the above-described method.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] 1. The lightweight domain adaptation module in this invention overcomes the parameter update bottleneck of traditional fine-tuning methods. Compared to existing solutions that require full parameter fine-tuning, this invention innovatively employs a Domain Hint Generation Module (TEPG) and a Channel Separation Module (TDCS), achieving target domain style adaptation by freezing the backbone network and optimizing only a small number of trainable parameters. This significantly reduces the computational cost of cross-domain adaptation, making it particularly suitable for the real-time requirements of rapid deployment of production line equipment.

[0018] 2. The method of this invention achieves multi-level feature alignment through an encoder-decoder collaborative decoupling mechanism. Compared with traditional single-step domain adaptation methods, this framework innovatively decouples image features into domain-invariant features (which are generated by the encoder and decoder). Features that participate in the generation and are fed into the prediction head are combined with domain-specific features. The object query is explicitly split in the Transformer decoder to form a stable, domain-invariant object query. (Focusing on the essence of defects) and adaptable domain-specific object queries (By carrying style information), the robustness in data-scarce scenarios is significantly improved. During the pre-training stage, by jointly optimizing the detection task (i.e., the part performed by the prediction head, which can accurately locate and identify defects in the image) and the feature separation task (i.e., separating features into "domain-invariant" and "domain-specific" parts), the robustness of the model to cross-domain feature shifts is significantly improved, solving the problem that existing methods ignore local defect features due to global feature alignment.

[0019] 3. For scenarios where data is scarce in new energy power plants, this invention employs a feature decoupling loss function based on adversarial learning. This addresses the problem that existing domain adversarial training methods typically directly align the overall feature distribution, leading to the detection head's inability to adapt to the characteristics of the target domain. This invention achieves cross-domain feature embedding alignment at the encoder layer through collaborative optimization of the feature aligner and the domain discriminator, while simultaneously separating the object query channel at the decoder layer. This enables the model to autonomously distinguish and retain domain-invariant features crucial for defect detection, maintaining excellent performance even when target domain labeled data is insufficient.

[0020] 4. When the intelligent inspection system of a new energy power plant detects an image of a new production line or a new equipment model, the present invention automatically activates the domain prompt update process and combines it with the manual review mechanism of low-confidence samples to form a continuously optimized target domain dataset, forming a dynamic closed-loop optimization, which solves the pain point that traditional static models cannot adapt to continuous domain shifts.

[0021] 5. This invention is aimed at static image detection in photovoltaic power plants and other applications. Based on the DETR detection framework, it decouples features at the encoding-decoding layer to cope with drastic domain shifts caused by changes in production lines and equipment. It also achieves image-level style adaptation through domain cues. In static image scenarios, this invention significantly improves mAP (5.2%~8.7%) when target domain data is scarce, and has high-precision specialization. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the feature decoupling domain adaptation model according to an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the overall two-stage training process in this invention. The first stage (left side) is feature decoupling pre-training, where the model utilizes data from multiple source domains and is jointly trained using feature decoupling (Dis.), alignment loss (Ali.), and detection loss (Detect). The second stage (right side) is domain cue adaptation. When faced with a target domain, the model freezes the backbone network and retains existing domain-invariant knowledge, incorporating new knowledge specific to the target domain only through a lightweight domain adaptation module, ultimately achieving efficient and robust cross-domain defect detection.

[0024] Figure 3 This invention demonstrates the effectiveness of its method by visualizing the feature distribution of different Transformer encoder layers and calculating the JS divergence: as the number of network layers increases (from Layer 1 to Layer 6), the JS divergence of feature distributions between different domains significantly decreases from 0.47 to 0.06, indicating that the model has successfully achieved cross-domain feature alignment at deeper layers, effectively extracting domain-invariant features, thereby significantly alleviating the domain offset problem and ensuring detection robustness in the target domain.

[0025] Figure 4This paper demonstrates the validation of the method of this invention on three cross-domain industrial defect datasets: electroluminescence (ELES), visible light (TTD), and infrared (MHHS). (a) Typical defect samples from different batches, cameras, and flight altitudes are shown, including cold solder joints, missing corners, and hot spots. (b) Performance comparison shows that the method of this invention (denoted as FDDA) is significantly better than baseline methods such as single-step adaptation and full fine-tuning in scenarios with domain shift. It effectively solves the problem of model performance degradation caused by differences in data distribution and verifies the strong generalization ability of the method on multi-source heterogeneous industrial data. Detailed Implementation

[0026] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments, but this is not intended to limit the scope of protection of this application.

[0027] Example 1:

[0028] The present invention provides a photovoltaic power plant detection method based on feature decoupling domain adaptation, comprising the following steps:

[0029] Step 1: Construct a dataset of potential hazards in multi-source domain renewable energy power plants

[0030] This study collects multimodal image data of new energy power plants, covering different production lines, equipment models, and production batches, including visible light, infrared, and electroluminescence images. Typical potential hazards in the images (such as photovoltaic hot spots, battery microcracks, and module stains) are finely annotated, including hazard category and location bounding boxes. Simultaneously, each sample is labeled with a domain tag (such as production line number, equipment model, and collection batch), constructing a multi-source domain training set with clearly defined domain attributes. This dataset must ensure domain diversity to simulate domain shift issues in real-world scenarios, providing a foundation for subsequent domain adaptation training.

[0031] Step 2: Establish a feature decoupling domain adaptation model. The feature decoupling domain adaptation model is based on the DETR detection framework and includes a backbone network for extracting image features, a Transformer encoder and a Transformer decoder for global context modeling and object query interaction, and a prediction head for outputting the detection result set. A domain prompt generation module TEPG is set between the backbone network and the Transformer encoder. The output of the Transformer encoder is connected to the Transformer decoder through a feature aligner, query filtering, and channel separation module TDCS.

[0032] The domain cue generation module TEPG includes 1×1 Conv, 3×3 DWConv, 1×1 Conv, 3×3 DWConv, cue filtering operation, and spatial pooling operation, and the target domain image features extracted by the backbone network. After 1×1 Conv processing, channel-adjusted features are obtained. These channel-adjusted features are then processed by 3×3 DWConv, 1×1 Conv, and 3×3 DWConv. The output of this process is added to the channel-adjusted features, and the resulting features are then used in the prompt selection operation. In this operation, the features are first activated by calculating importance weights using a sigmoid function. Then, a Top-K selection mechanism is used to select the few most critical feature elements. Finally, these selected features are used to generate domain prompts (Domain Prompts) representing the style of the target domain, i.e., the target domain prompts, through spatial pooling. This provides precise guidance for the rapid adaptation of the model;

[0033] The channel separation module TDCS includes a multilayer perceptron FFN-MLP, which queries domain-invariant objects obtained through query filtering. and domain-specific queries The target domain prompt obtained by the domain prompt generation module TEPG and domain-specific queries After vector concatenation, the target domain specific channels are adjusted through feedforward neural network-multilayer perceptron-MLP processing, and then queried with domain-invariant object queries. Concatenate the vectors to obtain the reconstructed query vector. ;

[0034] Reconstruct the query vector Querying specific domains The concatenated data is used as input to the Transformer decoder, and the detection result is output after processing by the prediction head.

[0035] Step 3: Two-Phase Training

[0036] Phase 1: Feature decoupling pre-training

[0037] Feature Decoupling Pre-training (FDP) refers to the process of initially training a model using multi-source domain data before formal target domain adaptation. The goal of pre-training is to enable the model to possess basic defect detection capabilities and to initially learn to decouple image features into domain-invariant features (common features applicable to all domains) and domain-specific features (style features related to a specific domain). The pre-training stage is not targeted at any particular target domain; instead, it enhances the model's robustness to cross-domain feature variations through joint training with multi-source domain data.

[0038] The DETR detection framework refers to DEtection TRansformer, an end-to-end object detection model based on the Transformer architecture. It abandons complex, hand-designed components such as anchor boxes and non-maximum suppression (NMS) found in traditional detection methods. Its core structure includes a CNN or ViT backbone network for extracting image features, a Transformer encoder-decoder structure for global context modeling and object query interaction, and a prediction head that directly outputs the detection result set. This embodiment uses DETR as the basic framework, leveraging its global attention mechanism and ensemble prediction capabilities, and introduces a feature decoupling mechanism to enhance its cross-domain robustness. Using DETR-based detection boxes as the model backbone, it includes a visual Transformer (ViT) backbone network, a Transformer encoder-decoder, and a prediction head. A feature aligner is added after the Transformer encoder, and object query separation is achieved in the decoder through channel slicing, realizing a two-branch feature decoupling mechanism. Domain-invariant feature extraction is then achieved by jointly optimizing the detection task and the feature separation task.

[0039] 1. Detecting loss ( The Hungarian matching algorithm is used to perform optimal binary matching between predictions and labels to determine the true target corresponding to each predicted bounding box. The classification loss is then calculated. (Cross-entropy function) and bounding box regression loss (GIoU distance) to ensure basic detection capabilities:

[0040] (1)

[0041] 2. Decoupling loss ( ):

[0042] Feature Alignment Loss: A feature aligner (FA) is set in the encoder output layer to minimize feature embeddings from different domains through adversarial training. Distribution differences. Define encoder alignment loss. for:

[0043] (2)

[0044] in It is an abbreviation for "Feature Aligner". For domain tags, This represents the number of spatial locations in the feature map. This represents the total number of layers in the encoder. Indicates the first Layer encoder at the Feature embeddings output from each spatial location.

[0045] Adversarial decoupling loss: In the decoder layer, object queries from the encoder output are processed through channel slicing. Separate to domain-invariant object query Domain-specific object queries At this point, the query filtering is located in the decoder. Specifically, the query vector... Split into two parts along the feature channel dimension: Used to learn the common characteristics of defects, Used to learn domain-related style features, i.e., domain-specific queries. Then, through a domain discriminator ( )maximize Domain distinctiveness. Define the decoder against loss. for:

[0046] (3)

[0047] here, For query vector The dimension; This represents the total number of layers in the decoder. For domain discriminators.

[0048] The total decoupling loss integrates feature alignment and adversarial decoupling through weight coefficients. balance, For encoder alignment loss, Decoder adversarial loss:

[0049] (4)

[0050] The final pre-trained objective function integrates detection and decoupling tasks, based on coefficients. Adjustment and optimization direction:

[0051] (5)

[0052] This invention decouples object queries from the decoder layer, enabling... Through the domain discriminator Maximize its domain discriminability to achieve feature separation in the core stage of the detection task. Focus on learning the essential characteristics of the defects in cross-domain generalization. Specifically captures domain-related style information, and ultimately adjusts only... It achieves efficient domain adaptation, breaking through the limitation of traditional methods that only perform coarse feature alignment at the encoder level.

[0053] After pre-training, the domain discriminator Remove the missing parts to obtain the pre-trained DETR detection framework.

[0054] Phase 2: Domain-Cue Adaptive DPA Training

[0055] Feature embedding first converts the original features extracted by the backbone network into a serialized embedding form, which is then input into the Transformer Encoder for deep semantic encoding.

[0056] For target domain data (such as new production line equipment), the pre-trained DETR detection framework (including the backbone network and Transformer core parameters) is frozen, and only the lightweight domain adaptation module is optimized to obtain the adapted model. The lightweight domain adaptation module includes the domain prompt generation module TEPG and the channel separation module TDCS. Specifically,

[0057] Domain Hint Generation Module (TEPG): Target domain image features extracted by the backbone network. Subsequently, after alternating processing with 1×1 Conv and depthwise separable convolution (DWConv), prompt filtering is performed. The specific process is: target domain image features... First, the flow passes through a 1×1 Conv for initial channel adjustment to obtain channel-adjusted features. Then, it enters a 3×3 DWConv to extract local style information. The extracted features are then fused and dimensionality-reduced through another 1×1 Conv. Finally, after processing by another 3×3 DWConv, the fused features are added to the channel-adjusted features to obtain the final fused features. The prompt filtering process is as follows: For fused features... Attention weights are calculated using the Sigmoid function, and the most significant cue features are selected using a Top-K selection mechanism. Finally, these selected pieces of information are used to generate Domain Prompts, which represent the style of the target domain, through space pooling operations. This provides precise guidance for the rapid adaptation of the model; the specific formula is defined as:

[0058] (6)

[0059] (7)

[0060] (8)

[0061] (9)

[0062] in, It is a 1×1 convolution; It is a 3×3 depth-separable convolution; Features containing local style information are extracted through depthwise separable convolution; Fusion features Perform a linear mapping; For the Sigmoid function; Select mechanism parameters for Top-K; This is the most prominent cue feature.

[0063] Channel Separation Module (TDCS): In the Channel Separation Module component, through the decoupling effect of Disentangle Loss, the channel is separated from the original query and finally included as part of the TDCS module output, along with the adapted domain-specific query. The concatenation is used together as input to the Transformer Decoder for final defect detection. Domain-invariant object lookup. (Based on a fixed query selection, the target domain prompt is obtained through a feedforward neural network-multilayer perceptron (FFN-MLP) combined with the domain prompt generation module TEPG.) Adjust target domain specific channels Then query the domain-invariant object. Concatenate the vectors to obtain the reconstructed query vector. for:

[0064] (10)

[0065] (11)

[0066] By using a small amount of target domain labeled data (approximately 1% to 5% of the source data volume), TEPG, TDCS, and the detection head are fine-tuned to achieve efficient cross-domain migration and lightweight domain adaptation.

[0067] Step 4: Deployment of online inspection systems for multiple production lines

[0068] The adaptive model is embedded into the intelligent inspection system of new energy power plants to construct a closed-loop optimization process. When the system detects an image of a new production line or a new equipment model, it automatically activates the second stage of the domain prompt update process, calling the pre-built TEPG module to generate a target domain prompt (a compact, vector-based feature representation generated by TEPG). Low-confidence suspected hazard samples output by the model are manually reviewed, and after confirmation, they are added to the target domain training set to gradually expand the diversity of target domain samples. During the system's idle period, the TDCS module and the detection head are automatically triggered for fine-tuning training, continuously optimizing the domain adaptation capability using accumulated target domain data to form a robust detection system that dynamically adapts to different production lines. The system can automatically use new data collected during actual operation to update TEPG, thereby dynamically optimizing model performance and achieving closed-loop online optimization.

[0069] Example 2:

[0070] This embodiment is based on a new energy power plant hidden danger detection method using Feature Decoupling Domain Adaptation (FDDA, see [link]). Figures 1-4 Specifically, it includes the following steps:

[0071] Step 1: Construct a dataset of potential hazards in multi-source domain renewable energy power plants

[0072] To achieve cross-domain robustness verification, a dataset containing multiple production lines, multiple equipment models, and multiple data collection conditions needs to be constructed, specifically divided as follows:

[0073] Given a multi-domain industrial defect detection dataset Each of the domains Includes images , mark ( For defect categories, For bounding box coordinates, For the first (Total number of defect categories in each image), domain label , For the number of fields, For the first Number of images in each domain.

[0074] Datasets are categorized by purpose into Feature Decoupling Pre-training (FDP), which includes... Source domain data, used for initial model training, Domain Hint Adaptation (DPA), target domain. A small amount of labeled data (1%~5% of the source domain data); test set, target domain Independent data.

[0075] Three types of experimental datasets:

[0076] The first type of ELES dataset consists of electroluminescence images from three production batches of a photovoltaic cell factory in northern China. It is divided into three groups: Group 1 (7,672 images), Group 2 (4,250 images), and Group 3 (1,302 images), containing five types of defects: microcracks (c=1), broken grids (c=2), black cores (c=3), poor solder joints (c=4), and contamination (c=5). When the target group is Group 3, the FDP dataset uses data from Group 1+2 (11,922 images), the DPA dataset uses 500 images from Group 3, and the test set uses a total of 1,000 images across all groups.

[0077] The second type of TTD dataset consists of visible light inspection images from a tile factory in southern China. It is divided into three domains based on the acquisition camera: Camera 1 (7,672 images), Camera 2 (4,250 images), and Camera 3 (1,302 images). It includes four types of defects: cracks (c=1), missing corners (c=2), color difference (c=3), and bubbles (c=4). Only 300 labeled images are used for each domain in the DPA stage.

[0078] The third type of MHHS dataset consists of infrared images of photovoltaic power plants collected by drones. It is divided into four domains according to flight altitude, with the following distributions at different flight altitudes: 28 meters (1,283 images), 35 meters (1,823 images), 47 meters (1,320 images), and 60 meters (2,722 images). Only hot spot defects are detected (c=1), and the test set contains 200 images in each domain.

[0079] Three types of domain random enhancement are applied to the FDP data: color dithering, randomly adjusting brightness (Δ∈[-0.2,0.2]), contrast (γ∈[0.8,1.2]), and saturation (σ∈[0.8,1.2]); spatial transformation, random rotation (θ∈[-15°,15°]) and cropping (scale ∈[0.8,1.0]); and domain obfuscation, with probability... Image features from two source domains ,in , Represents the features of the mixed image. is the coefficient in formula (5).

[0080] Step 2: Establish a feature decoupling domain adaptation model. The feature decoupling domain adaptation model is based on the DETR detection framework and includes a backbone network for extracting image features, a Transformer encoder and a Transformer decoder for global context modeling and object query interaction, and a prediction head for outputting the detection result set. A domain prompt generation module TEPG is set between the backbone network and the Transformer encoder. The output of the Transformer encoder is connected to the Transformer decoder through a feature aligner, query filtering, and channel separation module TDCS.

[0081] Step 3: Two-stage training

[0082] Feature decoupling pre-trained FDP, using the DETR detection framework, includes a ViT backbone network, a Transformer encoder / decoder, a decoupling module, and a detection head. The specific implementation is as follows:

[0083] 1. Backbone Network

[0084] Input image Segmented by Patch Embedding piece( ),pass The layered Transformer encoder obtains global features. ,

[0085] in, For sequence length, Number of channels; , These represent the height and width of the image, respectively.

[0086] 2. Decoupling module

[0087] Encoder Alignment: A Feature Aligner (FA) is inserted after each encoder layer, which consists of 3 layers of MLP (dimensionality...) The output domain is composed of the discrimination score. The loss function is defined as:

[0088] (12)

[0089] in The goal of this optimization is to make the Feature Analyzer unable to distinguish the domain from which the feature originates, with the domain being the domain label. This represents the number of spatial locations in the feature map. This represents the total number of layers in the encoder. Indicates the first Layer encoder at the Feature embeddings output from each spatial location.

[0090] Decoder decoupling: Object querying from encoder output Split into domain-invariant queries Used for common defect characteristics, domain-specific queries (Used for domain-related features). Maximize the domain discriminant QD (same structure as FA). Domain discriminability, the decoder loss function is defined as:

[0091] (13)

[0092] here, For query vector The dimension; This represents the total number of layers in the decoder. For domain discriminators.

[0093] 3. Joint optimization

[0094] The overall loss function integrates the detection task and the decoupling objective:

[0095] (14)

[0096] in Control the intensity of the confrontation. Balance the task weights and train for 100 epochs (batch size=16).

[0097] Domain-cued adaptive DPA training:

[0098] For the target domain Perform a lightweight adaptation:

[0099] 1. Domain hint generation

[0100] Target domain image features extracted by backbone network First, a 1×1 convolution is used for channel adjustment. Then, local style information is extracted by sequentially passing it through a 3×3 depthwise separable convolution (DWConv), a 1×1 convolution, and another 3×3 DWConv. This extracted style information is then added to the channel-adjusted features to obtain the fused features. Then, filter the operation using the prompt: [The text abruptly ends here, likely due to an incomplete sentence or a formatting Attention weights are calculated using the Sigmoid function, and then the most significant cue features are extracted using a Top-K selection mechanism. Then, through spatial pooling, a target domain prompt is generated. :

[0101] (15)

[0102] (16)

[0103] (17)

[0104] (18)

[0105] 2. Query adaptation

[0106] freeze Adjusting domain-specific queries through FFN-MLP :

[0107] (19)

[0108] in, This represents vector concatenation, and the MLP structure is as follows: .

[0109] 3. Fine-tuning strategy

[0110] Only the TEPG, TDCS, and detector head parameters are optimized. The dataset is trained for 12 epochs using DPA data (learning rate 5e-5), and the loss function is:

[0111] (20)

[0112] This is the total loss function for the DPA phase; This is the classification loss, used to measure the difference between the predicted class and the true class; This is the bounding box regression loss, used to measure the positional difference between the predicted bounding box and the true bounding box.

[0113] Step 4: Deployment of the online detection system

[0114] The deployment process includes: real-time detection and dynamic updating. Real-time monitoring of the input target domain image. Load the pre-trained DETR detection framework and lightweight domain adaptation module to output defect detection results. :

[0115] (twenty one)

[0116] in These are the parameters for the frozen backbone.

[0117] Dynamically update low-confidence samples ( (This triggers manual review; once confirmed, it is added to the target domain dataset.) The system automatically starts incremental training during idle periods.

[0118] (twenty two)

[0119] in The momentum coefficient enables smooth updates of the cue vector; The updated target domain prompt; This is the momentum coefficient, used to control the weight ratio between historical prompts and new prompts to achieve smooth updates.

[0120] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A photovoltaic power plant detection method based on feature decoupling domain adaptation, characterized in that, The method includes the following: Construct a dataset of potential hazards in multi-source domain new energy power plants; A feature decoupling domain adaptation model is established. The feature decoupling domain adaptation model is based on the DETR detection framework and includes a backbone network for extracting image features, a Transformer encoder and a Transformer decoder for global context modeling and object query interaction, and a prediction head for outputting the detection result set. A domain prompt generation module TEPG is set between the backbone network and the Transformer encoder. The output of the Transformer encoder is connected to the Transformer decoder through a feature aligner, query filtering, and channel separation module TDCS. The domain prompt generation module TEPG includes 1×1 Conv, 3×3 DWConv, 1×1 Conv, 3×3 DWConv, prompt filtering operation, and spatial pooling operation, and the target domain image features extracted by the backbone network. After 1×1 Conv processing, channel-adjusted features are obtained. These channel-adjusted features are then processed by 3×3 DWConv, 1×1 Conv, and 3×3 DWConv. The output of this process is added to the channel-adjusted features, and the resulting feature enters the prompt selection operation. In the prompt selection operation, the features are first activated by calculating importance weights using a sigmoid function. Then, a Top-K selection mechanism is used to select the few most critical feature elements. Finally, these selected information are used to obtain the target domain prompt through spatial pooling. ; The channel separation module TDCS includes a multilayer perceptron FFN-MLP, which queries domain-invariant objects obtained through query filtering. and domain-specific queries The target domain prompt obtained by the domain prompt generation module TEPG and domain-specific queries After vector concatenation, the target domain specific channels are adjusted through feedforward neural network-multilayer perceptron-MLP processing, and then queried with domain-invariant object queries. Concatenate the vectors to obtain the reconstructed query vector. ; Reconstruct the query vector Querying specific domains The concatenated data is used as input to the Transformer decoder, and the detection result is output after processing by the prediction head. Two-stage training: Phase 1: The DETR detection framework, consisting of a backbone network, a Transformer encoder, a Transformer decoder, and a prediction head, is pre-trained using a multi-source domain new energy power plant hidden danger dataset. Image features are decoupled into domain-specific features and domain-invariant features through adversarial learning and feature alignment loss. During training, detection loss and decoupling loss are set to obtain the pre-trained DETR detection framework. Phase 2: For the target domain data, freeze the core parameters of the DETR detection framework and optimize only the lightweight domain adaptation module composed of the domain hint generation module TEPG and the channel separation module TDCS to obtain the adaptation model; The adapted model is embedded into the intelligent inspection system for new energy power plants for photovoltaic power plant inspection.

2. The method according to claim 1, characterized in that, The process of constructing the multi-source domain new energy power plant hidden danger dataset is as follows: collect multimodal image data of new energy power plants covering different production lines, acquisition equipment models and production batches, including visible light, infrared and electroluminescence images; perform fine annotation on typical hidden dangers in the images, including hidden danger category and location bounding box, and label each sample with domain labels to construct a multi-source domain training set with clear domain attributes.

3. The method according to claim 1, characterized in that, The backbone network is either CNN or VIT.

4. The method according to claim 1, characterized in that, The training loss function for the first stage is: ， in, Total loss; To detect the loss; β represents the decoupling loss; β is a coefficient that adjusts the optimization direction.

5. The method according to claim 1, characterized in that, When the intelligent inspection system for new energy power plants detects images of new production lines or new equipment models, it automatically activates the second-stage prompt update process of the domain, calls the preset domain prompt generation module TEPG to generate target domain prompts, manually reviews low-confidence suspected hidden danger samples output by the adaptation model, and adds them to the target domain training set after confirmation, gradually expanding the diversity of target domain samples; during the system's idle period, it automatically triggers the fine-tuning training of the TDCS module and the detection head, continuously optimizes the domain adaptation capability using the accumulated target domain data, and forms a robust detection system that dynamically adapts to different production lines.

6. The method according to claim 1, characterized in that, During the first stage of training, the DETR detection framework includes a domain discriminator. The domain discriminator and the feature aligner have the same structure, both consisting of a three-layer MLP. Through the collaborative optimization of the feature aligner and the domain discriminator, cross-domain feature embedding alignment is achieved at the encoder layer, while the object query channel is separated at the decoder layer, enabling the model to autonomously distinguish and retain domain-invariant features that are key to defect detection.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it can implement the method of claim 1.

Images