Electric power unmanned aerial vehicle inspection defect detection method and system based on target incremental learning

Abstract

The embodiment of the invention provides an electric power unmanned aerial vehicle inspection defect detection method and system based on target incremental learning, and belongs to the technical field of computer vision, deep learning and intelligent operation and maintenance of electric power equipment. The method comprises the following steps: constructing and training an accurate retrieval model based on multi-modal feature fusion; constructing an initial defect detection model based on multi-scale information fusion; when new inspection task data is received, key historical samples are retrieved by using the accurate retrieval model; performing updating training on the defect detection model by adopting a hybrid incremental learning strategy in combination with the new task data and the key historical sample; and performing defect detection by using the updated defect detection model. The method and the system can continuously adapt to new defect types and new inspection environments, effectively relieve disastrous forgetting, and comprehensively utilize multi-mode and multi-scale information to realize high-precision and high-efficiency retrieval and detection of electrical equipment defects.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for power equipment, specifically to a method and system for detecting defects in power equipment using unmanned aerial vehicles (UAVs) based on target incremental learning. Background Technology

[0002] In the smart grid operation and maintenance system, drone inspection has become the mainstream method for troubleshooting power equipment due to its flexibility and efficiency. It can quickly collect multimodal inspection data such as high-resolution images, infrared videos, and equipment sensor data. However, the existing technology system has the following key technical bottlenecks: 1. The catastrophic forgetting problem of incremental learning: The types of defects in power grid equipment continue to increase with the years of operation and environmental changes. When learning new defect types, traditional detection models are prone to forgetting the old defect identification capabilities they have mastered, making them unable to adapt to the dynamic task requirements of power scenarios. Among the existing single incremental learning methods, the replay method has the drawback of high storage costs, the regularization method is limited in effect when there are large differences in cross-task distribution, and the parameter isolation method will lead to an exponential increase in the number of model parameters, making it difficult to balance anti-forgetting performance and resource consumption.

[0003] 2. The challenge of retrieving and matching massive inspection data: The volume of multimodal data generated by UAV inspections is growing explosively. Traditional retrieval methods rely on only single modal features for matching, which has problems such as single feature dimension, low matching accuracy, and long retrieval time. It is difficult to quickly locate key data with defects and it is difficult to support the supply of high-quality samples for incremental learning.

[0004] 3. Shortcomings in the detection accuracy of multi-scale defects: Defects in power equipment have significant multi-scale characteristics. Global faults in large-sized equipment (such as tower deformation) and minor damage to small-sized components (such as micro-cracks in insulation layers and wear of wire filaments) require feature support at different scales. Existing detection models mostly adopt a single-scale feature extraction architecture, which is prone to losing local details or global correlation information, resulting in the problem of missed detection of small defects and high false detection rate of large targets. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting defects in power grid unmanned aerial vehicle (UAV) inspections based on target incremental learning. This method and system can achieve real-time and high-precision detection of inspection defects, significantly improving the intelligence level and safety assurance capabilities of power grid operation and maintenance.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for defect detection in power plant unmanned aerial vehicle (UAV) inspections based on target incremental learning, the method comprising: Construct and train a precise retrieval model based on multimodal feature fusion; Construct an initial defect detection model based on multi-scale information fusion; When new inspection task data is received, the precise retrieval model is used to retrieve key historical samples; By combining the new task data with the key historical samples, a hybrid incremental learning strategy is used to update and train the defect detection model. Defect detection is performed using the updated defect detection model.

[0007] Preferably, the construction of the accurate retrieval model includes: Extract visual features from inspection images / videos, semantic features from associated text, and temporal features from sensor data; The visual features, semantic features, and temporal features are fused using a cross-modal attention mechanism to generate a joint feature representation; Based on the joint feature representation, a deep metric learning model is trained to perform similarity retrieval based on text or images.

[0008] Preferably, the construction of the initial defect detection model includes: Using a backbone network with a feature pyramid structure, multi-scale features are extracted from the input image. The multi-scale features are adaptively weighted and fused using channel attention and spatial attention mechanisms. Based on the fused features, the output predicts the category and location of defects.

[0009] Preferably, the hybrid incremental learning strategy includes at least knowledge distillation based on playback and regularization based on parameter importance, wherein, The knowledge distillation based on playback involves using the key historical samples to constrain the model's output on samples to ensure consistency with the old model's output. The parameter importance-based regularization limits the update magnitude of model parameters during training on new tasks based on their importance on old tasks.

[0010] Preferably, the regularization based on parameter importance is specifically an elastic weight solidification method, which uses the diagonal elements of the Fisher information matrix of the parameters as importance weights.

[0011] Preferably, the update training step further includes dynamically adding a task-specific trainable adapter module to the defect detection model based on the distribution differences between the new task and the historical task, wherein the adapter module is trained in conjunction with the original parameters of the model.

[0012] On the other hand, the present invention provides a power plant unmanned aerial vehicle (UAV) inspection defect detection system based on target incremental learning, for implementing the above method, the system comprising: A multimodal data processing and retrieval module is used to perform the training and sample retrieval functions of the precise retrieval model; The multi-scale defect detection model module is used to identify and locate defects in the input inspection images; The incremental learning management module is used to call the multimodal data processing and retrieval module to obtain key historical samples when new task data is received, and to configure a hybrid incremental learning strategy to update the multimodal data processing and retrieval module. The real-time detection and feedback module is used to deploy models, perform online detection, and feed new data back to the system.

[0013] Preferably, the system further includes: The data preprocessing and labeling module is used to clean, denoise, and label defects in the raw inspection data. The model deployment and service module is used to deploy the updated detection model to edge devices or the cloud and provides a real-time defect detection API interface; The feedback loop module is used to collect new data and feedback generated by online detection and use it as input to trigger a new round of incremental learning tasks.

[0014] In another aspect, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method described above.

[0015] Furthermore, the present invention also provides a machine-readable storage medium storing instructions that cause a machine to perform the method described above.

[0016] Based on the above technical solution, this invention constructs a robust hybrid incremental learning framework by integrating playback-based knowledge distillation and EWC-based regularization, and introducing a lightweight adapter when necessary. This framework significantly suppresses the model's forgetting of old knowledge when learning new defects with limited computational and storage overhead, enabling the detection system to adapt to the dynamic changes in power grid inspection scenarios over the long term. Simultaneously, through multimodal feature fusion and depth metric retrieval, the system can accurately locate the most valuable samples (positive examples, difficult negative examples, and out-of-domain samples) for the current learning objective from a massive, heterogeneous historical data pool, providing a high-quality, high-information training set for incremental learning, significantly improving the efficiency of model optimization and the final performance ceiling. Furthermore, by combining a feature pyramid and a dual attention mechanism in an adaptive multi-scale fusion network, the model can fully coordinate and utilize features at different levels to capture the global structure of large targets and the subtle textures of small targets, thereby comprehensively improving the detection rate and localization accuracy of defects at various scales. This seamless integration of precise retrieval, hybrid incremental learning, and high-precision detection forms a complete closed loop from data management, knowledge retrieval, model optimization to online applications. It not only achieves current high-precision detection but also possesses the ability to continuously learn and adapt autonomously for the future, powerfully promoting the development of power grid operation and maintenance towards intelligence and autonomy.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the defect detection method for power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning provided by the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0021] See Figure 1 The first aspect of this invention provides a defect detection method for power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning, the method comprising: Construct and train a precise retrieval model based on multimodal feature fusion; Construct an initial defect detection model based on multi-scale information fusion; When new inspection task data is received, the precise retrieval model is used to retrieve key historical samples closely related to the current task from historical data. By combining the new task data with the key historical samples, a hybrid incremental learning strategy is used to update and train the defect detection model. By utilizing the updated defect detection model to perform defect detection on new task data and historical tasks, the model's performance can be continuously improved.

[0022] In this embodiment, the construction of the accurate retrieval model includes: Convolutional neural networks are used to extract visual features from inspection images / videos, natural language processing models are used to extract semantic features from associated text (such as inspection reports and equipment nameplate information), and temporal models (such as LSTM and Transformer) are used to extract temporal features from sensor data. The visual features, semantic features, and temporal features are deeply fused through a cross-modal attention mechanism to generate a unified joint feature representation rich in contextual information. Based on the joint feature representation, a deep metric learning model (such as a model based on contrastive learning or triple loss) is trained, enabling the model to efficiently and accurately retrieve the most semantically or visually similar samples from the feature library based on text or image queries.

[0023] In this embodiment, the construction of the initial defect detection model includes: Use a backbone network with a feature pyramid structure (such as FPN+ResNet or EfficientDet) to extract multi-level, multi-scale feature maps from the input image; The multi-scale features are adaptively weighted using channel attention mechanisms (such as the SE module) and spatial attention mechanisms (such as the spatial attention module in CBAM) to highlight channels and spatial regions that are important for defect identification. Then, the weighted features are fused across scales. Based on the fused strong discriminative features, the detection head (such as RCNN head or YOLO head) is used to output the category confidence of the defect and the accurate location bounding box prediction.

[0024] Furthermore, the aforementioned hybrid incremental learning strategy includes at least replay-based knowledge distillation and parameter importance-based regularization, wherein... The knowledge distillation based on playback is as follows: when training a new task, the retrieved key historical samples are input into the model to be updated and the saved old model. By designing a distillation loss function (such as KL divergence), the output prediction of the model for these historical samples in the update is constrained to be as consistent as possible with the output of the old model, so as to "distill" the old knowledge into the new model and prevent forgetting.

[0025] The parameter importance-based regularization is as follows: before training a new task, evaluate the importance of each parameter in the model to the learned old task (e.g., by measuring the degree of influence of the parameter change on the old task loss function), add a regularization term to the loss function of training the new task, impose stronger update constraints on parameters with high importance, limit their deviation from the optimal value of the old task, and thus protect the learned knowledge.

[0026] In this embodiment, the parameter importance-based regularization specifically employs the Elastic Weight Solidification (EWC) method, which uses the diagonal elements of the Fisher information matrix of a parameter as its importance weight. Parameters with high importance will be subject to greater penalties when training on new tasks to limit their updates.

[0027] Furthermore, the preferred update training steps also include dynamically adding lightweight, task-specific trainable adapter modules to the defect detection model based on the distribution differences between the new task and historical tasks. These adapter modules are inserted after certain layers of the backbone network. During incremental learning, most of the original model parameters are frozen or slightly adjusted, while the model adapts to the new task primarily by learning the adapter modules. This approach achieves efficient isolation and fusion of knowledge, controlling the growth of the total number of model parameters.

[0028] To implement the above method, a second aspect of the present invention provides a power plant unmanned aerial vehicle (UAV) inspection defect detection system based on target incremental learning, the system comprising: The multimodal data processing and retrieval module is used to perform the training and sample retrieval functions of the precise retrieval model and manage the multimodal feature library; The multi-scale defect detection model module is the core detection unit, which integrates the multi-scale information fusion mechanism to perform real-time, high-precision defect identification and localization on the input inspection images. The incremental learning management module, acting as the "brain" of the system, coordinates the multimodal data processing and retrieval module to obtain key historical samples when new task data is received. It then uses a hybrid incremental learning strategy (knowledge distillation, regularization, and adapter) to safely and efficiently update the multi-scale defect detection model module based on the task configuration. The real-time detection and feedback module is used to deploy the updated model to the actual environment (such as the onboard computing unit of a drone or an edge server), perform online detection tasks, and feed the generated new data and detection results back to the system.

[0029] Furthermore, the aforementioned system also includes: The data preprocessing and labeling module is used to automatically clean, denoise, enhance, and semi-automatically / automatically label raw inspection data, providing a high-quality data foundation for model training. The model deployment and service module provides a toolchain for model compression, conversion, and deployment, supporting the deployment of optimized detection models to diverse hardware platforms and providing standardized API interfaces for other systems to call. The feedback loop module is responsible for collecting new data generated by online detection, false positive / false negative cases, and feedback from maintenance personnel. It evaluates and filters these data, and qualified data will trigger the system to start a new round of incremental learning tasks, forming a continuous optimization loop of "data-model-application-feedback".

[0030] In summary, this invention firstly employs a hybrid incremental learning strategy that combines knowledge distillation based on playback with elastic weight solidification regularization. Simultaneously, it introduces a lightweight adapter module. This approach achieves efficient retention of old knowledge through knowledge distillation, restricts key parameter updates through parameter regularization, and adapts to new task distributions through the adapter module. It balances anti-forgetting performance with model lightweighting, effectively solving the catastrophic forgetting problem of traditional models when learning new defects. It can dynamically adapt to the task requirements of continuously increasing types of power grid defects.

[0031] Secondly, the multimodal feature fusion accurate retrieval model constructed in this invention breaks through the limitations of traditional single-modal retrieval. It achieves deep fusion of visual, semantic, and temporal features through a cross-modal attention mechanism, which significantly improves the retrieval accuracy and efficiency of key historical samples. At the same time, only a small number of highly similar key samples need to be retrieved to support incremental learning, which greatly reduces the storage cost and retrieval time of historical samples, and provides an efficient sample supply solution for incremental learning.

[0032] Furthermore, the initial defect detection model of this invention adopts a feature pyramid structure combined with a channel-space dual attention mechanism, which can simultaneously extract and fuse shallow detail features and deep semantic features, thereby achieving accurate detection of large-size global defects and small-size minute defects. This effectively reduces the false detection rate of small defects and the false detection rate of large targets, and significantly improves the overall detection accuracy of power equipment defects.

[0033] Finally, the detection system of this invention constructs a complete feedback loop, which can use new data from online detection and manual feedback as trigger inputs for incremental learning, realize continuous iterative optimization of the model, and enable the model to continuously improve its detection capabilities as power grid operation and maintenance data accumulates, providing continuous technical support for the long-term stable operation and maintenance of the power grid.

[0034] The following provides a specific embodiment to illustrate the present invention: Example The first step in this embodiment is to use a real UAV inspection dataset from a provincial power grid company. This dataset contains more than 100,000 images of power equipment inspection, more than 5,000 infrared temperature measurement videos, corresponding equipment ledger text data and sensor time series data, covering 15 types of power equipment defects such as insulator self-explosion, conductor strand breakage, tower deformation, bolt loosening, and equipment overheating. The first 10 types are old defect types, and the last 5 types are newly added defect types, which are used to simulate the task scenario of incremental learning.

[0035] The second step is to build and train a precise retrieval model: 1. Multimodal feature extraction Visual features: The visual features of the inspection images are extracted using a pre-trained ViT-B / 16 model. The images are divided into blocks and input into the model. The output corresponding to the [CLS] label of the model is taken as the visual feature vector of the image, with a dimension of 768.

[0036] Semantic features: The pre-trained BERT-base model is used to extract the semantic features of the equipment ledger text. The text is segmented and encoded before being input into the model. The output of the [CLS] tag is taken as the semantic feature vector with a dimension of 768.

[0037] Temporal features: The TCN network is used to extract the temporal features of the sensing time series data. The sensing data includes five types of time series, such as device temperature, voltage, and current. After the sequence is normalized, it is input into the TCN, and the output of the last layer of the network is taken as the temporal feature vector with a dimension of 256.

[0038] 2. Cross-modal feature fusion A cross-modal attention fusion module is constructed. Visual features, semantic features, and temporal features are concatenated and input into the attention module. The module calculates the attention weights between different modal features and performs weighted fusion of each modal feature to generate a joint feature representation with a dimension of 1024.

[0039] 3. Training of deep metric learning models Based on joint feature representation, a deep metric learning model is trained using TripletLoss as the loss function. During training, each batch contains one anchor sample, one positive sample (sample of the same defect type), and one negative sample (sample of different defect types). By optimizing TripletLoss, the feature distance between the anchor sample and the positive sample is made smaller than the feature distance between the anchor sample and the negative sample. After training, the model can achieve accurate similarity retrieval based on text or images.

[0040] The third step is to construct and train the initial defect detection model: 1. Multi-scale feature extraction ResNet50 is used as the backbone network, and multi-scale features are extracted by combining the FPN feature pyramid structure. The outputs of C2, C3, C4 and C5 layers of ResNet50 correspond to feature maps of 1 / 4, 1 / 8, 1 / 16 and 1 / 32 respectively. FPN generates fused feature maps of four scales, P2, P3, P4 and P5, through top-down upsampling and lateral connections.

[0041] 2. Dual attention feature fusion After the FPN output, the channel attention module (SE module) and spatial attention module are sequentially connected: the SE module assigns weights to each channel feature through a squeeze-excitation operation; the spatial attention module generates a spatial attention mask by performing average pooling and max pooling on the feature map, and then weights the feature map. Adaptive fusion of multi-scale features is achieved through these dual attention modules.

[0042] 3. Detection head training The YOLOv8 detection head is used to classify and locate defects based on the fused multi-scale features. The loss function is a weighted sum of classification loss (CIoULoss), confidence loss (BCELoss), and class loss (CrossEntropyLoss). After training on the old defect dataset, the initial defect detection model is obtained.

[0043] Step 4: Update the incremental learning model: 1. Key historical sample retrieval When new inspection task data containing 5 new types of defects is received, the multimodal joint features of the new data are input into the accurate retrieval model to retrieve 20 high-similarity key samples for each type of old defect from the old defect sample library, for a total of 200 key historical samples.

[0044] 2. Hybrid Incremental Learning Training Knowledge distillation based on playback: The initial defect detection model is used as the teacher model. During the training process, in addition to fitting the true labels of the new task data and key historical samples, the new model also needs to fit the output probability distribution of the teacher model for key historical samples. The temperature coefficient of knowledge distillation is set to 3, and the distillation loss weight is set to 0.3.

[0045] Elastic weight solidification regularization: The diagonal elements of the Fisher information matrix of the initial model parameters are used as importance weights. A weight threshold is set. For parameters with weights higher than the threshold, their update magnitude is limited to 10% of the original magnitude, while the remaining parameters are updated normally.

[0046] Adding an adapter module: Add a lightweight adapter module (consisting of two 1×1 convolutional layers and an activation function) after the outputs of layers C3, C4, and C5 of ResNet50. The parameters of the adapter module are used in training, and the original model parameters are updated according to the regularization rules.

[0047] After training, the updated defect detection model is obtained.

[0048] The final step is to verify the test results: The updated model was compared with the traditional static model and the single incremental learning model. The test metrics included the old defect detection accuracy (mAP@0.5), the new defect detection accuracy (mAP@0.5), the number of model parameters, and the retrieval time. The results are shown in Table 1 below: Table 1

[0049] Results Analysis: Traditional static models show a significant drop in the accuracy of old defect detection after learning new defects, reaching only 42.3%, indicating a serious catastrophic forgetting problem. While single replay incremental models retain some old defect detection capabilities, retrieval is time-consuming, and the accuracy of new defect detection decreases slightly. Single regularized incremental models have high accuracy in old defect detection, but their ability to adapt to new defects is insufficient. The model of this invention achieves an old defect mAP@0.5 of 88.7% and a new defect mAP@0.5 of 82.5%, while the number of model parameters only increases slightly to 47.8M, and the single sample retrieval time is only 36ms. It balances the accuracy of new and old defect detection, model lightweightness, and retrieval efficiency, and its overall performance is significantly better than other comparative models.

[0050] In addition, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0051] Furthermore, the present invention provides a machine-readable storage medium storing instructions that cause a machine to perform the methods described above.

[0052] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0053] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0057] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0058] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0059] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0060] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A defect detection method for power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning, characterized in that, The method includes: Construct and train a precise retrieval model based on multimodal feature fusion; Construct an initial defect detection model based on multi-scale information fusion; When new inspection task data is received, the precise retrieval model is used to retrieve key historical samples; By combining the new task data with the key historical samples, a hybrid incremental learning strategy is used to update and train the defect detection model. Defect detection is performed using the updated defect detection model.

2. The method for defect detection in power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning according to claim 1, characterized in that, The construction of the accurate retrieval model includes: Extract visual features from inspection images / videos, semantic features from associated text, and temporal features from sensor data; The visual features, semantic features, and temporal features are fused using a cross-modal attention mechanism to generate a joint feature representation; Based on the joint feature representation, a deep metric learning model is trained to perform similarity retrieval based on text or images.

3. The method for defect detection in power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning according to claim 1, characterized in that, The construction of the initial defect detection model includes: Using a backbone network with a feature pyramid structure, multi-scale features are extracted from the input image. The multi-scale features are adaptively weighted and fused using channel attention and spatial attention mechanisms. Based on the fused features, the output predicts the category and location of defects.

4. The defect detection method for power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning according to claim 1, characterized in that, The hybrid incremental learning strategy includes at least replay-based knowledge distillation and parameter importance-based regularization, wherein... The knowledge distillation based on playback involves using the key historical samples to constrain the model's output on samples to ensure consistency with the old model's output. The parameter importance-based regularization limits the update magnitude of model parameters during training on new tasks based on their importance on old tasks.

5. The defect detection method for power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning according to claim 4, characterized in that, The parameter importance-based regularization is specifically an elastic weight solidification method, which uses the diagonal elements of the Fisher information matrix of the parameters as importance weights.

6. The method for defect detection in power plant unmanned aerial vehicle (UAV) inspection based on target incremental learning according to claim 1, characterized in that, The update training step also includes dynamically adding a task-specific trainable adapter module to the defect detection model based on the distribution differences between the new task and the historical task. The adapter module is trained in conjunction with the original parameters of the model.

7. A power plant unmanned aerial vehicle (UAV) inspection defect detection system based on target incremental learning, used to implement the method described in any one of claims 1-6, characterized in that, The system includes: A multimodal data processing and retrieval module is used to perform the training and sample retrieval functions of the precise retrieval model; The multi-scale defect detection model module is used to identify and locate defects in the input inspection images; The incremental learning management module is used to call the multimodal data processing and retrieval module to obtain key historical samples when new task data is received, and to configure a hybrid incremental learning strategy to update the multimodal data processing and retrieval module. The real-time detection and feedback module is used to deploy models, perform online detection, and feed new data back to the system.

8. The power plant unmanned aerial vehicle (UAV) inspection defect detection system based on target incremental learning according to claim 7, characterized in that, The system also includes: The data preprocessing and labeling module is used to clean, denoise, and label defects in the raw inspection data. The model deployment and service module is used to deploy the updated detection model to edge devices or the cloud and provides a real-time defect detection API interface; The feedback loop module is used to collect new data and feedback generated by online detection and use it as input to trigger a new round of incremental learning tasks.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-6.

10. A machine-readable storage medium storing instructions thereon, characterized in that, This instruction is used to cause the machine to perform the method as described in any one of claims 1-6.

Images