Substation inspection incremental training method and medium
By introducing a pseudo-label and cross-attention module for incremental training in substation inspection, the problems of low model update efficiency and poor adaptability in substation inspection are solved. This achieves efficient learning of new equipment categories and stability of the detection capability of learned equipment, thereby improving the model's adaptability and detection performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HEFEI ZHONGKE LEINAO INTELLIGENCE TECH CO LTD
- Filing Date
- 2026-01-20
- Publication Date
- 2026-05-12
AI Technical Summary
In substation inspection, existing incremental learning methods suffer from problems such as high computational resource consumption, long training cycles, complex model update processes, and difficulty in adapting to changes in equipment type and environmental conditions, leading to decreased model detection performance and catastrophic forgetting.
By acquiring incremental inspection image data, generating pseudo-labels, constructing incremental training tasks, and introducing a cross-attention module into the basic detection model, fine-tuning the training using task prompt vectors and instance prompt vectors, restricting shared parameter updates, and combining pseudo-labels with a category mapping update mechanism, we can achieve effective learning of new device categories and stability of the detection capability of already learned devices.
It enables efficient and reliable model updates without relying on replaying historical data, improves adaptability to new equipment categories and complex inspection scenarios, reduces computational resource consumption, shortens the training cycle, and maintains model stability and detection performance.
Smart Images
Figure CN121544988B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to an incremental training method and medium for substation inspection. Background Technology
[0002] In intelligent operation and maintenance scenarios for critical infrastructure such as substation inspections, target detection models typically need to continuously adapt to changes in the field environment and the constant updates in equipment types after initial training. As the inspection cycle progresses, new inspection image data will be continuously generated on-site, and new types of equipment or components that have not appeared before may be introduced, placing higher demands on the model's continuous learning capabilities.
[0003] In related technologies, a common approach is to merge newly added inspection data with historical data and then retrain the detection model. While this method can maintain the overall performance of the model to some extent, it requires repeatedly loading the complete historical dataset and performing multiple rounds of training, resulting in significant computational resource consumption and a long training cycle. This makes it difficult to meet the requirements of substation inspection tasks for update efficiency and real-time performance.
[0004] To reduce the cost of retraining, some studies have attempted to introduce incremental or continuous learning mechanisms, enabling models to update without fully relying on historical data. However, these incremental target detection methods still have significant shortcomings in practical substation inspection applications. On the one hand, many methods still rely on the storage and replay of historical samples, increasing storage and management burdens in large-scale inspection data scenarios, making model update processes complex and difficult to deploy stably in engineering systems. On the other hand, many incremental detection methods for related categories assume a relatively stable detection environment. When both equipment type and environmental conditions change simultaneously in the inspection scenario, the model is prone to performance degradation in detecting learned equipment, or even catastrophic forgetting, affecting the model's reliability in long-term operation. Furthermore, domain-adaptive detection methods designed for environmental changes typically only focus on feature distribution transfer, making it difficult to effectively learn new equipment categories while adapting to environmental changes, thus failing to meet the practical needs of substation inspections where equipment updates and scenarios are constantly changing. Summary of the Invention
[0005] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the objective of this invention is to propose an incremental training method and medium for substation inspection, which, while reducing model update costs, achieves effective learning of newly added equipment categories and maintains the stability of the detection capabilities of already learned equipment.
[0006] To achieve the above objectives, a first aspect of the present invention proposes an incremental training method for substation inspection, comprising:
[0007] Acquire incremental inspection image data, label the newly added equipment categories in the incremental inspection image data, and generate pseudo-labels for labeling the learned equipment categories in the incremental inspection image data;
[0008] Based on the incremental inspection image data, the newly added equipment category annotation results, and pseudo-labels, an incremental training task is constructed.
[0009] A prompting mechanism is constructed based on the incremental training task, generating task prompt vectors and instance prompt vectors for the incremental training task. A cross-attention module is introduced before the detection head of the basic detection model to fuse the task prompt vectors and instance prompt vectors into the cross-attention module in an attention manner. The basic detection model is a target detection model trained based on labeled initial inspection image data.
[0010] Under the condition that at least some parameters in the basic detection model are shared for updates, the cross-attention module and the detection head are fine-tuned using the incremental training task, and the category mapping and model parameters of the basic detection model are updated.
[0011] In addition, the substation inspection incremental training method of the above embodiments of the present invention may also have the following additional technical features:
[0012] According to one embodiment of the present invention, before constructing the incremental training task, the method further includes:
[0013] Load the model parameters from the previous incremental training phase as the initial model parameters for the current incremental training task, so as to inherit the shared parameters, task parameters and cue parameters learned in the previous incremental task during the current incremental training process.
[0014] According to one embodiment of the present invention, generating the instance hint vector includes:
[0015] Extract the global image features from the incremental inspection image data;
[0016] Based on the global features of the image, a vector that matches the environmental features of the incremental inspection image is dynamically generated and used as the instance prompt vector.
[0017] According to one embodiment of the present invention, the step of fusing the task cue vector and the instance cue vector into the cross-attention module in an attention manner includes:
[0018] The task prompt vector and the instance prompt vector are weighted and averaged to generate a composite prompt vector;
[0019] Perform a linear transformation on the composite hint vector to split it into a key vector and a value vector;
[0020] The key vector and the value vector are used as prefixes for the key and value, respectively, and concatenated into the key matrix and value matrix of the corresponding layer of the detection head.
[0021] According to one embodiment of the present invention, updating the model parameters of the basic detection model includes:
[0022] Based on the initial shared parameters of the basic detection model, the historical shared vector formed by the previous incremental task and the current shared vector formed by the current incremental training task are calculated respectively.
[0023] For the shared parameters corresponding to different network depths in the basic detection model, calculate the corresponding layer retention factor and adaptation factor;
[0024] The historical shared vector and the current shared vector are weighted and fused using the hierarchical retention factor and the adaptation factor to obtain the updated shared parameters.
[0025] According to an embodiment of the present invention, updating the model parameters of the basic detection model includes:
[0026] The task parameters already learned in the previous incremental task are retained, and the task parameters corresponding to the newly added device category are incrementally concatenated with the task parameters already learned in the previous incremental task.
[0027] According to one embodiment of the present invention, updating the model parameters of the basic detection model further includes:
[0028] Based on the contribution of each component in the cue vector to the category discrimination, the corresponding key cue components are selected from the historical cue vectors of the previous incremental task and the current cue vector generated by the current incremental task.
[0029] A consistency judgment is made between the historical key prompt component in the historical prompt vector and the current key prompt component in the current prompt vector;
[0030] Based on the consistency judgment result, the consistent key prompt components are fused, and the inconsistent key prompt components inherit the historical key prompt components to obtain the fused prompt parameters.
[0031] The prompt vectors include task prompt vectors and instance prompt vectors.
[0032] According to one embodiment of the present invention, a composite loss function is used for constraint during fine-tuning training. The composite loss function includes a main detection loss to ensure the accuracy of new equipment, and an improved distillation loss, orientation consistency loss, and sparsity loss to maintain the consistency of learned equipment features.
[0033] According to one embodiment of the present invention, the method further includes:
[0034] After completing the incremental training task, the updated basic detection model is evaluated for detection performance based on the complete set of categories, including the learned device categories and the newly added device categories, to verify the effectiveness of the incremental training.
[0035] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the steps of the above-described substation inspection incremental training method.
[0036] The substation inspection incremental training and media of this invention introduces a feature regulation method based on a prompting mechanism during incremental training, and performs targeted fine-tuning of the detection head under the condition of limiting the shared parameter update range in the basic detection model. This enables the model to complete incremental updates without relying on the replay of historical inspection samples. On the one hand, the pseudo-label and category mapping update mechanism ensures the continuous maintenance of the detection capability of learned equipment categories, mitigating the catastrophic forgetting problem. On the other hand, task prompt vectors and instance prompt vectors guide the model to focus on key features related to the current inspection task and environment, improving the model's adaptability to new equipment categories and complex inspection scenarios. Simultaneously, the collaborative update strategy combining shared parameters, task parameters, and prompt parameters effectively balances model stability and plasticity, enabling the detection model to achieve efficient and reliable continuous evolution in long-term operation, improving the adaptability and overall detection performance of the detection model in complex substation inspection scenarios. Attached Figure Description
[0037] Figure 1 This is a flowchart illustrating the incremental training method for substation inspection in one embodiment.
[0038] Figure 2 This is a flowchart illustrating the process of fusing task prompt vectors and instance prompt vectors in one embodiment;
[0039] Figure 3 This is a schematic diagram of the process for updating shared parameters in one embodiment;
[0040] Figure 4 This is a schematic diagram of the process for updating prompt parameters in one embodiment. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0042] The implementation details of the technical solutions of the embodiments of the present invention are described in detail below.
[0043] In one embodiment, such as Figure 1 The diagram illustrates a flowchart of an incremental training method for substation inspection, which may include the following steps:
[0044] Step S101: Obtain incremental inspection image data, label the newly added equipment categories in the incremental inspection image data, and generate pseudo-labels for labeling the learned equipment categories in the incremental inspection image data.
[0045] Incremental inspection image data consists of inspection images continuously collected during actual substation operation, including data on new equipment categories and new environmental data. New equipment categories may include equipment defects such as meter malfunctions, equipment damage, or structural deformation. New environmental data includes not only inspection scenarios corresponding to new equipment categories but also new environmental inspection scenarios caused by changes in lighting conditions, weather conditions, or site layout, such as nighttime lighting, rain, snow, fog, and new substation sites. Accordingly, the data size for a single newly added equipment category in the incremental inspection image data is no less than a preset number (e.g., 3000 images) to ensure the complete coverage of the samples under different shooting angles, shooting distances, and environmental conditions.
[0046] For incremental inspection image data, targets belonging to the newly added equipment category in the image are manually labeled. The labeling content includes at least the target's bounding box position, equipment category name, and corresponding defect attribute information, ensuring that the labeling accuracy is ≥98%, thereby forming the true labeling results of the newly added equipment category.
[0047] For learned device category targets appearing simultaneously in an image, manual annotation is no longer performed. Instead, the pre-trained basic detection model is used to perform forward inference on the incremental inspection image data. Detection results with confidence levels meeting preset conditions (e.g., confidence level ≥ 0.65) are selected from the prediction results as pseudo-labels to represent the spatial location and category information of learned device categories in the image. The annotation format of the pseudo-labels is consistent with that of newly added device categories. By generating pseudo-labels, both newly added and learned device categories can be explicitly labeled simultaneously in the same incremental inspection image, effectively reducing the risk of false detections caused by scene changes and background shifts.
[0048] Step S102: Based on the incremental inspection image data, the newly added equipment category annotation results, and pseudo-labels, construct an incremental training task.
[0049] Based on the newly added device categories involved in this incremental update, the set of new device categories corresponding to the current incremental training task is determined. The new device category set does not overlap with the device category set from the previous incremental task and the initial training phase (i.e., the already learned device categories). Simultaneously, by combining the scene information from the incremental inspection image data acquisition, the scene set corresponding to the current incremental training task is determined. The scene set is used to represent the inspection environment characteristics that are different from the previous training phase, such as nighttime, rainy weather, or new substation sites.
[0050] Based on this, the incremental inspection image data is preprocessed uniformly to form the training sample set for the current incremental training task. The manual annotation results for newly added equipment categories and the pseudo-labels for already learned equipment categories will be uniformly converted into annotation information in a preset format. For example, the YOLO annotation format that meets the training requirements of object detection models.
[0051] Thus, an incremental training task description for this round of model updates is constructed. This approach enables incremental training tasks to simultaneously incorporate supervised learning information for newly added device categories and constraint information for already learned device categories within the same training framework. This provides a foundation for category expansion and knowledge retention during subsequent incremental training. Furthermore, this incremental training task simultaneously supports new device category learning and inspection environment adaptation, overcoming the limitation of only being able to handle a single incremental scenario.
[0052] Step S103: Construct a prompting mechanism based on the incremental training task, generate task prompting vectors and instance prompting vectors for the incremental training task, and introduce a cross-attention module in front of the detection head of the basic detection model to fuse the task prompting vectors and instance prompting vectors into the cross-attention module in an attention manner.
[0053] For the current incremental training task, a prompting mechanism is constructed, including task prompting branches and instance prompting branches. The task prompting branch represents the overall semantic information of the incremental training task and learns a trainable task prompting vector for each incremental training task. It is used to characterize the set of newly introduced device categories and their corresponding task semantic features in the current incremental training task, so that the model can explicitly perceive the differences between the current task and the previous task during the training process.
[0054] Meanwhile, the instance hint branch is used to characterize instance-level environmental and scene features, generating instance hint vectors based on incremental inspection image data. This is used to reflect instance-level contextual information such as lighting conditions, background structure, and equipment distribution in the current inspection scenario, thereby improving the model's feature adaptation capability under complex environmental changes. In practical applications, task cue vectors... and instance hint vector It can consist of multiple cue vectors, for example, with a dimension of 8×256.
[0055] Obtaining the task hint vector With instance hint vector Then, the two are fused to form a prompt representation for guiding feature interaction. This prompt representation is then introduced into the cross-attention module of the basic detection model to modulate and guide the detection features during feature interaction, thereby achieving effective modeling of new device categories and new environmental scenarios.
[0056] The cross-attention module is positioned between the feature extraction network of the base detection model and the detection head. It introduces cue vectors to weight and interact with intermediate features before the detection head performs classification and regression prediction, thereby improving the model's feature selectivity and task adaptability during incremental training. The base detection model is an object detection model trained on labeled initial inspection image data, and the mainstream object detection model YOLO11 can be selected as the base model. During the initial training phase, the base detection model is trained on inspection image data collected in the substation's baseline inspection scenario. This inspection image data covers typical operating conditions such as sunny days and daytime, and includes initial equipment categories such as energy storage equipment, switching equipment, flip-up signs, and disconnectors. During the initial training phase, the training parameters of the base detection model can be configured as a learning rate of 0.001, batch size of 64, and training epochs of 100, using standard detection loss, classification loss, and bounding box regression loss for training. After initial training, the basic detection model meets the preset detection accuracy requirements in the benchmark inspection scenario (e.g., the average accuracy mAP@0.5≥85% under the condition that the cross-union ratio threshold is 0.5), thus serving as the initial model basis for subsequent incremental training.
[0057] It should be noted that during the data preparation phase, the preprocessing procedure for incremental inspection image data is consistent with that of the original model training phase, in order to avoid the interference of feature distribution differences on the evaluation results.
[0058] In one embodiment, feature extraction is performed on incremental inspection image data to obtain global image features that characterize the overall semantics and scene structure of the image. These global image features can be output by the feature extraction network of the basic detection model and used to reflect the equipment layout, background structure, and overall environmental information in the current inspection image.
[0059] After obtaining the global image features, these features are input into the P2IOD branch. This branch is positioned between the feature fusion network of the base detection model and the detection head, and employs a multilayer perceptron (MLP) bottleneck structure to perform nonlinear mapping and feature compression on the global image features, thereby dynamically generating instance cue vectors that match the environmental features of the current incremental inspection image. .
[0060] In one embodiment, Figure 2 The flowchart illustrating the fusion of task hint vectors and instance hint vectors is shown and may include the following steps:
[0061] Step S201: Perform a weighted average fusion of the task prompt vector and the instance prompt vector to generate a composite prompt vector.
[0062] Based on preset or learnable fusion weights, a weighted average is performed on the task cue vector and the instance cue vector to generate a composite cue vector that simultaneously contains task-level semantic information and instance-level environmental information. This allows the cue representation to adapt to instance differences while maintaining task consistency. Composite cue vector Specifically, it can be expressed as:
[0063]
[0064] Step S202: Perform a linear transformation on the composite hint vector to split it into a key vector and a value vector.
[0065] For composite cue vectors A linear transformation is performed to map it to a representation consistent with the feature space of the cross-attention module, and the composite cue vector is then split into key vectors after the linear transformation. AND value vector , to be used for the key branch and value branch in subsequent attention calculations, respectively.
[0066] Step S203: The key vector and value vector are used as prefixes for the key and value, respectively, and concatenated into the key matrix and value matrix of the corresponding layer of the detection head.
[0067] key vector AND value vector These are introduced as prefixes for the keys and values, respectively, into the corresponding layer of the detection head. Specifically, the key vector... Concatenate the vectors into the key matrix of the corresponding layer of the detection head, and then concatenate the vectors into the key matrix of the corresponding layer of the detection head. The values are concatenated into the value matrix of the corresponding layer of the detection head, so that during the cross-attention calculation process, the detection features can interact and be weighted under the guidance of the cue vector, thereby realizing the modulation and guidance of the cue vector on the target detection process.
[0068] Step S104: Under the condition that at least some parameters are shared in the basic detection model, the cross-attention module and the detection head are fine-tuned using the incremental training task, and the category mapping and model parameters of the basic detection model are updated.
[0069] During the fine-tuning phase of incremental training, to prevent the current incremental learning process from damaging the learned knowledge, the update range of at least some shared parameters in the base detection model is limited to effectively preserve the learned knowledge. These shared parameters mainly correspond to the network part of the base detection model responsible for general feature extraction. During fine-tuning, the parameters of the backbone network and feature fusion network in the base detection model can be frozen to maintain relative stability throughout the incremental training phase, thereby ensuring that the model's ability to represent general visual features does not drift with the incremental task.
[0070] Building upon this foundation, parameter updates are focused on the cross-attention module and the detection head. The cross-attention module receives and fuses the cue vectors generated by the cue mechanism to modulate the interaction process of detection features. The detection head is used to classify and predict the location of newly added device categories and already learned device categories. By utilizing the labeled results of newly added device categories and pseudo-labeled samples from the incremental training task, backpropagation updates are performed on the cross-attention module and the detection head, enabling the model to gradually adapt to the newly added device categories and their corresponding inspection scene features while maintaining its original detection capabilities.
[0071] Simultaneously, during the fine-tuning training process, the category mapping relationship of the basic detection model is expanded and updated. Specifically, the existing set of learned device categories is updated. Based on this, the new device categories introduced in the current incremental training task will be... Add to the category mapping table to form a complete category set containing both historical and newly added device categories. ,Right now This enables the updated basic detection model to uniformly identify and output all device categories during the inference phase.
[0072] In the actual training process, it is necessary to configure the relevant training parameters of the basic detection model. The learning rate is set to one-tenth of the original training learning rate, for example, 0.0001, and a one-cycle learning rate scheduling strategy (OneCycleLR, One Cycle Learning Rate Policy) is adopted. The number of training epochs is set to 50.
[0073] After completing the incremental training task, the updated model parameters are converted into a deployment-friendly format, such as Open Neural Network Exchange (ONNX) or TensorRT, to enable efficient inference on different hardware platforms. For edge computing devices, such as inspection robots or drone terminals, inference optimization accelerates model operation, ensuring inference latency is controlled within 50 milliseconds per frame. During deployment, the model interface remains consistent with the original model, with inspection images as input and output including equipment category, bounding box coordinates, confidence score, and defect information, thus eliminating the need to modify the existing inspection system's calling logic and achieving seamless integration. Finally, the optimized model is deployed to substation inspection terminals or cloud inference platforms, completing the replacement and launch of the new version, ensuring that the incremental training results can be applied to actual inspection tasks in a timely and efficient manner.
[0074] In the incremental training described above, since training only requires a small amount of newly added data, the training time for a single incremental task can be controlled within 2 hours. Compared to full retraining, the training efficiency is improved by more than 80%, while the GPU / CPU resource consumption is significantly reduced, thus achieving fast, efficient, and reliable model updates. This method can stably adapt to new equipment categories and inspection scenarios while inheriting historical knowledge, ensuring the accuracy, adaptability, and deployment efficiency of the model in substation inspection tasks.
[0075] In one embodiment, before constructing the incremental training task, the model parameters from the previous incremental training phase are first loaded. This is then used as the initial model parameters for the current incremental training task. This is the complete set of model parameters obtained at the end of the previous incremental training phase, used to ensure that the model can inherit the knowledge learned in the previous incremental task.
[0076] In practical implementation, model parameters can be decomposed into shared parameters. Task parameters And prompt parameters. Among them, shared parameters Corresponding to the backbone network and feature fusion neck in the basic detection model, it is used to extract feature representations that are universal for different equipment categories and inspection scenarios; task parameters Corresponding to the classification and regression branches in the detection head, it is used to adapt to the currently learned set of device categories; the prompting parameters are used to support the construction of the prompting mechanism, including trainable parameters for generating task prompting vectors and relevant parameter configurations for prompting fusion and modulation.
[0077] By loading the shared parameters, task parameters, and cue parameters from the previous incremental training phase, the model can inherit the feature extraction capabilities, category adaptation capabilities, and cue expression capabilities formed in the previous incremental task without relying on the replay of historical samples during the current incremental training process. This provides a stable and continuous model initialization foundation for subsequent incremental training.
[0078] It is understandable that model parameters include shared parameters, task parameters, and cue parameters. Therefore, after incremental training is completed, these three parameters need to be updated separately. The following describes the updating of model parameters in detail through different examples.
[0079] In one embodiment, Figure 3 A flowchart illustrating the process of updating shared parameters may include the following steps:
[0080] Step S301: Based on the initial shared parameters of the basic detection model, calculate the historical shared vector formed by the previous incremental task and the current shared vector formed by the current incremental training task.
[0081] After the initial training of the base detection model is completed, its shared parameters are recorded as the initial shared parameters. In subsequent rounds of incremental training, the model will form a corresponding shared parameter state after each round of incremental task is completed. In the... In the incremental training, the preceding incremental training phase (i.e., the first) is obtained. Shared parameters at the end of the incremental training and combine it with the initial shared parameters By performing comparative calculations, a historical shared vector representing the cumulative effect of historical incremental knowledge is obtained. Specifically, it can be expressed as:
[0082]
[0083] After fine-tuning the current incremental training task, obtain the shared parameters corresponding to the current stage. And based on this parameter and the initial shared parameter The difference between the numbers is used to calculate the current shared vector. .
[0084] Step S302: For the shared parameters corresponding to different network depths in the basic detection model, calculate the corresponding layer retention factor and adaptation factor.
[0085] Because different network layers in the basic detection model have varying levels of feature abstraction, their sensitivity to historical knowledge also differs during incremental training. Therefore, a layer-preserving factor is introduced. With adaptation factors The update strategy for shared parameters at different network depths is adaptively adjusted.
[0086] Specifically, for historical shared vectors With the current shared vector The changing trends at each network layer are analyzed, and the corresponding layer retention factor is calculated for the shared parameters of each layer. With adaptation factors The hierarchical retention factor controls the proportion of historical shared vectors retained in the parameter update of that layer, while the adaptation factor adjusts the strength of the introduction of the current shared vector in the parameter update of that layer.
[0087] Step S303: Using the hierarchical retention factor and the adaptation factor, the historical shared vector and the current shared vector are weighted and fused to obtain the updated shared parameters.
[0088] Based on the obtained retention and adaptation factors at each level, the historical shared vector... With the current shared vector Layered weighted fusion is performed. Specifically, at each network layer, based on the corresponding... and The updated shared parameters are formed by linearly combining the historical shared vectors and the current shared vectors. In this way, the updated shared parameters can inherit the general features that have been stably learned in the previous incremental task, and can also appropriately absorb the new knowledge introduced in the current incremental training task.
[0089] In one embodiment, after completing fine-tuning training for the current incremental training task, the task parameters are incrementally updated. Specifically, the task parameters learned in the previous incremental training phase are retained. Furthermore, it generates corresponding task parameters for newly introduced device categories in the current incremental training task. .
[0090] By using the task parameters learned in the preceding incremental task Newly generated task parameters in the current incremental training task Perform incremental concatenation to form an updated set of task parameters. This allows the detection model to maintain its ability to detect already learned device categories while also supporting the detection of newly added device categories.
[0091] In this way, the updated model parameters can simultaneously include the updated shared parameters and the expanded task parameters, thereby achieving unified detection of historical device categories and newly added device categories.
[0092] In one embodiment, Figure 4A flowchart for updating the prompt parameters may include the following steps:
[0093] Step S401: Based on the contribution of each component in the prompt vector to the category discrimination, the corresponding key prompt components are selected from the historical prompt vectors of the previous incremental task and the current prompt vector generated by the current incremental task.
[0094] The process involves retrieving historical cue vectors saved from previous incremental tasks and simultaneously acquiring current cue vectors generated during the current incremental training task. For historical cue vectors, based on the contribution of each component to the detection accuracy or attention weights of learned categories, the top-k% of cue components with the highest contribution are retained as key historical cue components. Similarly, for current cue vectors, based on the feature modeling capabilities for newly added categories and the current scene, the top-m% of cue components with the highest contribution are retained as key current cue components. This selection process ensures that only cue components that significantly contribute to category discrimination participate in subsequent fusion, thereby improving the effectiveness and stability of the cue information.
[0095] Step S402: Perform a consistency judgment between the historical key hint component in the historical hint component and the current key hint component in the current hint vector.
[0096] The consistency of directional or semantic features between historical and current key hint components in the vector space is compared, for example, by calculating cosine similarity or other vector similarity metrics. If historical and current key hint components are highly consistent in direction or semantics, they are determined to be consistent components; otherwise, they are determined to be inconsistent components.
[0097] Step S403: Based on the consistency judgment result, perform fusion processing on consistent key prompt components, and inherit historical key prompt components for inconsistent key prompt components to obtain fused prompt parameters.
[0098] For key cue components that are determined to be consistent, a weighted average of the historical and current key cue components is performed to generate a fused cue component. For inconsistent key cue components, the historical key cue components are directly inherited, maintaining their original feature expressions. Finally, all fused key cue components are integrated with the inherited historical cue components to form cue parameters. The fused cue parameters retain the cue knowledge learned in the previous incremental task while effectively incorporating the environment and category information of the current incremental task, thus providing a stable and adaptive cue vector for the cross-attention module during the incremental training phase.
[0099] In practical applications, the cue vectors include task cue vectors and instance cue vectors. In the above processing, the key cue components of the two types of cue vectors can be filtered, judged and fused to generate unified fused cue parameters at the task level and instance level. These parameters are then injected into the cross-attention module to modulate the interaction process of the detection features, improve the detection accuracy of new categories and maintain the performance of historical categories.
[0100] In one embodiment, the base detection model uses a composite loss function to constrain the model parameter update process during the fine-tuning training phase. The composite loss function is defined as the total loss function, specifically expressed as follows:
[0101]
[0102] in, This is the YOLO11 standard detection loss function, used to ensure the positioning accuracy and classification accuracy of newly added device categories in target detection tasks; To improve the distillation loss, a dynamic threshold mechanism is introduced to filter high-confidence historical prediction results and selectively constrain historical knowledge to alleviate the knowledge bias problem caused by changes in background distribution during incremental training. The direction consistency loss is used to penalize the conflict between the parameter update direction and the historical learning direction, thereby stabilizing the parameter update process of the model. Sparse loss is used to constrain the sparsity of cue parameters or related modulation parameters in the model, reducing the impact of redundant parameters on training stability.
[0103] Regarding training configuration, the loss weight parameters are set. , , This balances the contribution of each loss term to the total loss. Through the above-mentioned composite loss constraint and training strategy configuration, the model can adapt to new device categories while maintaining stable modeling of the learned device features.
[0104] In one embodiment, after completing the incremental training task for substation inspection, a systematic performance evaluation of the updated basic detection model is performed. During the evaluation, the official YOLO11 evaluation script is used to validate the model. Evaluation metrics include mAP@0.5 as the core accuracy metric, and the average accuracy mAP@0.5:0.95 across multiple cross-validation ratio (CUP) thresholds as the comprehensive accuracy metric. Precision, recall, and inference speed (frames per second) are also considered to comprehensively evaluate the model's detection performance and engineering applicability. The evaluation process fully complies with the standard YOLO11 evaluation specifications.
[0105] Detection performance was evaluated on both the original validation set and the incremental validation set. The original validation set consisted of inspection image data used in the preliminary training phase of the base model, divided according to a certain ratio, containing only learned device categories, and was used to evaluate the model's ability to retain historical knowledge after incremental training. The incrementally updated model was tested on this validation set, and its performance was compared with that of the model before incremental training. The decrease in mAP@0.5 was controlled to be no more than 5%, and the difference in inference speed between the incrementally updated model and the model before incremental training was no more than 2 frames per second, to ensure that incremental training did not significantly affect the original detection capability.
[0106] The incremental validation set is obtained by dividing the incremental inspection image data according to a certain ratio (e.g., 9:1). The incremental training set is used for incremental training of the model, while the incremental validation set is used to evaluate the model's adaptability to new equipment categories and new inspection scenarios. This validation set includes samples of new equipment categories and inspection images in the new environment. After testing the model, the mAP@0.5 for the new category reaches no less than 75%, while the mAP@0.5 for the learned equipment categories in the new environment reaches no less than 80%, to meet engineering application requirements.
[0107] In the above embodiments, efficient learning and adaptation of new device categories can be achieved through incremental inspection of image data and pseudo-label information without relying on the original training data. In the incremental training task, task-level semantic information and instance-level environmental features are fused by constructing task-level cue vectors and instance-level cue vectors, and introducing a cross-attention module before the detection head of the basic detection model. This allows the model to fully utilize historical knowledge while considering the individual feature differences of the current image when processing new device categories. During fine-tuning training, the general feature extraction part of the basic detection model is frozen, and only the parameters of the cross-attention module and the detection head are updated. A composite loss function is used for constraint, ensuring both the detection accuracy of new device categories and effectively maintaining the feature consistency and detection performance of already learned device categories.
[0108] Furthermore, this method achieves a systematic update of the model's shared parameters, category adaptability, and prompting mechanism through hierarchical weighted fusion of historical and current shared vectors, incremental concatenation of task parameters, and filtering and fusion of prompting parameters. This enhances the model's stability and adaptability during incremental training. Performance evaluation based on a complete category set verifies that the model's ability to identify newly added equipment categories is significantly improved while inheriting learned knowledge. In conclusion, this method enables rapid, efficient, and reliable detection of newly added equipment in substation inspection scenarios, enhancing the model's generalization ability and adaptability, and providing stable and accurate technical support for practical inspection tasks.
[0109] In one embodiment, a computer storage medium is provided on which a computer program is stored, and when the computer program is executed by a processor, it implements a substation inspection incremental training method.
[0110] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0111] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0112] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for incremental training in substation inspection, characterized in that, include: Acquire incremental inspection image data, label the newly added equipment categories in the incremental inspection image data, and generate pseudo-labels for labeling the learned equipment categories in the incremental inspection image data; Based on the incremental inspection image data, the newly added equipment category annotation results, and pseudo-labels, an incremental training task is constructed. A prompting mechanism is constructed based on the incremental training task, generating task prompt vectors and instance prompt vectors for the incremental training task. A cross-attention module is introduced before the detection head of the basic detection model to fuse the task prompt vectors and instance prompt vectors into the cross-attention module in an attention manner. This includes: performing a weighted average fusion of the task prompt vectors and instance prompt vectors to generate a composite prompt vector; performing a linear transformation on the composite prompt vector to split it into a key vector and a value vector; using the key vector and the value vector as prefixes for the key and value, respectively, and concatenating them into the key matrix and value matrix of the corresponding layer of the detection head, where they participate in the cross-attention calculation of the detection head as prefix keys and values. The basic detection model is a target detection model trained based on labeled initial inspection image data; the task prompt vector represents the semantic information of the incremental training task; and the instance prompt vector represents the instance-level context information of the inspection scene. Under the condition that at least some parameters in the basic detection model are shared for updates, the cross-attention module and the detection head are fine-tuned using the incremental training task, and the category mapping and model parameters of the basic detection model are updated.
2. The substation inspection incremental training method according to claim 1, characterized in that, Before constructing the incremental training task, the method further includes: Load the model parameters from the previous incremental training phase as the initial model parameters for the current incremental training task, so as to inherit the shared parameters, task parameters and cue parameters learned in the previous incremental task during the current incremental training process.
3. The substation inspection incremental training method according to claim 2, characterized in that, Generating the instance hint vector includes: Extract the global image features from the incremental inspection image data; Based on the global features of the image, a vector that matches the environmental features of the incremental inspection image is dynamically generated and used as the instance prompt vector.
4. The substation inspection incremental training method according to claim 2, characterized in that, The updating of the model parameters of the basic detection model includes: Based on the initial shared parameters of the basic detection model, the historical shared vector formed by the previous incremental task and the current shared vector formed by the current incremental training task are calculated respectively. For the shared parameters corresponding to different network depths in the basic detection model, calculate the corresponding layer retention factor and adaptation factor; The historical shared vector and the current shared vector are weighted and fused using the hierarchical retention factor and the adaptation factor to obtain the updated shared parameters.
5. The substation inspection incremental training method according to claim 2, characterized in that, The updating of the model parameters of the basic detection model includes: The task parameters already learned in the previous incremental task are retained, and the task parameters corresponding to the newly added device category are incrementally concatenated with the task parameters already learned in the previous incremental task.
6. The substation inspection incremental training method according to claim 2, characterized in that, The updating of the model parameters of the basic detection model also includes: Based on the contribution of each component in the cue vector to the category discrimination, the corresponding key cue components are selected from the historical cue vectors of the previous incremental task and the current cue vector generated by the current incremental task. A consistency judgment is made between the historical key prompt component in the historical prompt vector and the current key prompt component in the current prompt vector; Based on the consistency judgment result, the consistent key prompt components are fused, and the inconsistent key prompt components inherit the historical key prompt components to obtain the fused prompt parameters. The prompt vectors include task prompt vectors and instance prompt vectors.
7. The substation inspection incremental training method according to claim 1, characterized in that, During the fine-tuning training process, a composite loss function is used for constraints. The composite loss function includes the main detection loss to ensure the accuracy of the new equipment, and the improved distillation loss, orientation consistency loss and sparsity loss to maintain the consistency of the learned equipment features.
8. The substation inspection incremental training method according to claim 1, characterized in that, The method further includes: After completing the incremental training task, the updated basic detection model is evaluated for detection performance based on the complete set of categories, including the learned device categories and the newly added device categories, to verify the effectiveness of the incremental training.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the substation inspection incremental training method according to any one of claims 1 to 8.