Power grid operation behavior time sequence detection method, system, equipment and medium

Abstract

The invention relates to the technical field of power grid scene action detection, and discloses a power grid operation behavior time sequence detection method, system and device and a medium, and the method comprises the steps: constructing an incremental learning model architecture; carrying out self-guided sampling processing on the new category data to generate a training sample; presetting a double-path distillation mechanism and a loss function, and performing model parameter updating and model optimization on the incremental learning model architecture according to the double-path distillation mechanism; and performing power grid operation behavior time sequence detection according to the incremental learning model architecture after model parameter updating and model optimization. By dynamically expanding an action category knowledge base and a double-path distillation mechanism, accurate time sequence positioning of multi-stage operation behaviors is realized under the condition that only a small number of new actions are needed to label samples; and finally, the recognition capability of original operation actions (such as wearing of insulating gloves) is kept, and meanwhile, rapid adaptation of newly-added actions is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power grid scene action detection, and in particular to a power grid operation behavior timing detection method, system, device and medium. BACKGROUND

[0002] In the power grid operation scene (such as climbing a power transmission tower and operating a substation device), the operator needs to complete actions such as climbing a pole, voltage testing, installing / removing a grounding wire, and descending a tower, which have strict timing constraints, and the behavior safety directly affects the quality of power grid operation and maintenance.

[0003] Although the existing action detection technology can identify basic violation behaviors, it faces three challenges under complex working conditions: first, the posture changes in the climbing process, the subtle differences in tool operation, and the blocking interference make it difficult to locate the action boundary, and the traditional R-C3D framework is prone to the phenomenon of "forgetting old actions" (such as misjudging the removal of a grounding wire as a tower descent) due to the sample distribution deviation of the proposal branch; second, new operation categories (such as new insulation testing processes) need to be re-labeled with a large number of video frames, and when the existing method updates the classifier through fine-tuning, the feature representation ability of the original action (such as safety belt hanging) is significantly degraded; finally, visual interference caused by sudden changes in light or device reflection and key features such as voltage tester holding angle cause the coupling error between proposal classification and boundary box regression to be amplified. Therefore, an incremental learning method for power grid operation is urgently needed to achieve continuous safety monitoring of multi-stage actions such as climbing, operation, and reset. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a power grid operation behavior timing detection method, system, device and medium, which can solve the problem of locating the action boundary in the power grid operation scene, while avoiding the phenomenon of "forgetting old actions", maintaining the feature representation ability of the original action, and effectively dealing with visual interference caused by sudden changes in light or device reflection.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a power grid operation behavior timing detection method, comprising:

[0008] An incremental learning model architecture is constructed, which includes a frozen original model and a trainable new model;

[0009] The frozen original model is used to generate new category data, and the new category data is labeled data including a new category;

[0010] The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer;

[0011] Perform self-guided sampling on new category data to generate training samples;

[0012] Preset a dual-path distillation mechanism and loss function, and perform model parameter updates and model optimization on the incremental learning model architecture according to the dual-path distillation mechanism;

[0013] The grid operation behavior timing detection is performed based on the incremental learning model architecture after the model parameters are updated and the model optimization is completed.

[0014] As a preferred solution of the method for detecting the timing of power grid operation behavior according to the present invention, the step of constructing an incremental learning model architecture includes:

[0015] Acquire an original action category set and annotation data for the original action category set;

[0016] Training the frozen original model based on the labeled data of the original action category set;

[0017] The loss function of the frozen original model is a combination of several loss functions;

[0018] The number of nodes in the classification layer of the frozen original model is expanded to adapt to the new category, and a new trainable model is generated.

[0019] This preferred solution obtains the original action category set and its annotated data, and trains the frozen original model based on this data, ensuring that the model accurately recognizes known action categories. At the same time, the frozen original model loss function, which uses a combination of multiple loss functions, can more comprehensively measure the model's performance and improve the model's generalization ability. In addition, by expanding the number of classification layer nodes of the frozen original model to adapt to new categories and generating a new trainable model, the model's incremental learning capability is achieved, allowing the model to continuously adapt to new action categories, thereby improving the model's practicality and flexibility.

[0020] As a preferred solution of the method for detecting time series of power grid operation behavior according to the present invention, the self-guided sampling process of the new category data to generate training samples includes:

[0021] generating new category data according to the frozen original model;

[0022] Preset threshold parameters for new category data;

[0023] The threshold parameter is used to screen foreground samples in new category data;

[0024] Preset regression offset, optimize the filtered new category data, and generate training samples.

[0025] As a preferred solution of the method for detecting the timing of power grid operation behavior according to the present invention, the preset dual-path distillation mechanism and loss function include:

[0026] Preset the cross entropy function based on temperature parameters as the classification probability soft target distillation loss;

[0027] Calculate the classification probability soft target distillation loss when the temperature parameter is two;

[0028] Only the classification output of the original category when no new category data is generated is retained to participate in the loss calculation.

[0029] As a preferred embodiment of the method for detecting the timing of power grid operation behavior according to the present invention, the preset dual-path distillation mechanism and loss function further include:

[0030] Presetting a confidence interval, and selecting distilled samples based on the candidate boxes in the output of the original model;

[0031] Control the number of distillation samples.

[0032] As a preferred embodiment of the method for detecting the timing sequence of power grid operation behavior according to the present invention, the timing sequence detection of power grid operation behavior according to the incremental learning model architecture after the model parameter update and model optimization is completed includes:

[0033] After model parameter updates and model optimization, the incremental learning model architecture uses non-maximum suppression to process candidate boxes;

[0034] Generate detection results through action classification probability and regression offset;

[0035] The final detection sequence is output when the termination category is detected or the maximum number of iterations is reached.

[0036] As an optimal solution of the power grid operation behavior timing detection method described in the present invention, the preset dual-path distillation mechanism and loss function also include the loss function including R-C3D basic loss, classification probability soft target distillation loss and feature map L2 norm distillation loss.

[0037] In a second aspect, the present invention provides a power grid operation behavior time sequence detection system, comprising:

[0038] A model architecture building module, configured to construct an incremental learning model architecture, wherein the incremental learning model architecture includes a frozen original model and a trainable new model;

[0039] The frozen original model is used to generate new category data, where the new category data is labeled data including new categories;

[0040] The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer;

[0041] The sample generation module is used to perform self-guided sampling processing on new category data to generate training samples;

[0042] An optimization module, configured to preset a dual-path distillation mechanism and a loss function, and to perform model parameter updates and model optimization on the incremental learning model architecture according to the dual-path distillation mechanism;

[0043] The detection module is used to perform timing detection of power grid operation behavior based on the incremental learning model architecture after model parameter updates and model optimization.

[0044] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when executing the computer program.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described above when the computer program is executed by a processor.

[0046] Compared with the existing technology, the present invention has the following advantages: A method for detecting the time series of power grid operation behaviors is proposed. The method constructs an incremental learning model architecture, which includes a frozen original model and a trainable new model. The architecture performs self-guided sampling on new category data to generate training samples. A dual-path distillation mechanism and loss function are pre-set, and model parameters are updated and optimized in the incremental learning model architecture based on the dual-path distillation mechanism. After the model parameter updates and model optimization are completed, the incremental learning model architecture is used to detect the time series of power grid operation behaviors. Freezing the original model ensures the stable inheritance of existing category knowledge and avoids the problem of forgetting old categories due to learning new categories. The trainable new model can be flexibly expanded to quickly adapt to the addition of new categories, improving the scalability and adaptability of the system. Self-guided sampling of new category data effectively addresses the scarcity of new category samples and reduces reliance on large amounts of labeled data for new categories. The pre-set dual-path distillation mechanism and loss function provide powerful guidance for model parameter updating and optimization, ensuring model performance and stability during the incremental learning process. Using the optimized incremental learning model architecture to detect the time series of power grid operation behaviors improves detection accuracy and efficiency, providing strong guarantees for the safety and reliability of power grid operations. By dynamically expanding the action category knowledge base and the dual-path distillation mechanism, accurate temporal positioning of multi-stage operation behaviors is achieved for key features such as the holding posture of the voltage tester and the installation angle of the grounding wire clamp under the condition of only a small number of new action annotation samples. Ultimately, while maintaining the recognition capability of the original operation actions (such as wearing insulating gloves), rapid adaptation of new actions is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flowchart of a method for detecting timing of power grid operation behavior provided by one embodiment of the present invention.

[0049] Figure 2 A schematic diagram of the overall architecture of a method for detecting timing of power grid operation behavior provided by one embodiment of the present invention.

[0050] Figure 3 This is a diagram of the internal structure of an electronic device for a method for detecting timing of power grid operation behavior provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present application.

[0052] Embodiment 1, reference Figure 1-Figure 2 For the first embodiment of the present application, the embodiment provides a power grid operation behavior timing detection method, comprising:

[0053] In the prior related art, there are some problems, such as difficult positioning of action boundaries, old action forgetting phenomenon, and large amount of newly added category labeling data.

[0054] The present application provides a method that can effectively solve the above-mentioned problems. Next, how to realize the power grid operation behavior timing detection method will be described in detail in combination with multiple embodiments;

[0055] Figure 1 A method flowchart of a power grid operation behavior timing detection method is shown, comprising:

[0056] S101, a incremental learning model architecture is constructed, the incremental learning model architecture comprises a frozen original model and a trainable new model;

[0057] It should be noted that in the power grid operation scene (such as power tower climbing and substation equipment operation), the operator needs to complete the actions of climbing the pole, voltage testing, grounding wire installation / removal and tower descending, which have strict timing constraints, and the behavior safety directly affects the quality of power grid operation and maintenance. Therefore, some means for detecting the timing of power grid operation behavior need to be established.

[0058] It should be noted that although the existing action detection technology has achieved certain results in identifying basic violation behaviors, it inevitably encounters three severe challenges when facing complex working conditions. First, during the climbing process, due to the continuous change of worker posture, the subtle difference of tool operation and the existence of shielding interference in the environment, these factors jointly make it extremely difficult to accurately position the action boundary.

[0059] When dealing with such problems, the traditional R-C3D framework is prone to sample distribution deviation in its proposal branch, which leads to the occurrence of the "old action forgetting" phenomenon. For example, during the identification process, the removal action of the grounding wire may be misjudged as the descending action of the tower.

[0060] Secondly, with the continuous emergence of new job categories, such as the introduction of new insulation testing processes, a large number of video frames need to be re-labeled to adapt to new detection requirements.

[0061] However, existing methods often cause significant degradation in the feature representation ability of original actions, such as fastening a seat belt, when updating the classifier through fine-tuning, which in turn affects the overall detection accuracy.

[0062] Finally, visual interference caused by sudden changes in lighting conditions or device reflections can confuse key features such as the angle at which the voltage tester is held. This confusion not only increases the difficulty of recognition but also further amplifies the coupling error between proposal classification and bounding box regression, increasing the uncertainty of the detection results and seriously affecting the reliability and accuracy of the action detection system.

[0063] Therefore, it is necessary to design a new method for timing detection of power grid operation behavior.

[0064] In an embodiment of the present invention, time series detection is performed by constructing an incremental learning model architecture.

[0065] In an optional embodiment, the incremental learning model architecture can be constructed using a neural network, particularly a deep learning network, such as a convolutional neural network (CNN) or a recurrent neural network (RNN) and their variants. These networks have powerful feature extraction and pattern recognition capabilities and are suitable for handling complex power grid operation behavior time series detection tasks. The specific steps for constructing the incremental learning model architecture using a neural network can be as follows:

[0066] First, a large amount of video data of power grid operation behaviors is collected and preprocessed, including video frame extraction, normalization, and labeling, to ensure data quality and consistency.

[0067] Then, the preprocessed data is used to train the neural network model so that it can learn the characteristic representation of power grid operation behavior.

[0068] During the training process, an incremental learning strategy can be adopted, that is, gradually introducing new job category data and updating the model parameters to maintain the model's adaptability to new data and its ability to remember old data.

[0069] In addition, regularization techniques such as L2 regularization or Dropout can be introduced to prevent model overfitting and improve the generalization ability of the model.

[0070] Finally, the model parameters are continuously adjusted through optimization algorithms such as stochastic gradient descent (SGD) or Adam until the model achieves optimal performance on the validation set.

[0071] In an optional embodiment, the incremental learning model architecture can also be constructed through transfer learning technology, using model parameters that have been trained on similar tasks as initial parameters to accelerate the training process of new tasks and improve the performance of the model. Transfer learning technology can effectively utilize existing knowledge, reduce dependence on new category labeled data, and improve the convergence speed and final performance of the model. Through transfer learning, the common features in the power grid operation behavior can be transferred to new tasks, so that the model can adapt faster and accurately perform time series detection when facing new categories. When using transfer learning technology, the specific steps are as follows:

[0072] First, a source task similar to the target power grid operation behavior detection task is selected, for which a trained neural network model already exists.

[0073] Then, the parameters of the source task model are used as the initial parameters of the target task to start training the target task model.

[0074] During the training process, the model can be fine-tuned according to the data characteristics of the target task to adapt to the new data distribution.

[0075] In this way, transfer learning technology not only accelerates the training process of new tasks, but also improves the convergence speed and final performance of the model, enabling the power grid operation behavior timing detection system to adapt faster and perform timing detection accurately when faced with new categories.

[0076] It's important to note that when building an incremental learning model architecture, freezing the original model is key to maintaining the stable inheritance of existing categorical knowledge. The frozen original model, trained based on historical data, accurately identifies known action categories in power grid operations. This model remains unchanged during the incremental learning process, ensuring the stability of the existing categorical knowledge.

[0077] At the same time, the new trainable model adapts to the new categories by expanding the number of nodes in the original classification output layer. This design allows the model to flexibly respond to the needs of newly added job categories without having to retrain the entire model. The new model is continuously optimized during the training process to improve its recognition ability for new categories.

[0078] In an embodiment of the present invention, freezing the original model is used to generate new category data, where the new category data is labeled data including new categories;

[0079] The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer;

[0080] In an embodiment of the present invention, constructing an incremental learning model architecture includes:

[0081] Obtaining an original action category set and labeled data for the original action category set;

[0082] The frozen original model is trained based on the labeled data of the original action category set;

[0083] Freeze the original model's loss function into a combination of several loss functions;

[0084] Expand the number of nodes in the classification layer of the frozen original model to adapt to the new categories and generate a new trainable model.

[0085] In the embodiment of the present invention, new categories may include but are not limited to new insulation test procedures, special equipment operations, etc. The introduction of these new categories requires the detection system to be able to quickly adapt and accurately identify them to ensure the safety and efficiency of power grid operations.

[0086] For example, construct the original action category set C o and its labeled dataset D o , where D o Contains only C o Action annotation;

[0087] Based on D o Train the R-C3D model to get the original model M o , its basic loss function is:

[0088]

[0089] in, and denote the proposal classification and regression losses respectively, and denote action classification and regression losses respectively;

[0090] Freeze the original model M o The number of nodes in the classification layer to adapt to the new category C n , generate a new model M that can be initialized and trained n .

[0091] It's important to note that we first define an initial set of action categories. These categories represent the basic actions that the model initially needs to recognize. Data related to this initial set of action categories is collected and annotated. This annotated data only contains action labels from the original set of action categories, ensuring the purity and relevance of the dataset.

[0092] It should be noted that the R-C3D model is a deep learning model for video behavior recognition that can handle three-dimensional convolution operations and is suitable for temporal behavior detection tasks.

[0093] Proposal classification loss, which is used to evaluate whether a candidate region (proposal) contains the target action.

[0094] Proposal regression loss optimizes the position and size of the proposal box so that it covers the target action more accurately.

[0095] Action classification loss, which determines the action category of the final candidate region.

[0096] Action regression loss further fine-tunes the position and size of the action bounding box to improve localization accuracy.

[0097] It should also be noted that the original model M o By minimizing the above loss function, the trained model can effectively identify action categories in the original action category set.

[0098] Expanding the number of nodes in the classification layer is to enable the model to recognize new action categories. It is necessary to increase the number of output nodes of the classification layer, and each new node corresponds to a new category.

[0099] Initialize a new model: Based on the original model, a new model is generated by adjusting the classification layer structure to prepare for subsequent incremental learning.

[0100] It should also be noted that the construction of an incremental learning model architecture, which includes freezing the original model and a trainable new model, can fully utilize existing model knowledge and data resources, while adapting to the addition of new categories, avoiding the problem of forgetting old categories caused by learning new categories in traditional methods. By freezing the original model, the stable inheritance of existing category knowledge is ensured, and the risk of destroying the original recognition ability due to model updates is avoided. At the same time, the trainable new model can quickly adapt to the addition of new categories through flexible expansion, improving the scalability and adaptability of the system. This incremental learning method not only reduces the dependence on new category labeled data, but also improves the model's recognition ability of new categories, providing a more accurate and reliable solution for the timing detection of power grid operation behavior.

[0101] S102, performing self-guided sampling processing on the new category data to generate training samples;

[0102] It should be noted that after S101 is performed, new action data for the new category will be obtained. The next step is to process the new action data and ultimately achieve targeted updates to the constructed architecture.

[0103] In an optional embodiment, new action data can be processed through various methods, such as data augmentation, data filtering, or data fusion, to enrich the diversity of training samples and improve the model's generalization capabilities. Data augmentation techniques, such as rotation, scaling, flipping, or adding noise, can simulate the various variations in actual operation scenarios, enabling the model to learn more robust feature representations. Data filtering ensures the high quality and representativeness of the training set by removing low-quality or redundant samples. Data fusion technology can integrate data from different sources to fully leverage the advantages of multiple information sources and further improve model performance. After obtaining the processed training samples, they are input into the incremental learning model architecture for training. An optimization algorithm continuously adjusts the model's parameters, enabling the model to accurately recognize new action categories while maintaining its ability to recognize existing categories. This step is key to achieving precise temporal location of multi-stage operation behaviors. Through continuous iteration and optimization, a high-performance power grid operation behavior temporal detection system is ultimately achieved.

[0104] In the embodiment of the present invention, self-guided sampling is performed on the new category data.

[0105] In an embodiment of the present invention, performing self-guided sampling processing on new category data to generate training samples includes:

[0106] Generate new category data based on the frozen original model;

[0107] Preset threshold parameters for new category data;

[0108] The threshold parameter is used to filter foreground samples in new category data;

[0109] Preset regression offset, optimize the filtered new category data, and generate training samples.

[0110] Specifically, the prediction results of the frozen original model are used to set the foreground threshold to 0.8 and the background threshold to 0.2, where the foreground samples meet the following conditions:

[0111]

[0112] Background samples meet the following requirements:

[0113]

[0114] In an optional embodiment, potential foreground samples are screened according to a threshold and the ratio of new and old action samples is controlled to be less than 1 / 5;

[0115] By M o Regression offsets to optimize bounding box annotations.

[0116] It is necessary to use the trained original model to predict new data and set two key thresholds: foreground threshold and background threshold based on the prediction results.

[0117] The foreground threshold is 0.8: This means that for any sample k, if its probability of being predicted as a certain action category is greater than 0.8, the sample is considered as a potential foreground sample. These samples are likely to contain instances of target actions with high confidence.

[0118] The background threshold is 0.2: Similarly, if a sample k is predicted to be less than 0.2, the sample is considered as a potential background sample. These samples are likely to not contain target actions or have low relevance to target actions.

[0119] Through such threshold setting, those most likely to be foreground or background samples can be effectively screened out, providing a basis for further data processing.

[0120] Furthermore, after screening out potential foreground and background samples, further processing of these samples is needed to ensure the balance between new and old action samples.

[0121] Screening potential foreground samples: Based on the above foreground threshold (0.8), select those samples that meet the conditions from all samples as potential foreground samples. These samples will be used for subsequent model training and optimization.

[0122] Control the ratio of new and old action samples below 1 / 5: In order to prevent too many new action category samples from being introduced, causing the model to overfit to learning new categories and ignoring the original action categories, the ratio of new and old action samples needs to be controlled.

[0123] Specifically, the number of new action samples should be kept below one-fifth of the number of old action samples. This can ensure that the model learns new knowledge while not forgetting existing knowledge, avoiding the occurrence of catastrophic forgetting phenomenon.

[0124] The last step is to optimize the annotation of the bounding box to improve the accuracy of detection.

[0125] Regression offset: Use the regression information predicted by the frozen original model to calculate the offset of each candidate box relative to the true bounding box. These offsets reflect the position and size differences between the candidate box and the true bounding box.

[0126] Optimize bounding box annotations: Based on the calculated offset, the position and size of the candidate boxes are adjusted to more accurately cover the target action. This step helps improve the model's detection performance in real-world applications, especially in complex scenarios such as changing lighting and device reflections, allowing it to better adapt to environmental changes and improve robustness.

[0127] It should be noted that self-guided sampling processing of new category data to generate training samples can significantly improve the training efficiency and recognition accuracy of the incremental learning model architecture. By accurately screening foreground and background samples and controlling the ratio of new and old action samples, the model can avoid overfitting to new categories during training while maintaining its recognition ability for old categories. In addition, by optimizing bounding box annotations, the detection effect of the model in practical applications can be further improved, enabling it to more accurately locate target actions in power grid operation behaviors. This self-guided sampling processing method not only simplifies the data processing process, but also improves the quality and diversity of training samples, providing strong support for building a high-performance power grid operation behavior time series detection system. In practical applications, this processing method can be widely used in various power grid operation scenarios, contributing to the intelligent and automated development of the power industry.

[0128] S103, presetting a dual-path distillation mechanism and a loss function, and updating model parameters and optimizing the incremental learning model architecture according to the dual-path distillation mechanism;

[0129] It should be noted that the initial incremental learning model architecture is relatively crude and cannot accurately identify all power grid operation behavior categories, especially newly introduced categories. To improve the model's recognition accuracy for new categories while maintaining stable recognition of old categories, this paper proposes a dual-path distillation mechanism and loss function.

[0130] It should be noted that the dual-path distillation mechanism consists of a teacher path and a student path. The teacher path consists of a frozen original model and is responsible for providing stable knowledge of previous categories. The student path, consisting of a new, trainable model, is responsible for learning knowledge of new categories. During training, the teacher and student paths jointly process input data and produce corresponding outputs. The output of the teacher path serves as a reference for knowledge of previous categories, while the output of the student path is used to identify new categories. By comparing the outputs of the teacher and student paths, a distillation loss is calculated. This loss guides the learning of the student path, enabling it to gradually learn the characteristics of new categories while maintaining knowledge of previous categories.

[0131] In the dual-path distillation mechanism, the design of the loss function is crucial. This paper proposes a combined loss function consisting of classification loss, distillation loss, and regression loss. The classification loss is used to evaluate the model's ability to distinguish between new and old categories, while the distillation loss measures the difference between the outputs of the student path and the teacher path, ensuring that the student path learns the stable knowledge of old categories from the teacher path. The regression loss is used to optimize the model's prediction of action bounding boxes, improving localization accuracy.

[0132] In an optional embodiment, the preset dual-path distillation mechanism and loss function may specifically include:

[0133] Define a teacher path and a student path, where the teacher path is based on the frozen original model and the student path is based on the trainable new model;

[0134] Construct a combined loss function that integrates classification loss, distillation loss, and regression loss to comprehensively evaluate model performance and guide model optimization;

[0135] During the training process, dual paths are used to jointly process the input data. By comparing the outputs of the teacher path and the student path, the distillation loss is calculated, and the model parameters of the student path are updated accordingly to achieve effective fusion of new and old category knowledge and gradual improvement of model performance.

[0136] In this way, the dual-path distillation mechanism and loss function not only improve the model's recognition accuracy for new categories, but also ensure the model's stable recognition of old categories, providing a more reliable and efficient solution for the timing detection of power grid operation behavior.

[0137] In this embodiment of the present invention, the preset dual-path distillation mechanism and loss function include:

[0138] Preset the cross entropy function based on temperature parameters as the classification probability soft target distillation loss;

[0139] Calculate the classification probability soft target distillation loss when the temperature parameter is two;

[0140] Only the classification output of the original category when no new category data is generated is retained to participate in the loss calculation.

[0141] In this embodiment of the present invention, the preset dual-path distillation mechanism and loss function further include:

[0142] Preset the confidence interval and select the distillation samples based on the candidate boxes in the output of the frozen original model;

[0143] Control the number of distillation samples.

[0144] In an embodiment of the present invention, the preset dual-path distillation mechanism and loss function also include a loss function including R-C3D basic loss, classification probability soft target distillation loss and feature map L2 norm distillation loss.

[0145] Specifically, calculate the classification probability soft target distillation loss L dist , using the cross entropy function with temperature parameter 2:

[0146]

[0147] in is the soft target probability output by the original model, is the regression offset;

[0148] In an optional embodiment, the feature map L2 norm distillation loss L is calculated feat :

[0149]

[0150] in and are the internal features of the original model and the new model respectively;

[0151] Only retain the original action category C o The classification output participates in the loss calculation.

[0152] In an optional embodiment, from M o In the generated candidate frames, RoIs with confidence higher than 0.7 are selected as distillation samples;

[0153] Control the number of distillation samples per video N5≤8, and the filtered RoI carries the original category C o Action information;

[0154] When calculating the classification probability soft target, the temperature parameter T=2 is used, so that:

[0155]

[0156] where z o (k) is the logits output by the original model.

[0157] In an optional embodiment, the incremental learning loss function is defined as:

[0158] L=L rc3d +αL dist +βL feat

[0159] Where α = 0.5, β = 0.3, respectively represents the weight distribution L between each loss function rc3d is the R-C3D base loss.

[0160] It should be noted that the preset dual-path distillation mechanism and the loss function can significantly improve the generalization ability and recognition accuracy of the model by updating the model parameters and optimizing the model architecture according to the dual-path distillation mechanism. By introducing the dual-path structure of the teacher path and the student path, the model can quickly adapt to the addition of new categories while maintaining the stable inheritance of old category knowledge. The teacher path uses the frozen original model to provide stable old category knowledge as a reference for the student path learning, effectively avoiding the common old category forgetting problem in traditional incremental learning. The student path is responsible for learning new category knowledge and can quickly adapt to changes in new categories through flexible expansion.

[0161] In addition, the design of the combined loss function also plays a key role. The classification loss is used to evaluate the model's recognition ability for new and old categories, ensuring that the model can accurately distinguish between different categories of actions. The distillation loss is used to measure the difference between the outputs of the student path and the teacher path. By comparing the outputs of the two, the distillation loss can be calculated, which guides the learning of the student path and enables it to learn the features of new categories while maintaining old category knowledge. The regression loss further optimizes the model's prediction of action bounding boxes, improving positioning accuracy.

[0162] In this way, the preset dual-path distillation mechanism and the loss function not only improve the model's recognition accuracy for new categories, but also ensure stable recognition of old categories. This mechanism enables the model to quickly adapt and accurately recognize new category data while maintaining stable recognition ability for old category data, providing a more reliable and efficient solution for power grid operation behavior timing detection. In practical applications, this mechanism can be widely used in various power grid operation scenarios, making important contributions to the intelligent and automated development of the power industry.

[0163] S104, according to the incremental learning model architecture after model parameter updating and model optimization, the power grid operation behavior timing detection is performed.

[0164] In the embodiment of the present application, the power grid operation behavior timing detection is performed according to the incremental learning model architecture after the model parameter updating and the model optimization, which includes:

[0165] The incremental learning model architecture after the model parameter updating and the model optimization is subjected to non-maximum suppression processing of the candidate frame;

[0166] The detection result is generated by the action classification probability and the regression offset;

[0167] When the termination category is detected or the maximum number of iterations is reached, the final detection sequence is output.

[0168] Exemplary, redundant bounding boxes are eliminated by non-maximum suppression (IoU threshold 0.5);

[0169] Decoding with parameterized bounding boxes:

[0170]

[0171] where μ = 0.5, σ = 0.1 are predefined mean variance parameters of the dataset.

[0172] Figure 2 The overall architecture of a power grid operation behavior timing detection method provided by an embodiment of the application is shown in the schematic diagram, wherein:

[0173] Data input: the original video frame is subjected to feature extraction through a convolutional layer to generate a feature map.

[0174] 3D ROI pooling: a three-dimensional region of interest (ROI) pooling operation is performed on the feature map to obtain a fixed-size feature vector f a .

[0175] The original model inputs the feature vector f a into the original model to output an action classification probability p a and a regression offset r a .

[0176] Foreground and background threshold setting: the foreground threshold is set to 0.8 and the background threshold is set to 0.2 using the prediction result of M0 to filter out potential foreground samples and background samples.

[0177] New and old action sample ratio control: potential foreground samples are filtered out according to the threshold, and the ratio of new and old action samples is controlled to be lower than 1 / 5.

[0178] Bounding box optimization: the bounding box label is optimized through the regression offset of M0 to improve the detection accuracy.

[0179] Expansion of the number of classification layer nodes: in order to adapt to the new class C n , the number of classification layer nodes of M0 is expanded to generate an initialized new model M n .

[0180] Linear layer and activation layer: the new model M n contains a linear layer and an activation layer, which are used to process the expanded classification task.

[0181] Double-path distillation mechanism: synchronous optimization of classification confidence and feature Figure 1 consistency, retaining the original action recognition capability while adapting to the new action.

[0182] Classification probability soft target distillation loss: ensures that the performance of the new model on the original action is not worse than that of the original model.

[0183] Feature map L2 norm distillation loss: maintains the consistency of feature representation between the new and old models.

[0184] Incremental learning loss function: includes R-C3D base loss, classification probability soft target distillation loss and feature map L2 norm distillation loss.

[0185] R-C3D basic loss: including proposal classification and regression loss, action classification and regression loss.

[0186] Classification Probabilistic Soft Target Distillation Loss: Optimizing Classification Confidence.

[0187] Feature map L2 norm distillation loss: optimize features Figure 1 Consistency.

[0188] Iterative training: Through multiple iterative training, the model parameters are continuously optimized to obtain the final incremental learning model.

[0189] Non-maximum suppression (NMS): Processes candidate boxes and generates the final detection results based on action classification probabilities and regression offsets.

[0190] The entire process starts with data input and goes through multiple steps such as feature extraction, model prediction, sample screening, bounding box optimization, model expansion, dual-path distillation, and loss function optimization. Ultimately, it realizes an incremental learning model that can dynamically expand the action category knowledge base and accurately locate multi-stage operation behaviors.

[0191] The power grid operation behavior timing detection method based on incremental learning of the present invention has the following advantages over the existing technology:

[0192] Reduce annotation dependency: Through self-guided sampling strategy, the original model M is used o By filtering the confident samples based on the prediction results, new action adaptation can be achieved with only 5-10 frames of annotation, which reduces the amount of annotation compared to traditional fine-tuning methods.

[0193] Improving detection accuracy: Dual-path distillation mechanism simultaneously optimizes classification confidence and features Figure 1 Consistency significantly improves the accuracy of misoperation detection during tower lifting;

[0194] Maintaining historical action recognition capabilities: Through soft target distillation and feature-level constraints, the representation capabilities of the original actions (such as fastening a seat belt) are retained during the training process of newly added actions, avoiding catastrophic forgetting;

[0195] Enhanced robustness: Parameterized bounding box decoding (μ=0.5, σ=0.1) effectively mitigates interference from sudden changes in illumination and device reflections.

[0196] The application has obvious application value in the field of power grid operation safety supervision, and can realize continuous monitoring of multi-stage actions such as climbing, operation and resetting, thereby providing technical support for power operation risk prevention and control.

[0197] To sum up, the application provides a power grid operation behavior timing detection method, an incremental learning model architecture is constructed, the incremental learning model architecture includes a frozen original model and a trainable new model; the new class data is subjected to self-guided sampling processing to generate training samples; a double-path distillation mechanism and a loss function are preset, the incremental learning model architecture is subjected to model parameter updating and model optimization according to the double-path distillation mechanism; and the power grid operation behavior timing detection is performed according to the incremental learning model architecture after the model parameter updating and the model optimization. The frozen original model ensures the stable inheritance of the existing class knowledge, and avoids the forgetting problem of the old class caused by the new class learning. The trainable new model can quickly adapt to the addition of the new class through flexible expansion, thereby improving the scalability and adaptability of the system. The new class data is subjected to self-guided sampling processing, thereby effectively solving the problem of the scarcity of new class samples and reducing the dependence on a large amount of new class labeled data. The preset of the double-path distillation mechanism and the loss function provides strong guidance for the model parameter updating and the model optimization, thereby ensuring the model performance and stability in the incremental learning process. The power grid operation behavior timing detection is performed according to the optimized incremental learning model architecture, thereby improving the detection accuracy and efficiency and providing a strong guarantee for the safety and reliability of the power grid operation. Through the dynamic expansion of the action class knowledge base and the double-path distillation mechanism, the precise timing positioning of the multi-stage operation behavior is realized under the condition of only a small amount of new action labeled samples, the key features such as the holding posture of the voltage tester and the installation angle of the grounding clamp are positioned, and finally the original operation action (such as the wearing of the insulating gloves) recognition ability is maintained while the rapid adaptation of the new action is realized.

[0198] In one preferred embodiment, three power grid operation behavior data sets are used: a power operation video data set (Power-Video), a substation operation behavior data set (Substation-Behavior) and a transmission line inspection data set (Transmission-Inspection). The application is trained and tested on the action classification, boundary box regression and class expansion tasks.

[0199] The application uses an R-C3D model as a basic detection framework, and the temperature parameter is set to 2.0.

[0200] The mean average precision (mAP) and the intersection over union (IoU) are used as evaluation indexes. The calculation process of the mAP is as follows: the model calculates the area under the precision-recall curve of the detection result of each action class, and finally takes the average value of all classes. The formula is as follows:

[0201]

[0202] Where C is the total number of categories, R is the number of recall thresholds, and P(r) is the precision at the rth recall.

[0203] The calculation formula for IoU is:

[0204]

[0205] Where A is the predicted box and B is the true box.

[0206] The present invention conducted multiple comparative experiments on the task of detecting the timing of power grid operation behavior, verifying the effectiveness of the proposed method. The following three methods were designed to comprehensively evaluate the performance advantages of the present invention:

[0207] 1. Direct Training (Direct)

[0208] Method description: In the new category dataset D n Directly train the new model M n , without any parameter freezing or adjustment.

[0209] Purpose: This baseline is used to verify catastrophic forgetting, which is when a model's performance on the original categories significantly degrades after training on newly added categories.

[0210] Implementation details: New model M n The number of nodes in the classification layer is expanded to adapt to the new category C n , but no incremental learning strategy was adopted.

[0211] 2. Fine-tuning (FT)

[0212] Method description: Freeze the original model M o The parameters of the action classification layer and regression layer (i.e., linear2 and linear3) are adjusted, and only some parameters of the extension are fine-tuned, and a low learning rate is used for training.

[0213] Purpose: To alleviate the large drift of the original model parameters and avoid damaging the performance of old categories.

[0214] Implementation details: The classification and regression layers of the original model are frozen, and only the newly extended classification layer is fine-tuned with a learning rate set to 1e-4.

[0215] 3. Joint Training

[0216] Method description: Using the original dataset D o And new category dataset Dn The complete R-C3D model is retrained on the joint data.

[0217] Purpose: To serve as a performance upper limit benchmark to compare the upper limit performance of incremental learning methods.

[0218] Implementation details: Retrain the entire model, including the original category C o and new category C n All parameters of .

[0219] Table 1 shows the performance of various detection models on the Power-Video, Substation-Behavior, and Transmission-Inspection datasets.

[0220] Table 1 Grid operation behavior detection results

[0221] method Power-Video Substation-Behavior Transmission-Inspection Direct Training 69.1 68.5 61.8 Fine-tuning 73.4 72.3 65.2 Joint Training 77.8 79.1 71.9 The present invention 77.3 79.6 71.2

[0222] The experimental results show that:

[0223] On the Power-Video dataset, the mAP of the proposed method reaches 77.3%, an increase of 8.2 percentage points over the baseline model.

[0224] On the Substation-Behavior dataset, the mAP reaches 79.6%, an increase of 8.1 percentage points over the baseline model.

[0225] On the Transmission-Inspection dataset, the detection accuracy is improved to 71.2%, which is better than the existing methods.

[0226] In summary, the present invention uses a dual-path distillation mechanism to make the power grid operation behavior timing detection performance close to the upper limit of joint training, while significantly reducing the computational cost.

[0227] This method effectively controls the ratio of new and old category samples through a self-guided sampling mechanism, enabling the model to effectively learn features of new categories while maintaining the detection performance of existing categories. A dual-path distillation mechanism significantly improves the model's generalization capability by jointly optimizing the soft target of classification probabilities and the L2 norm of feature maps. Experimental results demonstrate the effectiveness of this method in detecting power grid operation behaviors.

[0228] Example 3, reference Figure 3 , this embodiment also provides a power grid operation behavior timing detection system, including:

[0229] The model architecture building module is used to build an incremental learning model architecture, which includes a frozen original model and a trainable new model;

[0230] The frozen original model is used to generate new category data, where the new category data is labeled data including new categories;

[0231] The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer;

[0232] The sample generation module is used to perform self-guided sampling processing on new category data to generate training samples;

[0233] The optimization module is used to preset the dual-path distillation mechanism and loss function, and update the model parameters and optimize the incremental learning model architecture based on the dual-path distillation mechanism;

[0234] The detection module is used to perform timing detection of power grid operation behavior based on the incremental learning model architecture after model parameter updates and model optimization.

[0235] The above-mentioned unit modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0236] This embodiment also provides an electronic device, which may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 3 As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for detecting the timing of power grid operation behavior is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse.

[0237] This embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following steps are implemented:

[0238] Build an incremental learning model architecture, which includes a frozen original model and a trainable new model;

[0239] The frozen original model is used to generate new category data, where the new category data is labeled data including new categories;

[0240] The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer;

[0241] Perform self-guided sampling on new category data to generate training samples;

[0242] Preset dual-path distillation mechanism and loss function, and update model parameters and optimize the incremental learning model architecture based on the dual-path distillation mechanism;

[0243] The grid operation behavior timing detection is performed based on the incremental learning model architecture after the model parameters are updated and the model optimization is completed.

[0244] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting the timing sequence of power grid operation behavior, characterized in that: include: Building an incremental learning model architecture, the incremental learning model architecture including a frozen original model and a trainable new model; The frozen original model is used to generate new category data, where the new category data is labeled data including new categories; The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer; Perform self-guided sampling on new category data to generate training samples; Preset a dual-path distillation mechanism and loss function, and perform model parameter updates and model optimization on the incremental learning model architecture according to the dual-path distillation mechanism; The grid operation behavior timing detection is performed based on the incremental learning model architecture after the model parameters are updated and the model optimization is completed.

2. A method for detecting a time sequence of a power grid operation behavior according to claim 1, characterized in that: The incremental learning model architecture is constructed as follows: Acquire an original action category set and annotation data for the original action category set; Training the frozen original model based on the labeled data of the original action category set; The loss function of the frozen original model is a combination of several loss functions; The number of nodes in the classification layer of the frozen original model is expanded to adapt to the new category, and a new trainable model is generated.

3. A method for detecting a time sequence of a power grid operation behavior according to claim 2, characterized in that: The self-guided sampling process for the new category data to generate training samples includes: generating new category data according to the frozen original model; Preset threshold parameters for new category data; The threshold parameter is used to screen foreground samples in new category data; Preset regression offset, optimize the filtered new category data, and generate training samples.

4. A method for detecting a time sequence of a power grid operation behavior according to claim 3, characterized in that: The preset dual-path distillation mechanism and loss function include: Preset the cross entropy function based on temperature parameters as the classification probability soft target distillation loss; Calculate the classification probability soft target distillation loss when the temperature parameter is two; Only the classification output of the original category when no new category data is generated is retained to participate in the loss calculation.

5. A method for detecting a time sequence of a power grid operation behavior according to claim 4, characterized in that: The preset dual-path distillation mechanism and loss function also include: Presetting a confidence interval, and selecting distilled samples based on the candidate boxes in the output of the original model; Control the number of distillation samples.

6. A method for detecting a time sequence of a power grid operation behavior according to claim 5, characterized in that: The time series detection of power grid operation behavior according to the incremental learning model architecture after the model parameter update and model optimization is completed includes: After model parameter updates and model optimization, the incremental learning model architecture uses non-maximum suppression to process candidate boxes; Generate detection results through action classification probability and regression offset; The final detection sequence is output when the termination category is detected or the maximum number of iterations is reached.

7. A method for detecting a time sequence of a power grid operation behavior according to claim 6, characterized in that: The preset dual-path distillation mechanism and loss function also include the loss function including R-C3D basic loss, classification probability soft target distillation loss and feature map L2 norm distillation loss.

8. A power grid operation behavior timing detection system, applying the method according to any one of claims 1 to 7, characterized in that: include: A model architecture building module, configured to construct an incremental learning model architecture, wherein the incremental learning model architecture includes a frozen original model and a trainable new model; The frozen original model is used to generate new category data, where the new category data is labeled data including new categories; The trainable new model adapts to the new category by expanding the number of nodes in the original classification output layer; The sample generation module is used to perform self-guided sampling processing on new category data to generate training samples; An optimization module, configured to preset a dual-path distillation mechanism and a loss function, and to perform model parameter updates and model optimization on the incremental learning model architecture according to the dual-path distillation mechanism; The detection module is used to perform timing detection of power grid operation behavior based on the incremental learning model architecture after model parameter updates and model optimization.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a method for detecting a time sequence of power grid operation behavior according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for detecting a time sequence of power grid operation behavior according to any one of claims 1 to 7 are implemented.

Images