Grid operation small sample violation behavior identification method, system and device based on transfer learning and saliency model fine tuning, and medium

Abstract

The invention discloses a power grid operation small sample violation behavior recognition method, system and device based on transfer learning and saliency model fine tuning and a medium, and belongs to the technical field of power operation behavior recognition, and the method comprises the steps: collecting field operation supervision video data, marking the data, and dividing the data into small sample data and large sample data; selecting a backbone network of the classification model, loading a pre-training weight of the classification model on the public data set, screening a network layer needing fine tuning according to a first index, and performing first-stage fine tuning training on the selected network layer by using large sample data; and measuring the difference between the fine-tuned network weight and the initial pre-training weight according to a second index, and adding a parameter mapping block behind the high-difference network layer. The output of the last parameter mapping block is sent to a small sample decision-making layer for distinguishing small sample categories, and two-stage small sample fine tuning training is carried out; and based on the trained complete network model, performing violation behavior identification on the operation video data in the power grid scene to obtain a test result.

Description

Technical Field

[0001] This invention relates to the field of power operation behavior recognition technology, specifically to a method, system, equipment, and medium for identifying small-sample violations in power grid operations based on transfer learning and fine-tuning of saliency models. Background Technology

[0002] With the development of power grid construction and renovation projects, power construction sites are widely distributed, often involving multiple construction teams, making safety risk management increasingly challenging. Consequently, violations in power grid operations typically involve operator carelessness, non-compliance with regulations, unfamiliarity with operating procedures, or equipment malfunctions. These behaviors can seriously impact the safety and reliability of the power system.

[0003] Currently, traditional power grid monitoring mainly relies on manual inspections and video playback to identify and prevent violations. This approach not only requires a significant investment of human resources but also suffers from inefficiency, subjectivity, and fatigue. Especially in complex and ever-changing power grid operation scenarios, some potential violations are often difficult to detect in a timely manner, posing serious safety hazards to power system operation and maintenance. Therefore, there is an urgent need to introduce intelligent monitoring technology, using computer vision and deep learning algorithms to achieve automatic identification and early warning of power grid operation behaviors, thereby significantly improving the safety, compliance, and efficiency of power system operation and maintenance. Furthermore, due to the complexity of power grid operations, new violations frequently occur, causing performance degradation of existing violation identification models for power grid operation scenarios. Moreover, the limited data containing new violations makes it difficult to iterate and update existing models, making them unsuitable for complex and ever-changing power grid scenarios. Even with sufficient data on new violations, iteratively updating all model parameters consumes enormous resources. Therefore, it is essential to research a model with strong generalization and rapid iterative updates based on transfer learning and saliency model fine-tuning strategies.

[0004] To achieve fine-tuning of the model, Ma Yanan proposed a hierarchical action recognition method based on semantic feature transfer that does not require additional annotation costs. This method leverages feature transfer to mine and utilize the inherent natural structures between categories. Without increasing annotation costs, it fully exploits the correlations between action categories in existing action recognition datasets, providing reasonable guidance and constraints for the learning of visual features (Ma Yanan. Research on Action Recognition Based on Deep Transfer Learning [D]. University of Chinese Academy of Sciences (Institute of Artificial Intelligence, Chinese Academy of Sciences), 2021. DOI:10.27824 / d.cnki.gzkdx.2020.000017.). Wang Jiahao et al. proposed a behavior recognition algorithm based on meta-learning and attention. This algorithm applies the ideas and methods of meta-learning to the field of behavior recognition, using meta-learning to train several tasks, learn prior knowledge, and summarize it to obtain initial parameters with high generalization ability. This allows the model to quickly update to a higher accuracy using only a small amount of new data (Wang Jiahao, Zhong Xin, Li Wenxiong et al. Research on User Behavior Recognition Based on Meta-Learning and Attention Mechanism [J]. Computer Science, 2023, 50(08):193-201.). Huang Wenqi et al. also proposed a defect detection method based on meta-learning. This method involves data augmentation processing of multiple power grid images to be detected to obtain a set of power grid images. A first power grid image is determined from this set, and second power grid images are determined from each subset of power grid images. The first and second power grid images are then input into a preset recognition model for feature extraction and defect detection to obtain the defect detection results. (Huang Wenqi, Zeng Qunsheng, Wu Yang, et al. Method, device and equipment for few-shot image defect detection based on meta-learning [P]. Guangdong Province: CN115564773B, 2023-04-18.) Jiang W et al. proposed a few-shot learning method based on multi-scale metric learning, which extracts multi-scale features, learns multi-scale relationships between samples, performs few-shot learning classification, and proposes a new loss function, namely intra-class and inter-class relationship loss, to optimize the proposed deep network. It has achieved good performance on the few-shot learning problem (Jiang W, Huang K, Geng J, et al. Multi-scale metric learning for few-shot learning [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(3):1091-1102.).Among these related methods, scholars have obtained initial parameters with high generalization ability by designing training methods, or improved the features extracted from small sample data by designing feature extraction parts. However, there is a lack of objective standards for the selection of fine-tuning network layers, and the model design and training costs are still relatively high, which greatly restricts the model detection performance and deployment efficiency, and cannot effectively adapt to complex and ever-changing power grid application scenarios. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by this invention is: how to design a two-stage fine-tuning training strategy for the complex and ever-changing power grid scenario, to slightly adjust the network parameters for the power grid scenario, to detect newly added violations, and to improve the performance of the behavior recognition model with a small sample size, model design cost and training cost, and further improve the model iteration update efficiency, thereby improving the accuracy and efficiency of identifying potential violations in typical power grid operation scenarios.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a method for identifying small-sample violations in power grid operations based on transfer learning and fine-tuning of saliency models, comprising,

[0008] Collect and label video data of on-site operation supervision in power grid scenarios, construct a dataset for identifying violations, and divide the target category into small sample data and large sample data.

[0009] Select the backbone network of the classification model, load the pre-trained weights of the classification model on the public dataset, select the network layers that need to be fine-tuned according to the first indicator, freeze the parameters of other network layers, and use large sample data to perform a one-stage fine-tuning training on the selected network layers.

[0010] Based on the second metric measuring the difference between the fine-tuned network weights and the initial pre-trained weights, a parameter mapping block is added after the high-discretionary network layer. The output of the last parameter mapping block is fed into the few-shot decision layer used to distinguish few-shot categories, and a two-stage few-shot fine-tuning training is performed on the high-discretionary network layer, the parameter mapping block, and the few-shot decision layer.

[0011] Based on the trained complete network model, the system identifies violations in operation video data under power grid scenarios and obtains test results.

[0012] Furthermore, the construction of the violation behavior recognition dataset and the division of target categories from it into small sample data and large sample data includes,

[0013] Video data of on-site operation supervision in power grid operation scenarios are collected and labeled to construct an initial dataset for identifying violations. The initial dataset is then classified according to different violations, and the sample size of different violations is counted. A violation with a small sample size is selected to simulate newly added violations in the power grid scenario. All videos of the current category are used as small sample data, and videos of other violations are used as large sample data.

[0014] Furthermore, the step of using the classification model to select network layers requiring fine-tuning based on the network's pre-trained weights on a public dataset, freezing the parameters of other network layers, and performing a one-stage fine-tuning training on the selected network layers using large sample data includes:

[0015] A classification backbone network with publicly available pre-trained weights is selected. Quantitative power grid sample data is fed into the classification network with pre-trained weights. The activation probability of each neuron during forward propagation is statistically analyzed as the primary indicator. Under-activated and over-activated network layers are screened out. The parameters of other network layers are frozen. Large sample data is used to perform a one-stage fine-tuning training on the under-activated and over-activated network layers.

[0016] The beneficial effects of this preferred technical solution are as follows: by screening out underactivated and overactivated network layers based on the first index, and only performing one-stage fine-tuning training on these parts, the number of parameters that need to be trained is effectively reduced, the computational resource consumption and time overhead of model training are significantly reduced, and unnecessary interference to non-critical network layer parameters is avoided, which helps to maintain the stability of the backbone network's recognition performance for large sample categories.

[0017] Furthermore, the step of measuring the difference between the fine-tuned network weights and the initial pre-trained weights according to the second metric, and adding a parameter mapping block after the high-discrepancy network layer, includes:

[0018] The parameter mapping block and the few-sample decision layer constitute a discriminative structure for processing the target few-sample category. The output of the high-discrepancy network layer is used as input, and the parameter mapping block performs feature transformation. The transformed output is then passed to the few-sample decision layer to generate the discrimination result of the target category.

[0019] In the two-stage fine-tuning training, only the parameters of the current discriminant structure and the high-discrepancy network layer are updated, while the parameters of the rest remain unchanged.

[0020] Furthermore, the selected quantitative power grid sample data is fed into a classification network loaded with pre-trained weights. The activation probability of each neuron during forward propagation is used as the first indicator to screen out underactivated and overactivated network layers.

[0021] Quantitative power grid sample data should cover various violations. The first indicator is the activation probability of neurons, which is calculated based on the proportion of activation functions that output a value greater than zero given input data. The expression is:

[0022]

[0023] in, Let C represent the activation probability of the l-th layer, N be the number of input data samples, and C be the activation probability of the l-th layer. l H l W l These represent the number of channels, height, and width of the l-th layer, respectively. This represents the activation function output at position (h, w) for the i-th sample, c-th channel, in layer l. It is an indicator function that returns 1 if the condition is met, and 0 otherwise.

[0024] Furthermore, the selected second metric measures the difference in network weights before and after the first-stage fine-tuning, and the network layers with high differences are selected, including:

[0025] The second metric uses the KL divergence value. The specific calculation method is as follows: Gaussian kernel density estimation is used to fit the probability density function of the weights in each layer. Based on the probability density function of the weights, the KL divergence of the network weights before and after the first-stage fine-tuning is calculated. The expression is:

[0026]

[0027] in, This represents the probability density function of the weights of the l-th layer before the first-stage fine-tuning. χ represents the probability density function of the weights of the l-th layer after one stage of fine-tuning, and χ represents the value space of the weights. A network layer with a KL divergence value greater than a given threshold is considered a highly disparate network layer.

[0028] Furthermore, the parameter mapping block and the few-sample decision layer constitute a discriminative structure for processing the target few-sample category. The output of the differential network layer is used as input, and feature transformation is performed by the parameter mapping block. The transformed output is then passed to the few-sample decision layer to generate the discrimination result for the few-sample target category.

[0029] A parameter mapping block is added after the high-discrepancy network layer. Each parameter mapping block consists of a stack of convolutional layers, BN layers, and ReLU layers.

[0030] The input features of the high-discrepancy network layer are used as the residual feature vector, and the output features of the high-discrepancy network layer are used as the unmapped feature vector. Both are fed into the parameter mapping block. The calculation formulas involved in the parameter mapping block are as follows:

[0031] T i =M i (F i ,R i ),

[0032] Among them, Fi R is the unmapped feature vector in the i-th parameter mapping block. i T is the residual feature vector in the i-th parameter mapping block. i M is the mapped feature vector in the i-th parameter mapping block. i (·) represents a compound operation in the i-th parameter mapping block;

[0033] The few-shot decision layer is a standalone classification layer composed of fully connected layers. It receives the output of the last parameter mapping block, predicts the probability of adding a new few-shot class, and concatenates this prediction with the results from the original classification network's prediction layer to obtain the final output value.

[0034] This invention provides a small sample violation identification system for power grid operations based on transfer learning and fine-tuning of saliency models.

[0035] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a small sample violation behavior identification system for power grid operations based on transfer learning and saliency model fine-tuning, comprising: a data collection module, a first-stage fine-tuning module, a second-stage fine-tuning module, and a result output module;

[0036] The data collection module collects and annotates video data of on-site operation supervision in power grid scenarios, constructs a dataset for identifying violations, and divides it into small sample data and large sample data for target categories.

[0037] The first stage fine-tuning module selects the backbone network of the classification model, loads the pre-trained weights of the classification model on the public dataset, selects the network layers that need to be fine-tuned according to the first indicator, freezes the parameters of other network layers, and uses large sample data to perform a one-stage fine-tuning training on the selected network layers.

[0038] The second-stage fine-tuning module measures the difference between the fine-tuned network weights and the initial pre-trained weights based on a second metric, and adds a parameter mapping block after the high-discrepancy network layer. The output of the last parameter mapping block is fed into the few-shot decision layer used to distinguish few-shot categories, and the high-discrepancy network layer, parameter mapping block, and few-shot decision layer are trained in two stages for few-shot fine-tuning.

[0039] The result output module is based on a trained complete network model to identify violations in video data under power grid scenarios and obtain test results.

[0040] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the method for identifying small sample violations in power grid operations based on transfer learning and fine-tuning of saliency models.

[0041] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the method for identifying small sample violations in power grid operations based on transfer learning and fine-tuning of a saliency model.

[0042] The beneficial effects of this invention are as follows: In view of the complex and ever-changing power grid scenarios, this invention designs a two-stage fine-tuning training strategy, which slightly adjusts the network parameters for the power grid scenario and detects newly added violations. With a small sample size, model design cost and training cost, the performance of the behavior recognition model is improved, and the efficiency of model iteration and update is further improved, thereby improving the accuracy and efficiency of identifying potential violations in typical power grid operation scenarios. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 The above is a flowchart of the overall process for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning, as provided in one embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of a ResNet block containing a high-discrepancy network layer in a method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning, provided by an embodiment of the present invention.

[0046] Figure 3 This is a parameter mapping block structure diagram in a method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning, provided by an embodiment of the present invention.

[0047] Figure 4 This is an overall network structure diagram of a method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning, provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0049] Example 1, referring to Figure 1 As one embodiment of the present invention, this embodiment provides a method for identifying small-sample violations in power grid operations based on transfer learning and fine-tuning of saliency models, including:

[0050] S1. Collect and label video data of on-site operation supervision in power grid scenarios, construct a dataset for identifying violations, and divide the target category into small sample data and large sample data.

[0051] S2. Select the backbone network of the classification model, load the pre-trained weights of the classification model on the public dataset, select the network layers that need to be fine-tuned according to the first indicator, freeze the parameters of other network layers, and use large sample data to perform a one-stage fine-tuning training on the selected network layers.

[0052] S3. Based on the second metric, measure the difference between the fine-tuned network weights and the initial pre-trained weights, and add a parameter mapping block after the high-discretionary network layer. Feed the output of the last parameter mapping block into the few-sample decision layer used to distinguish few-sample categories, and perform two-stage few-sample fine-tuning training on the high-discretionary network layer, the parameter mapping block, and the few-sample decision layer.

[0053] S4. Based on the trained complete network model, identify violations in operation video data under power grid scenarios and obtain test results.

[0054] With the continuous advancement of power grid construction and renovation, on-site operation scenarios are becoming increasingly complex, and workers may engage in various violations under different working conditions. Traditional power grid operation behavior recognition methods typically rely on large-scale sample training to form classification models. When new or rare violation categories appear on-site, the existing models are significantly insufficient in their recognition ability for these categories due to the extremely small number of data samples. Furthermore, to adapt to new categories, existing methods often require retraining the entire model structure or updating a large range of parameters. This not only leads to high training costs and high computational resource consumption but also easily damages the recognition performance of the original large-sample categories, affecting the overall stability and reliability of the model. Therefore, how to effectively adapt to small-sample new categories and achieve efficient iterative updates while ensuring the model's ability to recognize large-sample categories has become an urgent problem to be solved in power grid operation small-sample violation behavior recognition technology.

[0055] This invention first uses the pre-trained weights of the classification model on a public dataset to select network layers requiring fine-tuning based on a first metric. Only key layers undergo one-stage fine-tuning training, reducing training costs and computational overhead while maintaining the stability of the original model's core discrimination ability. Subsequently, this invention uses a second metric to measure the change in network weights before and after the one-stage fine-tuning, identifying highly dissimilar network layers. Parameter mapping blocks are then added to these layers, and a small-sample decision layer is constructed to adapt to newly added small-sample categories. This scheme achieves rapid adaptation and efficient updating of small-sample categories through two-stage fine-tuning of the local structure, avoiding the adverse effects of updating all network parameters on the original recognition performance, and improving the accuracy and model iteration efficiency of small-sample violation recognition in power grid operation scenarios.

[0056] Example 2, an embodiment of the present invention, provides a method for identifying small-sample violations in power grid operations based on transfer learning and fine-tuning of a saliency model, based on the previous embodiment, including:

[0057] S1. Collect and label video data of on-site operation supervision in power grid scenarios, construct a dataset for identifying violations, and divide the target category into small sample data and large sample data.

[0058] Video data from on-site inspections of power grid operations were collected. Professional personnel annotated the video data to construct a violation identification dataset, categorizing the video data according to the type of violation. A category with fewer video samples was selected to simulate newly introduced violations in the power grid scenario. All videos of this category were used as a small sample dataset, while videos from other categories were selected as large sample datasets.

[0059] Step S11: Collect on-site operation supervision video data in power grid operation scenarios, and have professionals annotate this video data to construct an initial dataset for identifying violations.

[0060] Step S12: Classify the initial dataset according to different violations and count the sample size of different violations.

[0061] Step S13: Select a type of violation with a small sample size to simulate newly added violations in the power grid scenario, use all videos of this type as small sample data, and select videos of other violations as large sample data.

[0062] Preferably, in step S11, video data containing worker actions in different scenarios are collected in typical power grid operation scenarios (e.g., power outage operation on a single distribution line), and labeled by professionals to obtain an initial dataset. The labeled actions include climbing, voltage testing, and grounding wire installation.

[0063] It is worth noting that, in the preferred embodiment of the present invention, collecting videos of typical power grid operation scenarios as a dataset is a non-limiting but preferred choice. Those skilled in the art can collect target videos from different scenarios as the original dataset according to the recognition requirements. Selecting a small sample size of action to simulate the situation after slight changes in the power grid scenario is also a non-limiting but preferred choice. Those skilled in the art can select different objective factors according to the actual situation.

[0064] S2. Select the backbone network of the classification model, load the pre-trained weights of the classification model on the public dataset, select the network layers that need to be fine-tuned according to the first indicator, freeze the parameters of other network layers, and use large sample data to perform a one-stage fine-tuning training on the selected network layers.

[0065] In a preferred embodiment of the present invention, the first indicator is the activation probability of a neuron.

[0066] Step S21: Select a classification backbone network with publicly available pre-trained weights.

[0067] In a preferred embodiment of the present invention, ResNet-50 is selected as the classification backbone network, and its pre-trained weights on the ImageNet dataset are selected.

[0068] It is worth noting that, in the preferred embodiment of the present invention, the selection of the classification backbone network is a non-limiting but preferred choice. Those skilled in the art can select different classification models and pre-training weights according to different situations.

[0069] Step S22: Select quantitative power grid sample data and feed it into a classification network with pre-trained weights. Calculate the activation probability of each neuron during the forward propagation process and screen out underactivated and overactivated network layers.

[0070] Step S23: Freeze the parameters of other network layers and use large sample data to perform a one-stage fine-tuning training on the underactivated and overactivated network layers.

[0071] Preferably, if Figure 2 As shown, in step S22, the quantitative power grid operation video sample data comes from large sample data. The first indicator is the activation probability of neurons, which is calculated based on the proportion of activation functions that output a value greater than zero under given input data. The expression is:

[0072]

[0073] in, Let C represent the activation probability of the l-th layer, N be the number of input data samples, and C be the activation probability of the l-th layer. l H l W l These represent the number of channels, height, and width of the l-th layer, respectively. This represents the activation function output at position (h, w) for the i-th sample, c-th channel, in layer l. It is an indicator function that returns 1 if the condition is met, and 0 otherwise.

[0074] In an optional embodiment of the present invention, the first indicator is: based on the output variance of neurons under large sample data input, the sensitivity of the layer to changes in input data is reflected by statistically analyzing the variance of the output values ​​of neurons in each network layer. Network layers with output variance below a first threshold are selected as underactivated network layers, and network layers with output variance above a second threshold are selected as overactivated network layers.

[0075] In practice, a suitable amount of power grid sample data is selected and fed into a classification network with pre-trained weights. The output variance of each layer of neurons is calculated, and the target network layer is selected according to the preset upper and lower thresholds. This is used to freeze the parameters of other network layers in the future, and only the target network layer is subjected to a one-stage fine-tuning training.

[0076] In one optional embodiment of the present invention, the first indicator is: based on the average activation amplitude of each network layer channel, the response intensity of each layer channel is evaluated by statistically analyzing the average activation values ​​of each layer channel during forward propagation using large sample data. Network layers with an average activation amplitude less than a preset lower threshold are identified as underactivated network layers; network layers with an average activation amplitude greater than a preset upper threshold are identified as overactivated network layers. In this embodiment, after selecting the target network layer, the parameters of other layers are frozen, and only the selected network layer undergoes a one-stage fine-tuning training using large sample data to guide the network in extracting features from the large sample data.

[0077] Furthermore, the current preferred embodiment uses the activation probability of neurons during forward propagation as the primary indicator to screen out underactivated and overactivated network layers. In the first stage of training, only the parameters of these underactivated layers are updated. This not only effectively reduces the number of parameters that need to be adjusted during model training, lowering computational resource consumption and training time, but also avoids unnecessary interference to stable feature extraction layers in the backbone network related to large sample categories, ensuring the recognition accuracy of the original large sample categories and the stability of the backbone structure. Simultaneously, this approach makes fine-tuning training more targeted, improving the model's efficiency in optimizing key network layer parameters and providing an efficient and robust foundation for subsequent small sample category adaptation.

[0078] S3. Based on the second metric, measure the difference between the fine-tuned network weights and the initial pre-trained weights, and add a parameter mapping block after the high-discrepancy network layer. Feed the output of the last parameter mapping block into the few-shot decision layer used to distinguish few-shot categories, and perform two-stage few-shot fine-tuning training on the high-discrepancy network layer, the parameter mapping block, and the few-shot decision layer.

[0079] In a preferred embodiment of the present invention, the second metric is: using KL divergence as the metric to measure the difference between the fine-tuned network weights and the initial pre-trained weights, and adding a parameter mapping block after the high-discrepancy network layer. The output of the last parameter mapping block is fed into the few-shot decision layer to distinguish the newly added few-shot categories. The parameters of other network layers are frozen, and two-stage fine-tuning training is performed on the high-discrepancy network layer, the parameter mapping block, and the few-shot decision layer using few-shot data.

[0080] Preferably, step S3 includes: step S31, using KL divergence as an indicator to measure the difference in network weights before and after the first-stage fine-tuning, and screening out network layers with high divergence.

[0081] Step S32: Add a parameter mapping block after the high-discrepancy network layer. Feed the output of the last parameter mapping block into the few-sample decision layer to distinguish newly added few-sample categories.

[0082] Step S33: Freeze the parameters of other network layers, and use small sample data to perform two-stage small sample fine-tuning training on the high-discrepancy network layer, parameter mapping block and small sample decision layer.

[0083] Preferably, in step S31, a Gaussian kernel density estimation is used to fit the probability density function of the weight values ​​for each layer, and the KL divergence of the network weights before and after the first-stage fine-tuning is calculated based on the probability density function of the weights. The calculation formula is as follows:

[0084]

[0085] in, This represents the probability density function of the weights of the l-th layer before the first-stage fine-tuning. χ represents the probability density function of the weights of the l-th layer after one stage of fine-tuning, and χ represents the value space of the weights. A network layer with a KL divergence value greater than a given threshold is considered a highly disparate network layer.

[0086] In a preferred embodiment of the present invention, the KL divergence screening threshold is set to 0.15. Network layers with KL divergence greater than this threshold have a large difference in weights before and after fine-tuning and are judged as high-difference network layers.

[0087] Preferably, if Figure 2 As shown, in step S32, a parameter mapping block is added after the high-difference network layer. The structure of the parameter mapping block is as follows: Figure 3 As shown, each parameter mapping block consists of stacked convolutional layers, batch normalization (BN) layers, and ReLU layers. The input features of the high-discrepancy network layers are used as residual feature vectors, and the output features of the high-discrepancy network layers are used as unmapped feature vectors, both fed into the parameter mapping block. The calculation formulas involved in the parameter mapping block are as follows:

[0088] T i=M i (F i ,R i ),

[0089] Among them, F i R is the unmapped feature vector in the i-th parameter mapping block. i T is the residual feature vector in the i-th parameter mapping block. i M is the mapped feature vector in the i-th parameter mapping block. i (·) represents a compound operation in the i-th parameter mapping block;

[0090] Preferably, if Figure 4 As shown, in step S32, the few-sample decision layer is a separate classification layer composed of fully connected layers. The few-sample decision layer receives the output of the last parameter mapping block, predicts the probability of adding a new few-sample category, and concatenates it with the prediction layer result of the original classification network as the final output value.

[0091] In one optional embodiment of the present invention, the second metric is the change in the average weight of each network layer before and after the first-stage fine-tuning training. Specifically, the average weight of each network layer before and after the first-stage fine-tuning is extracted, its change is calculated, and network layers with a change greater than a preset threshold are selected as high-discrepancy network layers. A parameter mapping block is added after the high-discrepancy network layer, and the output of the last parameter mapping block is sent to the few-sample decision layer to distinguish the newly added few-sample categories. The parameters of other network layers are frozen, and the high-discrepancy network layer, parameter mapping block, and few-sample decision layer are used for two-stage fine-tuning training using few-sample data to adapt to the target few-sample category discrimination task.

[0092] In one optional embodiment of the present invention, the second metric is the change in the variance of the weights of each network layer before and after a one-stage fine-tuning training. Specifically, statistical analysis is performed on the distribution variance of the weights of each network layer before and after the one-stage fine-tuning, the change magnitude is calculated, and network layers with change magnitudes exceeding a preset threshold are identified as high-discrepancy network layers. A parameter mapping block is added after the high-discrepancy network layer, and the output of the parameter mapping block is sent to the small sample decision layer to distinguish the newly added small sample categories. Under the condition of freezing the parameters of other network layers, two-stage fine-tuning training is performed on the high-discrepancy network layer, the parameter mapping block, and the small sample decision layer using small sample data to improve the model's ability to distinguish small sample categories.

[0093] It should be further explained that the beneficial effect of the current preferred embodiment is that, after the first-stage fine-tuning training, KL divergence is used as a second indicator to measure the change in network weights, accurately identifying high-discrepancy network layers with significant parameter changes. Parameter mapping blocks are added after these high-discrepancy network layers, and a few-sample decision layer is constructed to form a discrimination path specifically for few-sample categories. In the second-stage training, only this path and related network layer parameters are optimized. This approach effectively reduces interference with global network parameters, prevents damage to the backbone network's ability to discriminate large-sample categories, and simultaneously achieves rapid adaptation and high-precision discrimination of newly added small-sample categories, significantly improving the performance of small-sample category recognition and significantly increasing the efficiency of model iteration updates.

[0094] S4. Based on the trained complete network model, identify violations in operation video data under power grid scenarios and obtain test results.

[0095] Preferably, in step S4, the classification model trained in two stages is deployed on the test system, and the test set data is used as input to perform violation behavior recognition and obtain the recognition result.

[0096] Example 3 is an embodiment of the present invention, which provides a method for identifying small sample violations in power grid operations based on transfer learning and fine-tuning of saliency models. To verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0097] In a preferred embodiment of the present invention, the initial dataset contains video data from 200 typical power grid operation scenarios. Among them, 42 videos involve violations of climbing at heights, 83 videos involve violations of voltage testing, and 75 videos involve violations of grounding wire connection. Therefore, the videos involving voltage testing and grounding wire connection are used as the large sample data, and the videos involving climbing at heights are used as the small sample data. Before iterative training, the dataset is randomly divided into a training set and a test set in an 8:2 ratio. The training set contains 126 videos of large sample data and 34 videos of small sample data. The test set contains 32 videos of large sample data and 8 videos of small sample data.

[0098] In a preferred embodiment of the invention, 10 videos of voltage testing and 10 videos of grounding wire installation were selected as input data and fed into a classification network with pre-trained weights. Activation thresholds were set to 0.2 and 0.8, respectively. Network layers with activation probabilities less than 0.2 were designated as underactivated layers, and those with activation probabilities greater than 0.8 were designated as overactivated layers. The first-stage fine-tuning training used a large sample dataset containing 126 videos, including voltage testing and grounding wire installation actions. Other network layer parameters were frozen, and only the parameters of the underactivated and overactivated layers were trained. The SGD optimizer was used as the training optimizer, with an initial learning rate of 3e-5 and a batch size of 16 samples. Training lasted for 20 epochs.

[0099] In a preferred embodiment of the invention, the fully connected layer of the classification network outputs a two-dimensional vector, representing the predicted values ​​of the voltage detection and grounding actions, respectively. The output of the fully connected layer in the newly added few-sample decision layer is a one-dimensional vector, representing the predicted value of the climbing action. The two-stage fine-tuning training uses 34 videos of few-sample data, containing only climbing actions. The parameters of other network layers are frozen, and only the high-discrepancy network layer, parameter mapping block, and few-sample decision layer parameters are trained. Simultaneously, the SGD optimizer is used as the training optimizer, with an initial learning rate set to 5e-4 and a batch size of 8 samples. After the 10th training epoch, the learning rate becomes 0.1 times the original value, and a total of 20 training epochs are performed.

[0100] The violation action recognition model obtained in the preferred embodiment of the present invention was tested using 40 test videos containing three categories of violation actions in typical power grid operation scenarios. These included 32 videos of grounding wire installation and voltage testing, and 8 videos of climbing at heights. The accuracy of the model in recognizing various violation action categories was recorded, and the indicators are shown in Table 1.

[0101] In a preferred embodiment of the present invention, the trained model is deployed on a testing system, and the generalization results of the model's action recognition in typical power grid operation scenarios are obtained using test set data. The testing system is configured with Python 3.8, PyTorch 1.9.0, CUDA 11.2, and PyCharm 2023.2 as its software, and with an RTX-3090 graphics card (24GB VRAM) as its hardware.

[0102] As shown in Table 1, the behavior recognition model in this embodiment, after testing on a typical power grid operation scenario dataset, achieved an accuracy rate of 91.61% for identifying violations in large sample data and 90.98% for identifying grounding wires, representing improvements of 2.68% and 4.52% respectively compared to the original baseline method. For identifying newly added violations in small sample data, the accuracy rate for identifying climbing was 88.24%, an improvement of 9.68% compared to the original baseline method. The baseline method used a ResNet-50 classification network, first training all network parameters with large sample data until convergence, then fine-tuning all network parameters with small sample data until convergence, and finally directly testing the results. The results indicate that the proposed method not only slightly improves the accuracy of identifying existing violations but also significantly improves the accuracy of identifying newly added small sample violations, achieving performance improvement of the behavior recognition model with less sample size, model design cost, and training cost.

[0103] Table 1. Test results of the model on the test set and comparison of different solutions.

[0104]

[0105] Further testing was conducted on the iterative update efficiency of the model. The training times for the baseline and the method described in this embodiment were recorded in Table 2. The total training time for the two phases of the baseline method was 1427.53 seconds. The total training time for the two phases of the method described in this embodiment was 762.75 seconds. The model in this embodiment improved the optimization iteration efficiency by 47.27% compared to the original baseline model.

[0106] Table 2 Comparison of training efficiency of the models on the test set.

[0107] Baseline method training time Training time for this method Optimize efficiency and improve 1427.53 seconds 762.75 seconds 47.27％

[0108] The beneficial effects of this invention are that, in response to the complex and ever-changing power grid scenarios, a two-stage fine-tuning training strategy is designed to slightly adjust network parameters for power grid scenarios and detect newly added violations. With a smaller sample size, model design cost, and training cost, the performance of the behavior recognition model is improved, and the efficiency of model iteration and update is further improved, thereby improving the accuracy and efficiency of identifying potential violations in typical power grid operation scenarios.

[0109] Example 4 is an embodiment of the present invention. This embodiment provides a small sample violation behavior identification system for power grid operations based on transfer learning and saliency model fine-tuning, including a data collection module, a first-stage fine-tuning module, a second-stage fine-tuning module, and a result output module.

[0110] The data collection module collects and annotates video data of on-site operation supervision in power grid scenarios, constructs a dataset for identifying violations, and divides it into small sample data and large sample data for target categories.

[0111] The first stage fine-tuning module selects the backbone network of the classification model, loads the pre-trained weights of the classification model on the public dataset, selects the network layers that need to be fine-tuned according to the first indicator, freezes the parameters of other network layers, and uses large sample data to perform a one-stage fine-tuning training on the selected network layers.

[0112] The second-stage fine-tuning module measures the difference between the fine-tuned network weights and the initial pre-trained weights based on a second metric, and adds a parameter mapping block after the high-discrepancy network layer. The output of the last parameter mapping block is fed into the few-shot decision layer used to distinguish few-shot categories, and the high-discrepancy network layer, parameter mapping block, and few-shot decision layer are trained in two stages for few-shot fine-tuning.

[0113] The result output module is based on a trained complete network model to identify violations in video data under power grid scenarios and obtain test results.

[0114] This embodiment also provides an electronic device applicable to the method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as proposed in the above embodiment.

[0115] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as proposed in the above embodiments.

[0116] The storage medium proposed in this embodiment and the method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0117] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying small-sample violations in power grid operations based on transfer learning and fine-tuning of saliency models, characterized by: include, Collect and label video data of on-site operation supervision in power grid scenarios, construct a dataset for identifying violations, and divide the target category into small sample data and large sample data. Select the backbone network of the classification model, load the pre-trained weights of the classification model on the public dataset, select the network layers that need to be fine-tuned according to the first indicator, freeze the parameters of other network layers, and use large sample data to perform a one-stage fine-tuning training on the selected network layers. Based on the second metric, the difference between the fine-tuned network weights and the initial pre-trained weights is measured, and parameter mapping blocks are added after the network layers with high differences. The output of the last parameter mapping block is fed into the few-sample decision layer used to distinguish the few-sample categories, and two-stage few-sample fine-tuning training is performed on the high-discretion network layer, parameter mapping block and few-sample decision layer. Based on a well-trained complete network model, violations are identified in video data of operations in power grid scenarios, and the identification results are obtained.

2. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 1, characterized in that: The construction of the violation behavior recognition dataset and the division of target categories from it into small sample data and large sample data include, Video data of on-site operation supervision in power grid operation scenarios are collected and labeled to construct an initial dataset for identifying violations. The initial dataset is then classified according to different violations, and the sample size of different violations is counted. A violation with a small sample size is selected to simulate newly added violations in the power grid scenario. All videos of the current category are used as small sample data, and videos of other violations are used as large sample data.

3. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 1, characterized in that: The steps include selecting the backbone network of the classification model, loading the pre-trained weights of the classification model on a public dataset, selecting network layers that need fine-tuning based on a first metric, freezing the parameters of other network layers, and performing a one-stage fine-tuning training on the selected network layers using large sample data. A classification backbone network with publicly available pre-trained weights is selected. A quantitative power grid sample data is fed into the classification network with pre-trained weights. The activation probability of each neuron during the forward propagation process is used as the first indicator to screen out underactivated and overactivated network layers. The parameters of other network layers are frozen. A large sample data is used to perform a one-stage fine-tuning training on the underactivated and overactivated network layers.

4. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 1, characterized in that: The step of measuring the difference between the fine-tuned network weights and the initial pre-trained weights according to the second metric, and adding a parameter mapping block after the high-discrepancy network layer, includes: The parameter mapping block and the few-sample decision layer constitute a discriminative structure for processing the target few-sample category. The output of the high-discrepancy network layer is used as input, and the parameter mapping block performs feature transformation. The transformed output is then passed to the few-sample decision layer to generate the discrimination result of the target category of the few samples. In the two-stage fine-tuning training, only the parameters of the parameter mapping block, the few-sample decision layer, and the high-discrepancy network layer are updated, while the parameters of the rest remain unchanged.

5. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 3, characterized in that: The selected quantitative power grid sample data is fed into a classification network with pre-trained weights. The activation probability of each neuron during the forward propagation is used as the first indicator to screen out underactivated and overactivated network layers. The quantitative power grid operation video sample data comes from a large sample dataset. The first indicator is the activation probability of neurons, which is calculated based on the proportion of activation functions that output a value greater than zero given input data. The expression is: in, Let C represent the activation probability of the l-th layer, N be the number of input data samples, and C be the activation probability of the l-th layer. l H l W l These represent the number of channels, height, and width of the l-th layer, respectively. This represents the activation function output at position (h, w) for the i-th sample, c-th channel, in layer l. It is an indicator function that returns 1 if the condition is met, and 0 otherwise.

6. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 1, characterized in that: The selected second metric measures the difference in network weights before and after the first-stage fine-tuning, and the network layers with high differences are selected. The second metric uses the KL divergence value. The specific calculation method is as follows: Gaussian kernel density estimation is used to fit the probability density function of the weights in each layer. Based on the probability density function of the weights, the KL divergence of the network weights before and after the first-stage fine-tuning is calculated. The expression is: in, This represents the probability density function of the weights of the l-th layer before the first-stage fine-tuning. This represents the probability density function of the weights of the l-th layer after one stage of fine-tuning. This represents the space of possible weight values. Network layers with KL divergence values ​​greater than a given threshold are considered high-discrepancy network layers.

7. The method for identifying small-sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in claim 4, characterized in that: The parameter mapping block and the few-sample decision layer constitute a discriminative structure for processing the target few-sample category. The output of the differential network layer is used as input, and feature transformation is performed by the parameter mapping block. The transformed output is then passed to the few-sample decision layer to generate the discrimination result for the small-sample target category. A parameter mapping block is added after the high-discrepancy network layer. Each parameter mapping block consists of a stack of convolutional layers, BN layers, and ReLU layers. The input features of the high-discrepancy network layer are used as the residual feature vector, and the output features of the high-discrepancy network layer are used as the unmapped feature vector. Both are fed into the parameter mapping block. The calculation formulas involved in the parameter mapping block are as follows: T i =M i (F i ,R i ), Among them, F i R is the unmapped feature vector in the i-th parameter mapping block. i T is the residual feature vector in the i-th parameter mapping block. i M is the mapped feature vector in the i-th parameter mapping block. i (·) represents a compound operation in the i-th parameter mapping block; The few-shot decision layer is a standalone classification layer composed of fully connected layers. It receives the output of the last parameter mapping block, predicts the probability of adding a new few-shot class, and concatenates this prediction with the results from the original classification network's prediction layer to obtain the final output value.

8. A small-sample violation identification system for power grid operations based on transfer learning and saliency model fine-tuning, employing the small-sample violation identification method for power grid operations based on transfer learning and saliency model fine-tuning as described in any one of claims 1 to 7, characterized in that, It includes: a data collection module, a first-stage fine-tuning module, a second-stage fine-tuning module, and a result output module; The data collection module collects and annotates video data of on-site operation supervision in power grid scenarios, constructs a dataset for identifying violations, and divides it into small sample data and large sample data for target categories. The first stage fine-tuning module selects the backbone network of the classification model, loads the pre-trained weights of the classification model on the public dataset, selects the network layers that need to be fine-tuned according to the first indicator, freezes the parameters of other network layers, and uses large sample data to perform a one-stage fine-tuning training on the selected network layers. The second-stage fine-tuning module measures the difference between the fine-tuned network weights and the initial pre-trained weights based on a second metric, and adds a parameter mapping block after the high-discrepancy network layer. The output of the last parameter mapping block is fed into the few-shot decision layer used to distinguish few-shot categories, and the high-discrepancy network layer, parameter mapping block, and few-shot decision layer are trained in two stages for few-shot fine-tuning. The result output module is based on a trained complete network model to identify violations in video data under power grid scenarios and obtain test results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for identifying small sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for identifying small sample violations in power grid operations based on transfer learning and saliency model fine-tuning as described in any one of claims 1 to 7.

Images