Switching operation action identification method and system based on improved CSN model

Abstract

The invention relates to a switching operation action recognition method and system based on an improved CSN model. In order to solve the problems of low monitoring efficiency of traditional switching operation and application of a deep learning model, an improved CSN model is used for action recognition. The method comprises the following steps: firstly, acquiring and preprocessing switching operation video data of a 10kV power distribution room, and then inputting the switching operation video data into an improved CSN model; in the model, an adaptive time shift module (ATSM) groups feature map channels and performs learnable time shift operation to capture time sequence dynamic characteristics; a gating bottleneck attention module (GBAM) generates gating signals according to the importance of input features, adjusts the attention degree of different space regions, and captures key action features. Experiments show that compared with a mainstream model, the improved model has higher precision in switching operation action recognition, specific action characteristics can be captured in a small sample scene, efficient balance of performance and computing resources is realized, and the improved model can be used for intelligent and accurate recognition and monitoring of switching operation actions.

Description

Technical Field

[0001] The present invention relates to the field of electric power technology, in particular to an intelligent monitoring technology for electric power systems, and in particular to a switching operation action recognition technology based on an improved CSN model. Background Art

[0002] As power systems continue to move toward intelligentization, switching operations in 10kV distribution rooms have become a critical component of safe production management. The standardization and safety of switching operations are directly related to the stability of power system operation and the personal safety of operators. However, traditional methods of monitoring switching operations have numerous drawbacks. Relying on manual monitoring and empirical judgment, they are not only inefficient but also susceptible to subjective factors. For example, prolonged manual monitoring can easily lead to fatigue, resulting in a loss of concentration and the potential omission of key operational details. Furthermore, the experience levels of different operators vary, making it difficult to standardize judgment criteria. These factors pose significant challenges to the accurate monitoring and standardized management of switching operations, making them difficult to meet the stringent requirements of intelligent power systems for efficient and accurate monitoring.

[0003] In recent years, research on the application of deep learning technology in power scene monitoring and recognition has continued to emerge. Liu Peizhen et al. proposed using wrist sensors to collect acceleration and magnetic field strength data, and using a long short-term memory neural network (LSTM) to capture the temporal characteristics of power operation and maintenance operations, thereby improving motion recognition capabilities to a certain extent. Zhang Xiaoqing et al. combined a hybrid expert system (MoE) with an LSTM to improve the recognition performance of switching operation actions using three-dimensional acceleration data collected by inertial sensors. Wu Tian et al. designed a method for identifying abnormal behavior in 10kV live video based on an improved spatiotemporal graph convolutional network (ST-GCN). This method uses video data to extract human skeleton information to identify abnormal behavior of people working on live lines.

[0004] As an efficient 3D convolutional network, the CSN model offers unique advantages in video classification tasks. Traditional 3D convolutional networks simultaneously consider channel and spatiotemporal interactions when extracting spatiotemporal features from videos. While this approach can capture complete feature information, the high computational complexity limits the model's efficiency and scalability in practical applications. The CSN model decouples channel and spatiotemporal interactions, using 1×1×1 convolutions to handle channel interactions and 3×3×3 depthwise convolutions to handle spatiotemporal interactions. It also proposes two new bottleneck block structures: interaction-reduced CSN (ir-CSN) and interaction-preserving CSN (ip-CSN), effectively balancing computational efficiency and accuracy. The ip-CSN excels in deep networks, significantly improving accuracy while maintaining high computational efficiency. Furthermore, when processing small sample datasets, its combination of 1×1×1 convolutions and 3×3×3 depthwise convolutions offers advantages in regularization, helping to mitigate overfitting.

[0005] Despite progress, practical applications of switching operations in 10kV distribution rooms still face numerous challenges. For one thing, the limited sample size of switching operations makes it difficult to meet the large-scale data requirements of deep learning models. Furthermore, human operators vary in their movements, rhythms, and habits, and the inherent complexity of switching operations limits the performance and generalization of recognition models in practical applications. Designing an action recognition method that can efficiently model spatiotemporal features, adapt to small sample sizes, and effectively account for individual differences remains a pressing technical challenge. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a switching operation action recognition technology based on an improved CSN model with higher recognition accuracy and better balance between computing resources and performance.

[0007] In order to solve the above technical problems, this application provides the following technical solutions:

[0008] The present invention provides a switching operation action recognition method based on an improved CSN model, which is characterized by comprising the following steps:

[0009] 1) Obtaining 10kV distribution room switching operation video data, and preprocessing the video data to obtain preprocessed video data;

[0010] 2) inputting the preprocessed video data into an improved CSN model, and performing feature extraction and action recognition on the video data using the improved CSN model, wherein the improved CSN model includes an adaptive time shift module and a gated bottleneck attention module;

[0011] 3) The adaptive time shift module is used to group the input feature map into channels and apply a learnable time shift operation to each group of channels to capture the timing dynamic characteristics of the switching operation;

[0012] 4) The gated bottleneck attention module is used to generate a gating signal according to the importance of the input features to dynamically adjust the degree of attention to different spatial regions, thereby capturing the key action features in the switching operation.

[0013] The present invention also provides a switching operation action recognition system based on an improved CSN model, which includes:

[0014] A data preprocessing module is used to obtain the switching operation video data of the 10kV distribution room and preprocess the video data to obtain preprocessed video data;

[0015] An action recognition module, configured to input the preprocessed video data into an improved CSN model, and perform feature extraction and action recognition on the video data using the improved CSN model, wherein the improved CSN model includes an adaptive time shift module and a gated bottleneck attention module;

[0016] The adaptive time shift module is used to group the input feature map into channels and apply a learnable time shift operation to each group of channels to capture the timing dynamic characteristics of the switching operation;

[0017] The gated bottleneck attention module is used to generate a gating signal according to the importance of the input features to dynamically adjust the degree of attention to different spatial regions, thereby capturing the key action features in the switching operation.

[0018] An electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the switching operation action recognition method based on the improved CSN model are implemented. A computer-readable storage medium is also provided, wherein the computer-readable storage medium stores the computer program, and when the processor executes the computer program, the steps of the switching operation action recognition method based on the improved CSN model are implemented.

[0019] Preferably, the pretreatment comprises the following steps:

[0020] uniformly extracting 32 frames of images from the video data;

[0021] Resizing the image to a shorter dimension of 256 pixels while maintaining the original aspect ratio;

[0022] Randomly crop the image to obtain an image with a size of 224×224 pixels;

[0023] Performing data enhancement processing on the image, including horizontal flipping or mirroring processing, and color dithering processing.

[0024] Preferably, the adaptive time shift module dynamically adjusts the time dependency of the channel groupings through a learnable time shift parameter. The specific working process of the adaptive time shift module may include:

[0025] The channels of the input feature map are divided into channels that participate in time shift and channels that do not participate in time shift, where the number of channels participating in time shift is controlled by the channel grouping factor;

[0026] Divide the channels involved in time shifting into G groups, each group contains C_group channels;

[0027] Applying a learnable time shift value to each group of channels, which is updated via backpropagation;

[0028] Concatenate the shifted channels with the unshifted channels to obtain the output feature map.

[0029] Preferably, the channel grouping factor fdf and the number of groups G are learnable parameters, and the initial value ranges are {4, 8, 16, 32} and {1, 2, 4, 8}.

[0030] Preferably, the gated bottleneck attention module dynamically fuses channel attention and spatial attention weights through a gating signal. Specifically, the specific working process of the gated bottleneck attention module may include:

[0031] Apply global average pooling to the input feature map in the spatial dimension to generate a pooled feature vector;

[0032] Inputting the pooled feature vector into two fully connected layers for mapping to generate a gating signal;

[0033] The input feature map is processed through the channel attention mechanism and the spatial attention mechanism to generate an enhanced feature map;

[0034] According to the gating signal, the enhanced feature map is weightedly fused with the original input feature map to obtain an output feature map.

[0035] Preferably, the specific working process of the channel attention mechanism includes:

[0036] Apply global average pooling to the input feature map to generate a pooling vector;

[0037] Pass the pooled vector through two 3D convolutional layers in sequence to generate channel attention weights;

[0038] The specific working process of the spatial attention mechanism includes:

[0039] The input feature map is reduced in dimension by 1×1×1 convolution, and normalized and nonlinearly mapped by combining BN and ReLU activation functions;

[0040] Use a 3×3×3 convolutional layer and a dilation rate to expand the receptive field and generate spatial context information;

[0041] The spatial attention weights are generated by 1×1×1 convolution and sigmoid activation function.

[0042] Preferably, in the bottleneck structure of the improved CSN model:

[0043] a. The adaptive time shift module is located before the convolution operation and is used to dynamically adjust the time dimension of the input feature map;

[0044] b. The gated bottleneck attention module is integrated at the convolution output through a residual connection, and the enhanced feature map is weighted and fused with the original input feature map according to the gating signal.

[0045] The adaptive time shift module and the gated bottleneck attention module are integrated into the bottleneck structure of the improved CSN model. The specific integration method is as follows:

[0046] The adaptive time shift module is located before the convolution operation and is used to dynamically adjust the time dimension of the input feature map;

[0047] The feature map processed by the adaptive time shift module is passed through the three-layer convolution operation of the backbone to obtain the bottleneck convolution output;

[0048] Passing the bottleneck convolution output to the gating mechanism of the gated bottleneck attention module to generate a gating signal, and obtaining an enhanced feature map through the BAM module of the gated bottleneck attention module;

[0049] According to the gating signal, the enhanced feature map is weightedly fused with the original input feature map to obtain an output feature map;

[0050] Through the residual connection, the input feature map and the processed output feature map are added to complete the update and transmission of information.

[0051] Compared with the existing technology, the present invention achieves efficient recognition of switching operation actions by introducing the adaptive time shift module (ATSM) and the gated bottleneck attention module (GBAM) into the CSN model, showing significant advantages in accuracy, resource utilization, and adaptability, and has at least the following beneficial effects:

[0052] 1. High Recognition Accuracy: On a 10kV distribution room switching operation dataset, the proposed model achieved a Top-1 accuracy of 83.47% and a Top-5 accuracy of 99.63%, outperforming mainstream models such as TimeSformer, Swin Transformer, TSN, SlowOnly, C3D, ir-CSN152, and ip-CSN152. The ATSM module dynamically adjusts time shift parameters to adapt to operator rhythm variations and effectively capture temporal dynamics. The GBAM module generates gating signals based on the importance of input features, precisely focusing on key action features. These two modules synergistically improve recognition accuracy.

[0053] 2. Excellent balance between computing resources and performance: Compared with other models, the proposed model achieves significant performance improvements with a moderate increase in computational load (increased by 17.9G Flops) and parameter count (increased by 18.8M Params). For example, TSN, while having low computational load and parameter count, has a recognition accuracy of only 58.44%. TimeSformer and Swin Transformer have high computational load and parameter count, but their accuracy does not match. By optimizing module design, the proposed model achieves an efficient balance between performance and computing resources on small-scale datasets.

[0054] 3. Strong adaptability in small sample scenarios: The proposed model excels in addressing the limited sample size of switching operations. While the Transformer model can face feature extraction challenges in data-constrained scenarios, the proposed model, based on the ip-CSN152, enhances its ability to capture specific action features through the ATSM and GBAM modules. This demonstrates significant advantages in small sample scenarios and effectively addresses data constraints in real-world applications.

[0055] 4. Robustness: Multiple operations during the data preprocessing phase enhance the robustness of the model. Uniformly extracting 32 frames from the video, scaling while maintaining aspect ratio, random cropping, horizontal flipping or mirroring, and color dithering are performed. This increases data diversity, enabling the model to adapt to varying shooting angles, lighting, and environmental conditions, improving its adaptability to changes in action position and reducing the risk of overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 (a), (b), and (c) are comparative diagrams of bottleneck block structures of a switching operation action recognition technology based on an improved CSN model of the present invention.

[0057] The structures of the ResNet bottleneck block, ir-CSN, and ip-CSN are displayed side by side, visually showing the differences in the composition of the convolutional layers of the three, and used to compare the design features of different structures in handling channel interaction and spatiotemporal interaction;

[0058] Figure 2 This is a structural diagram of an adaptive time shift module of a switching operation action recognition technology based on an improved CSN model of the present invention;

[0059] Figure 3 This is a schematic diagram of the principle of a gated bottleneck attention module for a switching operation action recognition technology based on an improved CSN model of the present invention;

[0060] Figure 4: This is a schematic diagram of the improved bottleneck structure integrated model architecture of the switching operation action recognition technology based on the improved CSN model of the present invention. It presents the overall architecture after integrating the adaptive time shift module (ATSM) and the gated bottleneck attention module (GBAM) into the ip-CSN bottleneck structure, and shows the data processing flow in the integrated structure.

[0061] Figure 5 :The present invention's self-made switching operation data set display example (photo schematic), presents the screen of some key behaviors in the self-made switching operation data set of 10kV distribution room, and intuitively displays samples of different action categories in the data set;

[0062] Figure 6 : Visualization chart of statistical results of significance grouping of different models of the present invention;

[0063] Figure 7 : Comparison diagram of thermal visualization of GradCAM model of the present invention. DETAILED DESCRIPTION

[0064] The present invention provides a switching operation action recognition method and system based on an improved CSN model, aiming to solve the problem of switching operation monitoring in a 10kV distribution room, improve recognition accuracy, and balance computing resources and performance.

[0065] The improved CSN model for switching operation action recognition of the present invention mainly includes: an adaptive time shift module, a gated bottleneck attention module, and the integration of ATSM and GBAM in the bottleneck structure.

[0066] 1. Adaptive Time Shift Module

[0067] Temporal dynamic features play a crucial role in video analysis tasks such as action recognition and video classification. In the specific scenario of switching operations, operators exhibit significant individual differences in the rhythm and distribution of their action time series due to varying habits and proficiency, further increasing the complexity of temporal modeling and the difficulty of recognition.

[0068] To this end, an Adaptive Temporal Shift Module (ATSM) is proposed. It is conceptually inspired by the Temporal Shift Module (TSM). The ATSM module dynamically optimizes the time shift parameters and adaptively adjusts the temporal dependencies of different channel groups to cope with the temporal feature changes and uncertainties brought about by individual differences, thereby more effectively capturing the timing dynamic characteristics of switching operations.

[0069] 1. Channel grouping

[0070] Given an input feature map F(X)∈R B×C×T×H×W , where B is the batch size, C is the number of channels, T is the number of frames, H and W are the height and width of the feature map respectively. The ATSM module first divides the feature channel C into two parts: one part of the channel participates in time shift, and the other part of the channel remains unchanged to retain the original spatiotemporal features. Channel C participating in time shift shifted By the channel grouping factor f d Control, this factor is used to control the proportion of channels involved in the shift, and the remaining channels C unshifted Remain unchanged. Figure 2 As shown in Figure 1, it details the internal composition and data processing flow of the Adaptive Time Shift Module (ATSM), including channel grouping, learnable time shift operation, and the formation of the final output feature map. shifted In the ATSM module, the module is further divided into G groups, each group contains C group channels, and the number of channel groups is G. The above process can be expressed as:

[0071]

[0072] C unshifted =CC shifted (2)

[0073]

[0074] This channel grouping strategy effectively reduces the mutual interference between large-scale channels and provides a basis for the subsequent application of dynamic time shift operations to each channel group, thereby supporting the effective modeling of different timing dynamic characteristics.

[0075] 2. Learnable Time Shift

[0076] The core feature of the ATSM module is that the time shift value applied to each group of channels is a learnable parameter, which allows the model to dynamically adjust the time shift operation during training to better capture the temporal dynamics of the input data. The shift value for each group of channels is represented by a parameter vector Δt i =[Δt1,Δt2,…,Δt G ] represents, where Δt i is the time shift value of the i-th group, initialized to a small random value. During training, these shift values ​​are updated via backpropagation, allowing the model to adaptively determine the optimal time displacement for each group. For each i-th group, the learned time shift value Δt i It is applied using a cyclic shift operation along the time dimension, which shifts the feature channels forward or backward in time by Δt i Unit. The above process can be expressed as:

[0077]

[0078] Y i =Roll(X i ,Δt i ,dim=T) (5)

[0079] Among them, η is the learning rate, L is the loss function, and the shift value Δt is adjusted according to the gradient information of i , thereby optimizing the time shift parameters. X i represents the i-th group of channels, Roll(·) represents the cyclic shift operation along the time dimension, and the learnable shift value Δt i Stored in the parameter vector Δt∈R G , G is the number of groups, and dim=T refers to the time dimension in which the shift is applied.

[0080] 3. Module output

[0081] After applying the learned shift to each group of channels, the output of the shifted group is concatenated with the unshifted channels to form the final output feature map Y∈R B×C×T×H×W The output feature map Y maintains the same shape as the input feature map F(X), but the temporal dynamic features of different channels are adjusted to adapt to different temporal dependencies. This operation is expressed as:

[0082] Yout=Concat([Y1,Y2,…,YG],Xunshifted) (6)

[0083] In summary, the ATSM module uses channel grouping and learnable time shifting to enable the model to flexibly adjust according to the characteristics of the channel when processing time dependencies, avoiding the limitations of the fixed shift strategy and further improving the model's timing modeling capability for identifying switching operations.

[0084] 2. Gated Bottleneck Attention Module

[0085] The various actions in switching operations exhibit significant heterogeneity in execution and spatial distribution, manifested in dynamic variations in movement amplitude, execution trajectory, and key regions across different steps. This characteristic requires the model to dynamically adjust its focus on different spatial regions, accurately capturing key action features and effectively adapting to the diversity of action categories.

[0086] To this end, the inventors proposed a Gated Bottleneck Attention Module (GBAM). The GBAM module combines a gating mechanism with a Bottleneck Attention Module (BAM) to dynamically adjust the level of attention paid to different spatial regions. It generates a gating signal based on the importance of input features and selectively activates the BAM module, allowing the model to focus on key features, suppress interference from irrelevant areas, and improve its ability to capture key action features during switching operations.

[0087] 1. Gating mechanism

[0088] The gating mechanism dynamically regulates the activation of the BAM module by generating a gating signal. Inspired by SENet (Squeeze-and-Excitation Network), this design simplifies the generation process of its channel weight vector and directly generates a scalar value as a gating signal to control the activation degree of the BAM module, thereby achieving adaptive activation under different input feature conditions.

[0089] like Figure 3 As shown, global average pooling is applied to the input feature map F(X) in the spatial dimension to extract global context features, compress the spatial information into a single value for each channel, and generate a pooled feature vector y∈R B×C . The pooled feature vector y is then input into two fully connected layers for mapping. The first fully connected layer introduces a reduction ratio r to reduce the dimension of the feature and implements nonlinear mapping through the ReLU activation function; the second fully connected layer restores the dimension and outputs a scalar value to describe the significance of each feature map. Finally, the output is normalized by the sigmoid function to generate a gating signal gate∈[0,1], which is broadcast to the spatial dimension of the feature map to control the activation degree of the BAM module. The above process can be expressed as:

[0090] gate=σ(ReLU(y·W1+b1)·W2+b2) (7)

[0091] Where W1 and W2 are learnable weights, b1 and b2 are bias terms, and σ(·) is the sigmoid function. Through this gating signal, the model can dynamically adjust the activation level of the BAM based on the saliency of the input feature map.

[0092] 2. BAM module

[0093] BAM consists of two parts: channel attention and spatial attention, which are used to enhance the expressiveness of feature maps in the channel dimension and spatial dimension, improve the model's attention to significant feature areas, and suppress interference from irrelevant areas.

[0094] The channel attention mechanism aims to enhance the ability to distinguish in the channel dimension by dynamically adjusting the weight of each channel. First, global average pooling is applied to the input feature map F(X) to reduce the spatial dimension and produce a pooled vector y∈R B×C , as the global descriptor of the feature map. The pooled vector passes through two 3D convolution layers in sequence. The first convolution layer compresses the number of channels and introduces a reduction ratio r to reduce computational overhead; the second convolution layer maps the reduced features back to the original number of channels and generates a normalized channel attention weight A through the sigmoid activation function. c ∈R B×C×1×1×1 The channel attention mechanism can be expressed as:

[0095]

[0096] A c =σ(Conv2(ReLU(Conv1(y)))) (9)

[0097] Among them, Conv1 and Conv2 are two 3D convolutional layers, and σ(·) is the sigmoid function. The above mechanism enables the model to better focus on discriminative channel features by adjusting the importance of each channel.

[0098] The spatial attention mechanism aims to capture the local and global correlation of feature maps in the spatial dimension, thereby dynamically modeling the importance of different spatial positions. First, the feature map F(X) is used as input, and the input features are reduced in dimension through 1×1×1 convolution, and normalized and nonlinearly mapped in combination with BN and ReLU activation functions. Subsequently, a 3×3×3 convolution layer is used in combination with the expansion rate d to expand the receptive field, thereby capturing richer spatial context information. Finally, the normalized spatial attention weight A is generated through 1×1×1 convolution and sigmoid activation function. s ∈R B×1×T×H×W , highlighting the most important spatial locations in the feature map. The spatial attention mechanism can be expressed as:

[0099] A s =σ(Conv3(ReLU(BN(Conv2(ReLU(F(X))))))) (10)

[0100] Conv1, Conv2, and Conv3 represent 3D convolutional layers, BN represents batch normalization, and σ(·) represents the sigmoid function. This mechanism effectively regulates the importance of specific spatial locations in the feature map, thereby enhancing the expressive power of the spatial dimension.

[0101] BAM will channel attention A c and spatial attention A sThe output of is combined with the original input feature map F(X) and feature enhancement is performed through element-wise multiplication. The BAM attention output can be expressed as:

[0102] BAM(F(X))=F(X)·A c +F(X)·A s (11)

[0103] 3. Module output

[0104] The final output O of the GBAM module is the weighted sum of the original input feature map F(X) and the feature map BAM(F(X)) after BAM processing. The gate signal gate determines the weight distribution of the two parts:

[0105] O=gate·BAM(F(X))+(1-gate)·F(X) (12)

[0106] When the gate is close to 1, GBAM has a strong influence on the output, and the model focuses on the features that are enhanced. When the gate is close to 0, the model primarily retains the original feature map, avoiding unnecessary computation. Through this dynamic control mechanism, GBAM effectively balances attention enhancement and computational efficiency, improving the model's perception of key features while suppressing interference from irrelevant areas, enhancing the model's ability to capture key action features during switching operations.

[0107] 3. Integration of ATSM and GBAM in the Bottleneck Structure

[0108] To improve the spatiotemporal modeling capabilities of the baseline ip-CSN model for video action recognition, this paper integrates the proposed ATSM and GBAM modules into the bottleneck structure of ip-CSN. While retaining the high efficiency of ip-CSN's channel-separated 3D convolutions, this design enhances the model's ability to model the complex motion features of switching operations through an adaptive temporal shift module and a gated bottleneck attention module.

[0109] In the bottleneck structure, the ATSM module is located before the convolution operation and is used to dynamically adjust the time dimension of the input feature map F(X). Figure 4As shown in the figure, after the adaptive time shift operation, the generated feature map F(X′) passes through the three-layer convolution operation of the backbone, where the second layer of convolution contains an additional 1×1×1 convolution operation to obtain the bottleneck convolution output y. This output is passed to the gating mechanism to generate the gating signal gate, and at the same time, the enhanced feature map BAM(y) is obtained through the BAM module. The gating signal gate controls the activation degree of BAM and is weightedly fused with BAM(y) to form the final output Out. Finally, through the residual connection, the input feature map F(X) is added to the processed output Out to obtain Output, completing the information update and transmission. The above process can be expressed as:

[0110] y=Conv3(Conv2b(Conv2a(Conv1(F(X′))))) (13)

[0111] Out=gate·BAM(y)+(1-gate)·y (14)

[0112] Output=Out+F(X) (15)

[0113] This design not only improves the model's representation capabilities, but also embodies the collaborative optimization effect between the ATSM and GBAM modules, further enhancing the model's adaptability to modeling spatiotemporal features, and can effectively meet the motion recognition requirements of switching operations in complex scenarios.

[0114] The technical solution of the present invention is described below with reference to specific embodiments in conjunction with the accompanying drawings:

[0115] like Figure 1 (a), (b), and (c) are schematic diagrams comparing the bottleneck block structures of a switching operation action recognition technology based on the improved CSN model of the present invention. The structures of the ResNet bottleneck block, ir-CSN, and ip-CSN are displayed side by side, intuitively presenting the differences in the convolutional layer composition of the three, which are used to compare the design features of different structures in processing channel interaction and spatiotemporal interaction.

[0116] The present invention provides a switching operation action recognition method based on an improved CSN model, and its application is taken as an example in a 10kV distribution room:

[0117] 1. Data preprocessing:

[0118] In a 10kV distribution room, workers captured video data of a switching operation. According to the invented method, 32 frames of images are first uniformly extracted from the video to ensure the temporal representativeness of the data for subsequent processing. The images are then scaled to 256 pixels on the short side while maintaining the original aspect ratio to accommodate the model input size requirements. Random cropping is then performed to obtain 224×224 pixel images, increasing data diversity. Finally, data augmentation processing, including horizontal flipping or mirroring and color dithering, is performed to further expand the data volume and improve the model's generalization capabilities.

[0119] 2. Improve CSN model construction and processing flow

[0120] 1) Adaptive Time Shift Module (ATSM):

[0121] refer to Figure 2 Taking a batch of input feature maps as an example, according to the present invention, the channel grouping factor fd is set to 8, and the number of channel groups G is set to 2. Channels that participate in time shifting and those that do not participate in time shifting are first divided into two groups, with the number of channels in each group determined by the total number of channels and the grouping. A learnable time shift value is applied to each channel group. This value is updated through backpropagation during model training to capture the timing dynamics of switching operations. Finally, the shifted and non-shifted channels are concatenated to obtain the output feature map.

[0122] 2) Gated Bottleneck Attention Module (GBAM):

[0123] Combine Figure 3 , Figure 3 This is a schematic diagram of the principle of the gated bottleneck attention module of the switching operation action recognition technology based on the improved CSN model of the present invention, which shows the working principle of the gated bottleneck attention module (GBAM), covering the gating mechanism, the construction of the BAM module (including channel attention and spatial attention), and the calculation method of the final output.

[0124] The input feature map is processed to generate a gating signal and an enhanced feature map. Global average pooling is first performed in the spatial dimension to generate a pooled feature vector, which is then input into two fully connected layers for mapping to obtain the gating signal. Regarding the channel attention mechanism, the input feature map is globally average pooled to obtain a pooled vector, which is then passed through two 3D convolutional layers to generate channel attention weights. Regarding the spatial attention mechanism, the input feature map undergoes dimensionality reduction using a 1×1×1 convolution, followed by normalization and nonlinear mapping using BN and ReLU activation functions. A 3×3×3 convolutional layer is then used to expand the receptive field and generate spatial context information. Finally, a 1×1×1 convolution and sigmoid activation function are used to generate spatial attention weights, resulting in an enhanced feature map. Based on the gating signal, the enhanced feature map is weightedly fused with the original input feature map to obtain the output feature map.

[0125] 3) Bottleneck structure integration:

[0126] Figure 4 :This is a schematic diagram of the bottleneck structure integrated model architecture improvement of the switching operation action recognition technology based on the improved CSN model of the present invention, presenting the overall architecture after integrating the adaptive time shift module (ATSM) and the gated bottleneck attention module (GBAM) into the ip-CSN bottleneck structure, and showing the data processing flow in the integrated structure. Figure 4 As shown in the figure, in the ip-CSN bottleneck structure of the improved CSN model, the ATSM is placed before the convolution operation and dynamically adjusts the temporal dimension of the input feature map. The processed feature map undergoes three layers of backbone convolution operations to produce a bottleneck convolution output. This output is passed to the gating mechanism of the GBAM to generate a gating signal, which is then passed through the BAM module to produce an enhanced feature map. Based on the gating signal, the enhanced feature map is weightedly fused with the original input feature map. Then, through a residual connection, the input feature map and the processed output feature map are added together to complete information update and transmission, improving the model's ability to capture complex motion features.

[0127] 3. Action recognition and result application:

[0128] The preprocessed video data is fed into the improved CSN model, which extracts features and recognizes actions using the aforementioned modules. In real-world scenarios, if the model identifies an irregular operation, the system immediately issues an alarm, reminding personnel to pay attention to operational safety, ensuring stable power system operation and enabling intelligent and precise identification and monitoring of switching operations.

[0129] 4. Experimental Results and Analysis

[0130] 1. Experimental Dataset

[0131] The 10kV distribution room switching operation dataset consists of actual on-site operation videos and online public material videos, with a total of 252 video samples, covering the key behaviors in the distribution room switching operation. The dataset is specifically divided into 7 behavior categories, including: checking safety tools, manual opening / closing, grounding knife switch, shaking circuit breaker, opening / closing cabinet door, disconnecting / closing secondary plug, and hanging / removing signboard. In order to ensure the scientific nature of the data and the standardization of the annotation, this dataset adopts the same standardized data organization and annotation method as UCF101 (University of Central Florida) to ensure that the action categories are clearly defined and the annotations are fine-grained and unified. Among them, the dataset is divided into training set and test set in a ratio of about 7:3, and ensure that each category maintains the same proportion in the two parts to avoid the impact of category distribution deviation on model training. Part of the dataset is shown as follows Figure 5 As shown, Figure 5(a), (b), (c), and (d) respectively use photos to show the self-made dataset of switching operations for practical application of the present invention, presenting images of some key behaviors in the self-made dataset of switching operations in a 10kV distribution room, and intuitively displaying samples of different action categories in the dataset.

[0132] 2. Data preprocessing

[0133] During the data preprocessing phase, a fixed frame sampling strategy was used. A uniform 32-frame sampling strategy was used to extract 1 frame every 2 frames from each video. This generated a frame-based dataset. The data distribution is shown in Table 1. The extracted frame sequence was decoded into frame-by-frame images and uniformly scaled to a shorter dimension of 256 pixels while maintaining the original aspect ratio to ensure image size consistency and avoid distortion caused by image stretching. Subsequently, the frame images were randomly cropped to generate samples with different spatial perspectives, enhancing the model's robustness to variations in motion position. The cropped frame images were resized to 224×224 pixels to ensure consistent input data size, meeting the model's input requirements.

[0134] Table 1 Dataset distribution

[0135]

[0136] Table 1 shows that the dataset used in this article suffers from a class imbalance, primarily due to process variations across different switching operation tickets. For example, "shaking the circuit breaker," as a core step in switching operations, is included in most switching operation ticket processes and, therefore, accounts for a relatively high proportion in the dataset. On the other hand, operations such as "opening / closing the cabinet door" and "hanging / removing the sign" appear only in specific switching operation ticket processes, resulting in relatively small data sets. This class imbalance, to a certain extent, reflects the characteristics of actual switching operation processes.

[0137] To further enhance sample diversity and strengthen the model's generalization capabilities, data augmentation methods are implemented. First, frames are horizontally flipped or mirrored with a 50% probability. During each augmentation, either horizontal flip or mirroring is randomly selected. This operation simulates scene changes from different shooting angles and is suitable for recognizing symmetrical or reversible actions. Next, color dithering is used to randomly adjust the image's brightness, contrast, saturation, and hue to simulate diverse lighting and environmental conditions, thereby enhancing the model's adaptability in complex scenes.

[0138] Analysis of the experimental results of the above switching data set:

[0139] To validate the recognition performance of our model, we compared it with mainstream models: TimeSformer, Swin Transformer, TSN, SlowOnly, C3D, ir-CSN152, and ip-CSN152. The experimental results are shown in Tables 2 and 3. Table 2 focuses on the recognition performance of each model on a self-developed switching operation dataset, while Table 3 provides a supplementary analysis based on computational complexity and parameter size. The following analysis combines these two sets of data to analyze the performance differences among the models and their causes.

[0140] Table 2 shows the experimental results of different models on the switching operation dataset. As can be seen from Table 2, the Transformer-based TimeSformer and Swin Transformer achieved Top-1 accuracy of 61.73% and 65.38%, respectively. While their accuracy surpasses that of traditional convolutional models, they still lag behind the proposed model. This may be due to the Transformer model's reliance on large-scale data. With limited samples, the complex spatiotemporal attention mechanism is difficult to fully train, resulting in insufficient feature extraction.

[0141] Table 2 Model performance comparison

[0142]

[0143] The performance of traditional convolutional models further illustrates the need for spatiotemporal modeling capabilities in switching operations. TSN, a 2D convolution-based action recognition model, achieved a Top-1 accuracy of 58.44% and a Top-5 accuracy of 96.37%. While superior to SlowOnly and C3D, it only models features from static frames and cannot fully capture the complex spatiotemporal dynamics of switching operations. In contrast, the 3D convolution-based SlowOnly and C3D models offer some improvements in spatiotemporal modeling, but their relatively shallow network structures still lack the ability to express complex actions.

[0144] The CSN series of models performed exceptionally well, with the ip-CSN152 achieving a Top-1 accuracy of 78.08% and a Top-5 accuracy of 98.81%. This demonstrates that ip-CSN, through its channel interaction-preserving design, effectively improves its ability to capture detailed features of action videos, outperforming ir-CSN. However, compared to the proposed model, the recognition accuracy of the CSN series of models still lags somewhat. This suggests that the improvements based on the ip-CSN152 in this paper effectively enhance the model's ability to capture the specific features of switching operations, helping to improve the model's adaptability to small-scale datasets.

[0145] The complexity comparison in Table 3 shows significant differences in computing resources and parameter scale across different models. TSN has the lowest computational load and parameter count, but its Top-1 accuracy is only 58.44%, making it difficult to meet the complex spatiotemporal dynamic modeling requirements of switching operations. In contrast, while the computational load and complexity of SlowOnly and C3D have increased, their Top-1 accuracy remains low. TimeSformer and Swin Transformer have larger computational loads and parameter counts (196.0G and 121.0M; 604.0G and 196.0M, respectively), with Top-1 accuracy rates of 61.73% and 65.38%, respectively. Although superior to TSN, their accuracy rates do not match the higher computational resource consumption, indicating that the increased model complexity in small sample scenarios has not significantly translated into performance improvements. The CSN series (ir-CSN152 and ip-CSN152) models effectively enhance spatiotemporal modeling capabilities through channel interaction mechanisms, achieving Top-1 accuracy of 77.26% and 78.08%, respectively, at moderate complexity. In contrast, the improved model based on ip-CSN152 in this paper, after adding two modules, increases computational complexity by 17.9 GB and parameter count by 18.8 MB, but improves Top-1 accuracy by 5.39%. Overall, by rationally increasing computational complexity and parameter count, this model improves its ability to more effectively capture the specific characteristics of switching operations in small sample scenarios, achieving an efficient balance between performance and computing resources.

[0146] Table 3 Comparison of model complexity

[0147]

[0148] Statistical significance analysis:

[0149] To verify whether the performance differences between different models in the switching operation recognition task are statistically significant, this study selected Top-1 accuracy as an analysis metric. This metric measures whether the model's first prediction of the input sample is correct, thus having stronger discriminative power and engineering applicability.

[0150] In terms of statistical methods, we first used the Levene test for homogeneity of variance to analyze the top-1 accuracy of each model. Table 4 shows the significance test results for different models. The test result was p = 0.106, which is greater than the significance level of 0.05 and meets the requirement for homogeneity of variance. A subsequent one-way ANOVA test showed that the performance differences between the models were statistically significant at the p level < 0.001.

[0151] Table 4 Significance test

[0152]

[0153] Based on the results of the single-factor ANOVA test showing significant overall differences, the Waller-Duncan post hoc test was further used to perform multiple comparisons of the models to identify significant differences between specific models. Figure 6 As shown in the figure, the letters at the top of the columns indicate statistical grouping. Identical letters indicate no significant difference between groups, while different letters indicate significant difference between groups. As can be seen from the figure, our model achieves the highest recognition accuracy, significantly outperforming the other compared models. There is no significant difference between ip-CSN152 and ir-CSN152, but both exhibit significant differences from our model. The remaining models also exhibit varying degrees of significant differences, but overall performance falls short of our model.

[0154] Figure 6 This is a visualization chart of the statistical results of the significance grouping of different models of the present invention. It shows the statistical grouping of the Top-1 accuracy of different models in the switching operation action recognition task in the form of bar charts and letter annotations, which is used to compare the significance of the performance differences between each model.

[0155] Figure 7 This is a thermal visualization comparison diagram of the GradCAM model of the present invention, showing the heat map of the attention distribution of the baseline model and the model after adding the BAM, GBAM, and ATSM modules when processing the input video, which is used to intuitively compare the differences in the model's attention to key areas under different module configurations.

[0156] In summary, the proposed model demonstrates significant performance advantages in comparison with multiple models, further verifying its effectiveness in the switching operation action recognition task.

[0157] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A switching operation action recognition method based on an improved CSN model, characterized in that: The following steps are involved: 1) Obtaining 10kV distribution room switching operation video data, and preprocessing the video data to obtain preprocessed video data; 2) inputting the preprocessed video data into an improved CSN model, and performing feature extraction and action recognition on the video data using the improved CSN model, wherein the improved CSN model is implemented by integrating an adaptive time shift module and a gated bottleneck attention module in a bottleneck structure; 3) The adaptive time shift module is used to group the input feature map into channels and apply a learnable time shift operation to each group of channels to capture the timing dynamic characteristics of the switching operation; 4) The gated bottleneck attention module is used to generate a gating signal according to the importance of the input features to dynamically adjust the degree of attention to different spatial regions, thereby capturing the key action features in the switching operation.

2. The switching operation action recognition method based on the improved CSN model according to claim 1 is characterized in that: The pretreatment comprises the following steps: uniformly extracting 32 frames of images from the video data; Resizing the image to a shorter dimension of 256 pixels while maintaining the original aspect ratio; Randomly crop the image to obtain an image with a size of 224×224 pixels; Performing data enhancement processing on the image, including horizontal flipping or mirroring processing, and color dithering processing.

3. The switching operation action recognition method based on the improved CSN model according to claim 1 is characterized in that: The adaptive time shifting module dynamically adjusts the time dependency of channel groupings through learnable time shifting parameters.

4. The switching operation action recognition method based on the improved CSN model according to claim 3 is characterized in that: The channel grouping factor fd and the number of groups G are learnable parameters, and the initial value ranges are {4, 8, 16, 32} and {1, 2, 4, 8}.

5. The switching operation action recognition method based on the improved CSN model according to claim 1 is characterized in that: The specific working process of the gated bottleneck attention module includes: Apply global average pooling to the input feature map in the spatial dimension to generate a pooled feature vector; Inputting the pooled feature vector into two fully connected layers for mapping to generate a gating signal; The input feature map is processed through the channel attention mechanism and the spatial attention mechanism to generate an enhanced feature map; According to the gating signal, the enhanced feature map is weightedly fused with the original input feature map to obtain an output feature map.

6. The switching operation action recognition method based on the improved CSN model according to claim 5 is characterized in that: The specific working process of the channel attention mechanism includes: Apply global average pooling to the input feature map to generate a pooling vector; Pass the pooled vector through two 3D convolutional layers in sequence to generate channel attention weights; The specific working process of the spatial attention mechanism includes: The input feature map is reduced in dimension by 1×1×1 convolution, and normalized and nonlinearly mapped by combining BN and ReLU activation functions; Use a 3×3×3 convolutional layer and a dilation rate to expand the receptive field and generate spatial context information; The spatial attention weights are generated by 1×1×1 convolution and sigmoid activation function.

7. The switching operation action recognition method based on the improved CSN model according to claim 1 is characterized in that: In the bottleneck structure of the improved CSN model: The adaptive time shift module is located before the convolution operation and is used to dynamically adjust the time dimension of the input feature map; The gated bottleneck attention module is integrated at the convolution output through a residual connection, and the enhanced feature map is weighted and fused with the original input feature map according to the gating signal.

8. A switching operation action recognition system based on an improved CSN model, characterized in that: include: A data preprocessing module is used to obtain the switching operation video data of the 10kV distribution room and preprocess the video data to obtain preprocessed video data; An action recognition module is used to input the preprocessed video data into an improved CSN model, and perform feature extraction and action recognition on the video data through the improved CSN model, wherein the structure of the improved CSN model is as defined in any one of claims 1-7.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images