High-voltage distribution room personnel behavior identification method and system based on improved SlowFast

Abstract

The invention discloses a high-voltage distribution room personnel behavior identification method and system based on improved SlowFast, and belongs to the field of electric power safety production monitoring. Aiming at the problem that the precision of the existing personnel behavior recognition technology is insufficient in a complex electric power environment, a high-voltage distribution room personnel behavior data set is constructed, videos of three behaviors of an inspection tool, opening and closing and inspection recording are acquired, and an AVA format data set is formed through processing such as data enhancement, frame extraction and labeling. Meanwhile, an S lowFast network is improved, a scene enhancement module is introduced into a slow channel, and multiple pooling operations are fused to enhance spatial feature extraction and anti-noise capability; an Act ion module comprising STE, CE and ME is introduced into a fast channel, and key space-time, channel time and adjacent frame motion trail features are extracted. Experiments show that the mAP (at) 0.5 of the improved model reaches 78.06% which is higher than that of a comparison model, the recognition effect is good in a complex scene, and the recognition accuracy of the behaviors of the high-voltage distribution room personnel is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of power production safety monitoring technology, specifically a technology for identifying the behavior of personnel in high-voltage distribution rooms based on an improved SlowFast network. This technology lies at the intersection of computer vision, deep learning, and power engineering. This technology uses direct video detection and an improved deep learning model to accurately identify the behavior of personnel in high-voltage distribution rooms, addressing the problem of insufficient recognition accuracy in complex power production environments with existing technologies. Background Art

[0002] In the field of power production, accurate identification of personnel behavior in high-voltage distribution rooms is crucial, as it is directly related to the safe and stable operation of the power system. Existing video action recognition methods can be divided into indirect detection methods and direct detection methods based on the different detection objects.

[0003] Currently, indirect detection methods are widely used in power generation scenarios. These methods use data from sensors worn by personnel to indirectly assess human behavior. However, the accuracy and response time of these sensors often fall short of real-time monitoring requirements, leading to data delays and inaccuracies. Furthermore, the high deployment and maintenance costs of these sensors increase the overall economic burden. Consequently, these methods present significant limitations.

[0004] With the development of deep learning technology, researchers have proposed a variety of deep learning techniques for video action recognition, aiming to directly detect different behaviors. Domestic and international scholars have conducted extensive research on video action recognition using existing convolutional neural networks. Unlike traditional image-based action recognition, video action recognition requires the simultaneous extraction of temporal and spatial features and the fusion of these features into spatiotemporal features.

[0005] Initially, models based on 2D convolution were widely used. For example, two-stream networks (Two-Stream) leveraged dual inputs of optical flow and RGB frames to effectively extract spatial and temporal features, while temporal networks (TSN) achieved action recognition over long time spans by sampling and fusing video clips. However, optical flow methods suffer from high memory consumption and computational costs associated with extracting optical flow features, hindering the implementation of end-to-end recognition models.

[0006] Subsequently, 3D convolutional networks gradually became the mainstream method for video action recognition. Instead of relying on optical flow to learn complex temporal features, they introduced an additional temporal dimension to facilitate end-to-end feature extraction and classification. C3D, for example, extended the convolution kernel of VGGNet from 2D to 3D, creating a milestone for 3D convolutional networks. Building on this foundation, 3D convolutional network models such as P3D, I3D, and R(2+1)D were subsequently proposed. These models were improved within the C3D framework and can more effectively capture spatial and temporal features in videos. However, these models typically use a fixed temporal resolution for feature extraction, making it difficult to fully capture the dynamic information of motion at different speeds in videos.

[0007] In recent years, significant progress has been made in exploring spatiotemporal modeling capabilities. For example, Timesformer uses a self-attention mechanism to model global spatiotemporal features, effectively enhancing the network's ability to capture long-range dependencies. However, this comes with high computational complexity, especially when processing high-resolution video, where the computational effort of the attention mechanism increases with the square of the input sequence length. The SlowFast network utilizes different feature extraction paths to effectively model video space and time, improving the accuracy of action recognition. While ensuring the capture of fast-moving information, it reduces overall computational resource consumption and improves efficiency. However, its accuracy is often low when faced with the complex real-world power production environment.

[0008] Therefore, there are still many difficulties in identifying the behavior of personnel in high-voltage distribution rooms. These include the small distance between equipment, mutual obstruction between equipment, complex behavior operations, and non-standard personnel operations. These problems greatly increase the difficulty of identifying operator behavior in high-voltage distribution rooms. Therefore, a more effective method is urgently needed to improve the accuracy of personnel behavior recognition in high-voltage distribution rooms. Summary of the Invention

[0009] The technical problem to be solved by the present invention is to provide a method and system for solving problems such as small equipment spacing, mutual occlusion, complex behavioral operations and non-standard personnel operations in high-voltage distribution rooms in actual power production environments without using additional sensors, only through direct detection by cameras to capture video images. The purpose is to improve the accuracy of existing algorithms in identifying staff behavior.

[0010] In order to solve the above technical problems, this application provides the following technical solutions:

[0011] The present invention is based on an improved SlowFast method and system for identifying the behavior of personnel in a high-voltage distribution room. The specific scheme is as follows:

[0012] 1. Constructing a behavioral dataset of personnel in high-voltage distribution rooms:

[0013] Videos of three activities—tool inspection, switch opening and closing, and inspection logs—were collected and augmented through data augmentation methods such as flipping and cropping. Frames were extracted at 30 frames per second, and the first frame was selected as the keyframe. Human objects were detected using YOLOv5s, and the detected human frames were annotated. The DeepSort algorithm was used to establish correspondence between adjacent keyframes containing the same person. Finally, these correspondences were integrated into an annotation file to form an AVA-formatted dataset, providing the data foundation for subsequent model training.

[0014] 2. Improve SlowFast network

[0015] (1) Scene enhancement module:

[0016] The scene enhancement module is introduced into the slow channel. The input feature map is subjected to global average pooling, global maximum pooling and global median pooling to obtain three pooling results; each pooling result is passed through a shared multi-layer perceptron (containing two 1

[0017] ×1 convolutional layer and a ReLU activation function. The first convolutional layer reduces the feature dimension from C to C / 4, the second convolutional layer restores the feature dimension to C, and finally the Sigmoid activation function compresses the output value to the range of [0,1] to obtain three attention maps; the three attention maps are added element by element to obtain the channel attention map; the channel attention map is element-wise multiplied with the original input feature map to obtain the weighted feature map, thereby enhancing the extraction of spatial features, improving the network's robustness in noisy environments, and improving its ability to understand the relationship between channels.

[0018] (2)Action module:

[0019] The Action module is introduced into the fast channel, which includes the STE module, CE module, and ME module.

[0020] STE module: It performs global channel pooling on the input tensor, reconstructs it and multiplies it with a 3×3×3 convolution kernel to obtain a new spatiotemporal feature map. It then uses the Sigmoid activation function to obtain the attention weight and adds it to the input to obtain the output, thereby extracting the key spatiotemporal features of the video.

[0021] CE module: Models mutual dependencies on channels, describes the temporal information of channel features by inserting a 1×1 convolutional layer between two fully connected layers, and obtains the temporal feature weights of actions across different channels.

[0022] ME module: It uses 1×1 convolution to reduce the dimension, 3×3 convolution and element-wise subtraction to find the feature relationship between adjacent frames, concatenate them and increase the dimension, and obtain the attention weight through the Sigmoid function. The output is added to the input to extract the trajectory features of the motion changes between adjacent frames.

[0023] 3. Training the Personnel Behavior Recognition Model: Based on the improved SlowFast network, we used ResNet-50 as the backbone network and initialized the weights with Kinetics-400. We set the training epoch to 100, the batch size to 5, the initial learning rate to 0.00125, and the weight decay parameter to 0.005. We used the stochastic gradient descent algorithm for optimization, using the Focal Loss loss function, and trained the constructed AVA dataset to obtain the Personnel Behavior Recognition Model.

[0024] 4. Identify the behavior of personnel in the high-voltage distribution room:

[0025] The trained personnel behavior recognition model is used to identify the behavior of personnel in the high-voltage distribution room, determine the specific category of personnel behavior, and achieve effective monitoring of the behavior of personnel in the high-voltage distribution room.

[0026] Through the above-mentioned invention scheme, the constructed data set covers a variety of scenarios and personnel targets, which improves the generalization ability of the model; the improved network structure enhances the feature extraction capability and improves the recognition accuracy; the ablation experiment verifies the superiority of each module in the corresponding channel, and finally realizes the accurate identification of the personnel behavior in the high-voltage distribution room in a complex power production environment.

[0027] Compared with the existing technology, the present invention uses the improved SlowFast network to identify the behavior of personnel in high-voltage distribution rooms, achieving significant technical effects in many aspects:

[0028] Constructing a dataset: To address the lack of public datasets for behavior monitoring in high-voltage distribution rooms, we collected 203 videos of three behaviors: tool inspection, opening and closing, and patrol records. Through spatial flipping and random cropping, data augmentation, frame extraction, person detection, and labeling, we constructed an AVA-format dataset. This dataset provides strong support for model training and evaluation. The dataset covers different personnel targets and distribution room scenarios, which helps improve the model's generalization capabilities.

[0029] Improving recognition accuracy: A scene enhancement module is designed for the slow channel, integrating global average pooling, global maximum pooling, and median pooling to construct a robust channel attention mechanism. This effectively extracts spatial features, enhances robustness to noisy environments, and enhances understanding of inter-channel relationships. An Action module, comprising STE, CE, and ME, is introduced for the fast channel to extract key spatiotemporal and channel temporal feature weights and adjacent frame motion trajectory features, respectively, improving the network structure. Comparative experiments show that the improved model achieves a mAP@0.5 of 78.06%, surpassing competing models such as C3D, I3D, TSN, and Timesformer. Visualization results demonstrate excellent detection performance in both indoor and outdoor scenarios, as well as in complex multi-person scenarios.

[0030] Identifying module advantages: Ablation experiments validated the superiority of the slow-path Scene Enhancement Module and the fast-path Action Module working together. The slow-path Scene Enhancement Module enhances the representation of static scene elements, providing more context for action recognition and improving accuracy. The fast-path Action Module leverages the temporal continuity of its high-frequency sampling to fully capture subtle differences in motion patterns, avoiding the loss of behavioral features caused by sparse temporal information and significantly improving model performance.

[0031] Enhanced noise resistance: The scene enhancement module effectively addresses noise issues through a unique pooling and attention mechanism design, enabling the model to more stably identify human behavior in complex environments. Compared with other possible alternatives, such as multi-scale feature fusion, it is more robust to noise.

[0032] Compared with the traditional channel attention mechanism (such as SE module), the scene enhancement module innovatively integrates median pooling, significantly improving the robustness to impulse noise ( Figure 7 Heat map comparison); the three modules in the Action module work in coordination to enhance the ability to extract continuity features of actions between two adjacent frames.

[0033] The following further illustrates the method and system for identifying personnel behavior in a high-voltage distribution room based on the improved SlowFast method of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The improved SlowFast network architecture for the personnel behavior recognition in the high-voltage distribution room based on the improved SlowFast of the present invention includes a slow pathway and a fast pathway.

[0035] Figure 2 This is the workflow of the STE module in the high-voltage distribution room personnel behavior recognition method and the system based on the improved SlowFast of the present invention;

[0036] Figure 3 The present invention is based on the improved SlowFast high-voltage distribution room personnel behavior recognition method and the structure of the CE module in the system;

[0037] Figure 4 The present invention is based on the improved SlowFast high-voltage distribution room personnel behavior recognition method and the structure of the ME module in the system;

[0038] Figure 5 This is a flowchart of producing an AVA format data set in the method and system for identifying personnel behavior in a high-voltage distribution room based on the improved SlowFast of the present invention;

[0039] Figure 6This is the visualization test result of the improved SlowFast model in the high-voltage distribution room personnel behavior recognition, including three behavior diagrams: opening and closing, checking tools, and inspection records (the blue detection box in the figure represents the test result);

[0040] Figure 7 The present invention is used to analyze the thermal map of the performance of the slow channel;

[0041] Figure 8 This is a heat map showing the performance of the present invention in observing the fast channel. DETAILED DESCRIPTION

[0042] like Figure 1 As shown in the figure, the present invention is based on the improved SlowFast method and system architecture for identifying personnel behavior in high-voltage distribution rooms. It is composed of a slow pathway and a fast pathway. The slow pathway innovatively incorporates a scene enhancement module, which embeds the median pooling operation into the channel attention mechanism. Combined with global average pooling and global maximum pooling, it forms an enhanced channel attention mechanism and is placed within the residual connection of each 3D Bottleneck block, after the 1×1×1 convolution. This design aims to deeply mine the spatial features of the video, effectively improve the network's recognition robustness in noisy environments, enhance the understanding of inter-channel relationships, and thus optimize the overall performance of the model. The Action module is introduced on the fast pathway. It contains three complementary attention modules: STE (spatiotemporal excitation), CE (channel excitation), and ME (motion excitation). It is placed within the residual connection of each 3D Bottleneck block and before the 3×1×1 convolution. This layout enables the convolution within the Bottleneck block to fully utilize spatiotemporal, channel, and motion features, significantly enhancing the network's ability to learn the temporal characteristics of behavior and motion, and improving the network architecture. The information processed by the two channels is ultimately used to generate accurate prediction results, providing strong support for the identification of personnel behavior in high-voltage distribution rooms.

[0043] Specifically, the present invention enhances the SlowFast network in two key aspects. First, a scene enhancement module is designed and implemented in the slow channel to extract the spatial features of the video, which enhances the network's robustness in recognition in noisy environments. In the slow channel with a large number of channels, the ability to understand the relationship between channels is enhanced, and the overall performance of the model is improved; at the same time, the scene understanding ability of the slow channel is enhanced by adding a median pooling operation. Secondly, on the fast channel, in order to enhance the network's ability to learn the temporal characteristics of behavioral motion, the Action module is introduced. The main task of this module is to extract motion features and enhance the network's ability to learn fine-grained features, thereby improving the architecture of the SlowFast network. Through the above two methods, the accuracy of the SlowFast network in video behavior understanding tasks has been significantly improved. The introduction of the scene enhancement module and the Action module enables the network to more effectively understand environmental information and capture motion features, thereby improving the understanding of complex behaviors in videos and the accuracy of classification.

[0044] Among them, in order to solve the noise problem and enhance the robustness of the channel attention mechanism, the present invention introduces a median pooling operation in the channel attention mechanism, combines global average pooling and global maximum pooling, and forms a more robust channel attention mechanism. It is added to the residual connection of each 3D Bottleneck block and after the 1×1×1 convolution to achieve the effect of further extracting channel features. The scene enhancement module generates a channel attention map by aggregating the global statistical information of the input feature map, thereby weighting the channel of the input feature. The specific process of the scene enhancement module is as follows:

[0045] (a) Pooling operation: Use global average pooling (AvgPool), global maximum pooling (MaxPool) and global median pooling (MedianPool) on the input feature map to obtain three different pooling results. The size of each pooling result is

[0046]

[0047] Where N is the batch size, T is the number of frames, and C is the number of channels.

[0048] (b) Shared MLP processing: Each pooling result is passed through a shared multi-layer perceptron (MLP), which consists of two 1×1 convolutional layers and a ReLU activation function. The first convolutional layer reduces the feature dimension from C to C / 4, where the dimensionality reduction ratio is set to 4, and the second convolutional layer restores the feature dimension to C. Finally, a sigmoid activation function is used to compress the output value into the range [0, 1], resulting in three attention maps.

[0049] (c) Fusion pooling result: The attention maps of the three pooling results are added element by element to obtain the final channel attention map.

[0050] (d) Weighted input feature map: The channel attention map is element-wise multiplied with the original input feature map to obtain the weighted feature map.

[0051] The formula is as follows:

[0052] F c =σ(MLP(AvgPool(F)))+σ(MLP(MaxPool(F)))+σ(MLP(MedianPool(F))) (2)

[0053] F′=F c ⊙F (3)

[0054] The Action module consists of three complementary attention modules: STE (Spatiotemporal Excitation), CE (Channel Excitation), and ME (Motion Excitation). The STE module extracts key spatiotemporal features of the video, the CE module obtains the temporal feature weights of the action across different channels, and the ME module extracts the trajectory features of motion changes between adjacent frames. The Action module is introduced within the residual connection of each 3D Bottleneck block and before the 3×1×1 convolution. This allows the convolution within the Bottleneck block to fully utilize the spatiotemporal features, channel information, and motion features.

[0055] See Figure 2 , is the workflow diagram of the STE submodule in the Action module of the present invention, which presents the specific process of extracting key spatiotemporal features of the video. Conventional spatiotemporal features are usually extracted using 3D convolution, but if 3D convolution is used directly, the computational load of the model will increase significantly. Therefore, the input tensor of the present invention is first subjected to channel global pooling to obtain a channel global spatiotemporal feature map, and then the map is reconstructed to adapt it to the three-dimensional convolution operation. Next, the reconstructed result is multiplied by the 3×3×3 convolution kernel to generate a new spatiotemporal feature map to achieve spatiotemporal excitation. After that, it is reconstructed again to match the dimension of the Sigmoid activation function, and the attention weight is obtained by the Sigmoid activation function. Finally, the attention weight is element-wise multiplied and added with the initial input, and the processed result is output to accurately extract the key spatiotemporal features of the video and provide key spatiotemporal information for subsequent behavior recognition.

[0056] The CE module can be found in Figure 3, is a structural diagram of the CE submodule in the Action module of the present invention. It is an attention module on the channel and has similarities with the SE module, but there are key differences. The CE module adaptively calibrates the channel feature response by modeling the mutual dependence between channels. Unlike the SE module, in order to strengthen the temporal dependency of the channel, the CE module inserts a 1×1 convolution layer in the temporal information between two fully connected (FC) layers to describe the temporal information of the channel features, thereby obtaining the temporal feature weights of the action across different channels and improving the network's ability to capture the temporal features of the action.

[0057] The ME module is mainly used to capture motion information related to the motion between two adjacent frames. Figure 4 Figure 2 shows the working diagram of the ME submodule in the Action module of the present invention. This module first reduces the dimensionality of the input current frame X(t) and the previous frame X(t+1) through 1×1 convolution. It then uses a 3×3 convolution kernel and element-wise subtraction to find the feature relationship between the two frames. After concatenating these relationships, it performs 1×1 convolution to increase the dimensionality. The sigmoid function is then applied to obtain the attention weights. Finally, the attention weights are element-wise multiplied and added with the initial input, and the processed results are output, effectively extracting the motion change trajectory features between adjacent frames.

[0058] In response to the current problem of lack of publicly available data sets for behavior monitoring in high-voltage distribution rooms, the present invention constructs a data set through public online materials to identify the behavior of staff in high-voltage distribution rooms. This specific embodiment collects 203 videos of three types of behaviors, namely, inspection tools, opening and closing switches, and inspection records, and performs spatial flipping and random cropping to form an original data set of 609 videos. After the original data set is established, the data is enhanced, framed, detected, tracked, and labeled, and finally an AVA format data set is formed as the input of the model. Figure 5 The figure below is a flow chart of the process for constructing a high-voltage distribution room personnel behavior recognition dataset, demonstrating the complete process from raw video to an AVA-formatted dataset. First, the input video is frame-sampled at 30 frames per second, with the first frame per second selected as the keyframe. Next, human targets are detected using YOLOv5s for each frame. The detected human frames are then annotated for their behavior. Finally, the DeepSort algorithm is used to process adjacent keyframes in each video, establishing a correspondence between the same person across different frames. Finally, these correspondences are integrated into the annotation file to generate an AVA-formatted dataset, providing high-quality data support for model training and testing.

[0059] Among them, (a) the video is framed at 30 frames per second, and the first frame per second is selected as the key frame; (b) YOLOv5s is used to detect human targets in each frame; (c) each person box generated in b is annotated with behavior; (d) the DeepSort algorithm is used for adjacent key frames of each video to generate correspondences between the same person in different frames; (e) the person correspondences generated in d are integrated into the annotation file generated in c, and finally form an AVA format dataset.

[0060] This example uses YOLOv5s from the YOLOv5 family to detect human targets. YOLO v5S is a lightweight model designed for low-resource devices, offering fast inference speed and relatively high detection accuracy. DeepSort combines the classic sorting algorithm with deep learning feature extraction. Building on the traditional Kalman filter and Hungarian algorithm, it leverages features extracted by a deep learning model, making it more robust when dealing with target appearance changes and occlusions. Therefore, DeepSort was chosen as the target tracking algorithm for this article.

[0061] After raw data processing, the label composition of the key frames of the dataset is shown in Table 1. This dataset uses different human targets, high-voltage distribution rooms in different locations, and scenes where one or more targets appear in the same video, including the recognition of three common actions.

[0062] Table 1 Label composition

[0063]

[0064] Example:

[0065] 1. Experimental Setup

[0066] The deep learning framework used in the experiment is Pytorch 1.8.0, the programming language is Python 3.8, the operating system is Ubuntu 20.04, and the GPU is Tesla V100-PCIE-32GB.

[0067] During model training, the epoch number was set to 100, the batch size was set to 5, the initial learning rate was set to 0.00125, and the weight decay parameter was set to 0.005. Stochastic Gradient Descent (SGD) was used for model optimization. The backbone network used in the experiment was ResNet-50, with Kinetics-400 used to initialize the original weights. The scene enhancement and action module weights were trained using the FocalLoss loss function.

[0068] 2. Dataset Segmentation

[0069] In this paper's experiments, the training and test sets were randomly divided into a 4:1 ratio based on the number of videos. The training set contained 487 videos (2719 key frames), and the test set contained 122 videos (643 key frames). All of the test set videos were from high-voltage distribution room scenes that were not used in the training, and included complex situations such as equipment occlusion and multi-person collaboration. The training set had 2719 key frames (the first frame of each second with a corresponding action label is considered a key frame), while the test set had 643 key frames.

[0070] 3. Evaluation Metrics

[0071] This paper uses mean average precision (mAP) as the experimental indicator. This value requires calculating the average precision (AP) value for each class in the dataset, which is also related to the precision V pre and recall V rec These two indicators are related:

[0072]

[0073] Where TP, FP, and FN are the number of true positive, false positive, and false negative samples, respectively.

[0074] Since V pre and V rec It is a pair of contradictory values, so it is more intuitive to use AP to evaluate the results. The calculation method is to count different V pre and V rec The area under the value curve V AP :

[0075]

[0076] Where: P and R are precision and recall respectively.

[0077] Real-world applications usually involve model performance evaluation under multiple categories, so mAP is used more frequently. It is the average of APs of multiple categories and can be used. mAP Expressed as:

[0078]

[0079] Where: O mAP is the value of the jth category; N is the total number of categories.

[0080] 4. Comparative experiment

[0081] In order to better evaluate the performance of the improved SlowFast in the recognition of personnel behavior in high-voltage distribution rooms, this paper selects C3D, I3D, TSN and Timesformer as comparison models. The pre-training weights are all trained based on the Kinetics-400 dataset, and then secondary training is performed on the self-made dataset in this paper to make the experimental results more convincing. The experimental results are shown in Table 2, where O mAP The overlap area threshold used in the calculation is set to 0.5. As shown in Table 2, the average recognition accuracy of the improved method proposed in this paper is higher than that of other models. The visualization results of the improved SlowFast are as follows: Figure 6 As shown in the figure, the blue detection box represents the detection result.

[0082] Table 2 Comparative experimental results

[0083]

[0084] like Figure 6 The visualization results show that the improved SlowFast has good detection results for all three behaviors and is highly robust in indoor and outdoor scenarios and in multi-person scenarios. It can solve the problems of many workers and complex working environments in actual applications. Figure 6 This visualization shows the improved SlowFast model used in this paper for identifying human behavior in a high-voltage distribution room. The detection boxes in the figure represent the detection results. The figure clearly demonstrates that the improved model demonstrates good detection results for three behaviors: opening and closing switches, checking tools, and logging inspections. The model demonstrates strong robustness in both indoor and outdoor scenarios, as well as in multi-person environments, effectively handling large numbers of personnel and complex working environments in real-world applications, demonstrating its effectiveness and reliability in real-world scenarios.

[0085] 5. Ablation Experiment

[0086] Ablation experiments added the scene enhancement module and the action module to different channels to verify the role of each module and the appropriate channel type. The ablation test results are shown in Table 3. Channel type represents the channel type in which the module is used, and - represents the channel type in which the module is not used.

[0087] Table 3 Ablation experiment results

[0088]

[0089] As shown in Table 3, by implementing the scene enhancement module in the slow channel and integrating the action module in the fast channel, significant performance improvements are achieved. The advantages of this design strategy are mainly reflected in the following two aspects: First, in response to the slow channel's core task of scene parsing, the scene enhancement module effectively strengthens the network's representation of static scene elements by constructing a multi-level feature fusion mechanism and attention weight distribution, which is highly compatible with its channel positioning. Second, because the fast channel undertakes the core function of temporal dynamic modeling, its high-frequency sampling temporal continuity, compared to the slow channel's low-sampling rate input, provides the action module with sufficient inter-frame correlation information. This enables the 3D convolution kernel and spatiotemporal feature extractor to more fully capture subtle differences in motion patterns, thereby avoiding the loss of behavioral features caused by sparse temporal information. Therefore, the solution in this paper is to add the scene enhancement module to the slow channel and the action module to the fast channel. Experimental results show that this solution achieves better performance than other solutions.

[0090] Figure 7 This is a heat map used by the present invention to analyze the performance of the slow channel. By showing the model's attention to environmental information on the slow channel, the functional performance of the slow channel is evaluated. As can be seen from the figure, the model itself has a strong ability to pay attention to environmental information on the slow channel. After the introduction of the scene enhancement module, this attention ability is further improved. During the fusion process of the fast and slow channels, the slow channel, with its enhanced ability to pay attention to environmental information, provides more valuable environmental information for action recognition, thereby improving the accuracy of action recognition.

[0091] Figure 8 This heat map, used by the present invention to observe the performance of the fast channel, shows the adjacent key frames (X-1), X, and (X+1). Observation revealed that the (X-1) frame is less affected by the X frame, resulting in a weaker fast channel recognition of motion information. To address this issue, the present invention introduces an Action module into the fast channel to enhance the mutual influence between adjacent frames, thereby improving the fast channel's recognition of motion information and optimizing the motion recognition performance of the entire network.

[0092] Some of the terms involved in this invention are explained as follows:

[0093] Optical flow: refers to the motion pattern of image pixels between consecutive frames due to the movement of objects or camera movement. Simply put, optical flow describes how each point (pixel) in the image moves in time.

[0094] YOLOv5: is a real-time object detection model based on deep learning. The goal of the YOLO series is to achieve fast and accurate object detection, suitable for various computer vision tasks.

[0095] DeepSORT (Deep Learning-based SORT): is an algorithm for target tracking that combines deep learning technology with the traditional SORT (Simple Online and Realtime Tracking) algorithm to improve the performance and accuracy of multi-target tracking.

[0096] Bottleneck Block: A bottleneck block is a common network structure in deep learning, particularly in convolutional neural networks (CNNs), most notably in ResNet (residual networks). The bottleneck block is designed to reduce the number of model parameters and computational complexity while maintaining or improving the network's expressive power.

[0097] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast, characterized in that: The following steps are involved: Constructing a dataset of personnel behavior in high-voltage distribution rooms: We collected videos of three behaviors: checking tools, opening and closing switches, and patrol inspections. Data enhancement was performed through flipping and cropping, and the videos were framed. The first frame per second was selected as the keyframe. The YOLOv5s model was used to detect human targets, and the character frames were annotated. The DeepSORT algorithm was used to generate correspondences between adjacent keyframes with the same person. These relationships were integrated into the annotation file to form an AVA-formatted dataset. We also improved the SlowFast network by introducing a scene enhancement module in the slow channel and an action module in the fast channel. The improved SlowFast network is used to train the AVA format data set to obtain a personnel behavior recognition model; and the personnel behavior recognition model is used to identify the personnel behavior in the high-voltage distribution room.

2. The method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast according to claim 1 is characterized in that: The scene enhancement module performs the following operations: 1) Perform global average pooling, global maximum pooling, and global median pooling on the input feature map to obtain three pooling results; 2) Each pooling result is processed through a shared multi-layer perceptron to obtain three attention maps; 3) Add the three attention maps element by element to obtain the channel attention map; 4) Multiply the channel attention map by the original input feature map element-wise to obtain a weighted feature map.

3. The method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast according to claim 2 is characterized in that: The multilayer perceptron contains two 1×1 convolutional layers and a ReLU activation function. The first convolutional layer reduces the feature dimension from C to C / 4, the second convolutional layer restores the feature dimension to C, and finally the Sigmoid activation function is used to compress the output value into the range of [0, 1].

4. The method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast according to claim 1 is characterized in that: The Action module includes the STE module, the CE module, and the ME module: STE module: performs global channel pooling on the input tensor, reconstructs it and multiplies it with a 3×3×3 convolution kernel to obtain a new spatiotemporal feature map, then uses a Sigmoid activation function to obtain the attention weight, and adds it to the input to obtain the output; CE module: Models interdependencies on channels and describes the temporal information of channel features through a cascade structure of fully connected layer → 1×1 convolutional layer → fully connected layer; ME module: Dimensionality reduction is performed through 1×1 convolution, 3×3 convolution and element-wise subtraction are used to find the feature relationship between adjacent frames, and the dimensions are increased after splicing. The attention weight is obtained by the Sigmoid function and added to the input to obtain the output.

5. The method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast according to claim 1 is characterized in that: The method for constructing a high-voltage distribution room personnel behavior dataset includes: We collected 203 videos of three behaviors, flipped and cropped them to get 609 videos; Extract frames from the video at 30 frames per second and select the first frame per second as the key frame; Use YOLOv5s to detect human targets and annotate the human frames; The DeepSort algorithm is used to generate the correspondence between the same characters in adjacent key frames and integrate them into the annotation file to form an AVA format dataset.

6. The method for identifying personnel behavior in a high-voltage distribution room based on improved SlowFast according to claim 1 is characterized in that: The method for training the AVA format data set using the improved SlowFast network includes: ResNet-50 is used as the backbone network, and Kinetics-400 is used to initialize the weights; The training epoch is set to 100, the batch size is 5, the initial learning rate is 0.00125, and the weight decay parameter is 0.005; The stochastic gradient descent algorithm is used for optimization, and the loss function is Focal Loss.

7. A high-voltage distribution room personnel behavior recognition system based on improved SlowFast, characterized in that: include: Dataset construction module: used to construct the personnel behavior dataset of the high-voltage distribution room; Network Improvement Module: This module is used to introduce the scene enhancement module into the slow channel of the SlowFast network and the Action module into the fast channel. Model training module: used to train the AVA format dataset using the improved SlowFast network to obtain a human behavior recognition model; Behavior recognition module: used to identify the behavior of personnel in the high-voltage distribution room using the personnel behavior recognition model.

8. The high-voltage distribution room personnel behavior recognition system based on improved SlowFast according to claim 7 is characterized in that: The scene enhancement module is used to: Perform global average pooling, global maximum pooling, and global median pooling on the input feature map to obtain three pooling results; Each pooling result is processed through a shared multi-layer perceptron to obtain three attention maps; Add the three attention maps element by element to get the channel attention map; The channel attention map is element-wise multiplied with the original input feature map to obtain a weighted feature map.

9. The high-voltage distribution room personnel behavior recognition system based on improved SlowFast according to claim 7 is characterized in that: The Action module includes the STE module, the CE module, and the ME module: STE module: It is used to perform global channel pooling on the input tensor, reconstruct it and multiply it with a 3×3×3 convolution kernel to obtain a new spatiotemporal feature map, then use the Sigmoid activation function to obtain the attention weight, and add it to the input to obtain the output; CE module: used to model interdependencies on channels and describe the temporal information of channel features by inserting a 1×1 convolutional layer between two fully connected layers; ME module: It is used to reduce the dimension through 1×1 convolution, find the feature relationship between adjacent frames through 3×3 convolution and element-wise subtraction, increase the dimension after splicing, obtain the attention weight through Sigmoid function, and add it to the input to get the output.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images