GTB-ResNet identification method for abnormal behaviors of personnel in petroleum station
By introducing Ghost convolution, Triplet attention, and Bi-GRU into the 3D ResNet network, feature extraction and temporal features are optimized, solving the problems of high computational cost and poor robustness in abnormal behavior recognition in oilfields, and achieving efficient and accurate abnormal behavior recognition.
Patent Information
- Application Number
- CN202410997519.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-19
AI Technical Summary
Existing video behavior recognition methods based on Two-Stream CNN and 3D CNN suffer from problems such as high computational cost, poor robustness, large number of parameters, difficulty in preserving spatiotemporal context correlation, and difficulty in identifying similar behaviors in oilfields, resulting in low accuracy of abnormal behavior recognition and easy misjudgment.
We employ a 3D ResNet network, combined with a lightweight residual module of Ghost convolution, a Triplet attention mechanism, and a Bi-GRU network, to optimize feature extraction and temporal feature extraction. The Ghost Bottleneck module reduces network parameters and computational cost, the Triplet attention mechanism enhances the focus on salient features, and Bi-GRU strengthens the extraction of long temporal features.
It achieves efficient and accurate abnormal behavior identification in the oilfield environment, reduces the number of model parameters and computational costs, and improves the accuracy and robustness of identification of salient features, making it suitable for deployment on edge devices.
Smart Images

Figure CN118982862A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of abnormal behavior recognition, and in particular to a GTB-ResNet recognition method for abnormal behavior of personnel at a petroleum station. Background Art
[0002] Since the working area of the oil station is large and the distance between each monitoring area is far, information transmission is difficult and the real-time detection is poor. During operation, workers may have abnormal behaviors such as smoking, making phone calls, and falling in the oil field, and the size of human targets in the video is small, which increases the difficulty of target identification.
[0003] The video behavior recognition method based on deep learning overcomes the heavy computational cost and weak robustness caused by the traditional manual feature extraction. It extracts trainable features through the model and automatically learns behavior representation from the video to complete classification. At present, behavior recognition methods can be divided into two categories, Two-Stream CNN and 3D CNN. One network of Two-Stream CNN uses RGB images to extract spatial dimension features, and the other network uses optical flow maps to make up for the lack of temporal information, and then obtains the prediction results through appropriate fusion methods. Although optical flow can represent the instantaneous dynamic information of the human body, due to the complex outdoor environment of the oil station, the optical flow is easily disturbed by changes in light intensity. The calculation of optical flow is time-consuming and cannot achieve real-time performance, which limits the Two-Stream CNN to a certain extent. 3D CNN is an extension of 2D CNN in video recognition. However, the 3D CNN based on 2D CNN used in oil stations is not suitable for real-time recognition. The CNN recognition method still has the following limitations: 1) Simply expanding 2D to 3D will lead to an increase in model parameters and computational costs, which in turn limits the network depth and is not conducive to network optimization; 2) Affected by irrelevant factors such as noisy background and lighting in the oil station, behavior recognition cannot provide sufficient information, and the experimental accuracy is not high and the robustness is poor; 3) Only the depth of the CNN layer is considered, while the parameter size and time calculation complexity are ignored; 4) In 3D convolution, useful time information may be lost in the later stage of the deep network. How to retain the spatiotemporal context correlation in long time series; when identifying similar behaviors of workers, it is difficult to determine whether it is abnormal; in actual environments, abnormal behaviors such as smoking and making phone calls are often associated with hands and faces, but when it comes to normal behaviors such as touching the mouth and ears, it is easy to cause misjudgment. Summary of the invention
[0004] In view of the shortcomings of the existing methods, the present invention selects 3D ResNet network for optimization; designs a lightweight residual module based on Ghost convolution; introduces the triplet attention mechanism in the feature extraction stage to increase the focus on significant features; and introduces the Bi-GRU enhancement model to extract key long-term features.
[0005] The technical solution adopted by the present invention is: the GTB-ResNet recognition method of abnormal behavior of oil station personnel includes the following steps:
[0006] Step 1: Collect abnormal behavior images of construction personnel at oil stations;
[0007] As a preferred embodiment of the present invention, abnormal behaviors include: using a mobile phone, smoking, and accidentally falling.
[0008] Step 2: Construct a GTB-ResNet network. The backbone network of the GTB-ResNet network includes stage 1 to stage 4. Stage 1 is connected by 3 Ghost Bottleneck modules with a stride of 1 and 1 Triplet attention module; stage 2 includes 4 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage 3 includes 6 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage 4 includes 3 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module. A lightweight residual module is used to extract spatiotemporal information from continuous frame images to reduce network parameters and computational complexity; Triplet attention is used to enhance the perception of channel and spatial position information, and to improve the positioning and attention to significant features of behavior recognition.
[0009] As a preferred embodiment of the present invention, the GhostBottleneck module with stride 1 includes: a first Ghost_Conv3D, BN+Relu, a second Ghost_Conv3D and BN, and is residually connected with the first Ghost_conv3D.
[0010] As a preferred embodiment of the present invention, the GhostBottleneck module with stride 2 includes: a third Ghost_Conv3D, BN+Relu, DWConv, BN, a fourth Ghost_Conv3D and BN, and performs a residual connection with the third Ghost_Conv3D.
[0011] As a preferred embodiment of the present invention, the output of the backbone network of the GTB-ResNet network is connected to the BI-GRU network.
[0012] Step 3: Input the abnormal behavior image into the GTB-ResNet network for abnormal behavior recognition.
[0013] As a preferred embodiment of the present invention, the GTB-ResNet network is evaluated using precision, recall, recognition accuracy, frame rate, floating-point operations and parameters.
[0014] As a preferred implementation of the present invention, a confusion matrix is used to calculate the confusion rate between abnormal behaviors and normal actions.
[0015] As a preferred embodiment of the present invention, a GTB-ResNet recognition system for abnormal behavior of oil station personnel includes: a memory for storing instructions executable by a processor; a processor for executing instructions to implement a GTB-ResNet recognition method for abnormal behavior of oil station personnel.
[0016] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, the GTB-ResNet recognition method of abnormal behavior of oil station personnel is implemented.
[0017] Beneficial effects of the present invention:
[0018] 1. To address the large number of parameters and high computational cost of 3D ResNet, a lightweight residual convolution module (Ghost Bottleneck) in the feature extraction network is designed. Ghost convolution is used instead of standard convolution to reduce model parameters while ensuring that the deep network can extract multi-scale feature information;
[0019] 2. In order to solve the problem that the monitoring range is wide, the target object is small, and the model is difficult to focus on significant features, the Triplet attention mechanism is introduced to obtain information interaction between space and channels;
[0020] 3. As short-term features can easily misjudge similar actions, a bidirectional gated recurrent network Bi-GRU is added to the feature extraction backbone network to ensure that the model extracts key long-term features and improve the accuracy of feature extraction by the model;
[0021] 4. Compared with the existing 3D ResNet network, which has a large number of parameters and is difficult to deploy in edge devices, a lightweight residual module based on Ghost convolution is used. In order to solve the problem of low accuracy of behavior recognition in the noisy environment of oil field well sites, a triplet attention mechanism is introduced in the feature extraction stage to increase attention to significant features; the similarity of action appearance leads to similar short-term spatiotemporal features extracted, which cannot provide key information for identifying actions. In order to solve the problem that similar behaviors are prone to misjudgment, a Bi-GRU enhancement model is introduced to extract key long-term features. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1It is a framework diagram of GTB-ResNet (Ghost Triplet Bi-GRU ResNet50) of the present invention;
[0023] Figure 2 (a) and (b) are the Ghost Bottleneck module and Ghost_conv3D of the present invention respectively;
[0024] Figure 3 It is the triplet attention mechanism;
[0025] Figure 4 This is the Bi-GRU structure diagram;
[0026] Figure 5 It is a dataset of abnormal behaviors of construction workers at oil stations;
[0027] Figure 6 This is the experimental result on the abnormal behavior dataset of construction workers at oil stations;
[0028] Figure 7 This is a visual diagram of the test experiment;
[0029] Figure 8 This is the experimental result on the abnormal behavior dataset of construction workers at oil stations;
[0030] Fig. 9 It is a confusion matrix diagram. DETAILED DESCRIPTION
[0031] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.
[0032] like Figure 1 As shown in the figure, the GTB-ResNet recognition method for abnormal behavior of oil station personnel includes the following steps:
[0033] Step 1: Use 3D ResNet50 network as the preliminary extraction network;
[0034] Firstly, a lightweight residual module is used to analyze the spatiotemporal information of consecutive frames in the image of the video stream. Then, the extracted feature vector sequence is input into the Bi-GRU module for temporal feature screening and updating. Finally, the behavior results are identified and classified according to the output of Bi-GRU. The proposed network introduces Ghost convolution in the feature extraction module and redesigns the residual module, which greatly reduces the model size and computational complexity. In addition, a Triplet attention mechanism is added after the backbone network to enhance the model's ability to learn significant features and better locate and analyze the dynamic information of the action.
[0035] Some efficient neural architecture designs are based on deep networks with fewer parameters and computational complexity. For example, MobileNet and ShuffleNet use deep convolution or channel shuffling operations to use smaller convolution filters to enhance the performance of lightweight models. However, for smaller targets, very little information is left after multiple layers of convolution, while the rich or even redundant information in the feature maps of deep neural networks can ensure a comprehensive understanding of the input data. Therefore, we prefer to use redundant feature maps in a cost-effective way. This article uses phantom convolution to replace the standard convolution in the original network. The traditional three-dimensional residual network has the problems of long feature extraction time and large network parameters. This is because the CNN composed of standard convolution methods has high computational cost and the generated feature maps have more redundancy. Phantom convolution (GhostConv3D) is used to replace part of the convolution in the residual module to obtain the Ghost Bottleneck module, which realizes a lightweight and efficient network model. The structural diagram is shown in the figure. Figure 2 As shown in Figure 2, when stride = 1, the Ghost Bottleneck module is as follows: Figure 2 In the left figure of (a), when stride = 2, the Ghost Bottleneck module is as follows Figure 2 (a) right figure.
[0036] Figure 1 As shown in the figure, the backbone network of the GTB-ResNet network includes stage1 to stage4. Stage1 is connected by 3 Ghost Bottleneck modules with a stride of 1 and 1 Triplet attention module; stage2 includes 4 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage3 includes 6 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage4 includes 3 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module.
[0037] Figure 2 In the left half of (a), the Ghost Bottleneck module with stride 1 includes: the first Ghost_Conv3D, BN+Relu, the second Ghost_Conv3D and BN, and is residually connected with the first Ghost_conv3D.
[0038] Figure 2In the right half of (a), the Ghost Bottleneck module with stride 2 includes: the third Ghost_Conv3D, BN+Relu, DWConv, BN, the fourth Ghost_Conv3D and BN, and is residually connected with the third Ghost_Conv3D.
[0039] like Figure 2 (b) is Ghost_conv3D. First, the ordinary convolution layer in the convolution network is divided into two parts. One part is a specific number of ordinary convolutions to generate the essential feature map. Then, the depth-wise separable convolution is used to perform linear transformation on the essential feature map to obtain the redundant feature map. Finally, the two feature maps are cascaded to obtain a feature map with the same size and channel size as the traditional convolution layer output. Since the linear transformation is performed on each channel, its computational cost is much lower than that of ordinary convolution, which can be expressed by equation (1):
[0040] Y'=X*f' (1)
[0041] Among them, Y' represents the generated feature map, X is the given input data, and f' represents the hyperparameter.
[0042] In order to further obtain the required n feature maps, a series of linear operations are performed on each inherent feature of Y' to generate s phantom features according to the following function, as shown in formula (2):
[0043]
[0044] Among them, y ij is the i-th feature map in Y', Φ i,j Is to generate the jth ghost feature map y ij The jth cheap linear operation of , that is, y i 'There can be one or more ghost feature maps
[0045] The last Φ i,s It is an identity mapping used to preserve the intrinsic feature map. Through the equation, n = m s feature maps Y = [y 11 ,y 12 ,...,y ms ] as the output of the ghost convolution.
[0046] After the 3D ResNet backbone network extracts the multi-channel feature map, an attention mechanism is added to increase the network's learning of the key feature maps.
[0047] In convolutional neural networks, the attention mechanism assigns higher weights to the area where the target of interest is located to increase the importance of the target features, and assigns lower weights to the background area to constrain invalid features, thereby giving priority to the area where the target of interest is located to obtain more information and enhance the network's ability to extract target features. The Triplet attention module is introduced after the Ghost Bottleneck module in stage 1 to stage 4 of the 3D ResNet backbone network. The structural diagram of the Triplet attention mechanism is shown in the figure below. Figure 3 shown.
[0048] The triplet attention mechanism calculates the attention weight by obtaining the interaction of different dimensions. For the input tensor, the triplet attention establishes the dependency between dimensions through rotation operation, then performs residual transformation, and encodes the inter-channel and spatial information with negligible computational overhead. Figure 3 There are three branches. The top branch is responsible for calculating the attention weights across the channel dimension C and the spatial dimension W. Similarly, the middle branch is responsible for the channel dimension C and the spatial dimension H. The last branch at the bottom is used to capture the spatial dependencies (H and W). In the first two branches, rotation operations are used to establish the connection between the channel dimension and any spatial dimension. Finally, the weights are aggregated by simple averaging.
[0049] For an input tensor χ∈R C×H×W , the process of obtaining the refined attention application tensor y from the Triplet attention is expressed as formula (3):
[0050]
[0051] Where σ represents the sigmoid function; ψ1, ψ2, ψ3 represent the standard two-dimensional convolutional layers defined by the kernel size K in the three branches of the triplet attention mechanism, which can be simplified as formula (4):
[0052]
[0053] Among them, ω1, ω2, and ω3 are the three cross-dimensional attention weights calculated in the triplet attention. It represents a 90° clockwise rotation to preserve the original input shape (C×H×W).
[0054] Bi-GRU is introduced after the output of the 3D ResNet backbone network to enhance the correlation and timing of spatiotemporal features. Bi-GRU can capture long-distance dependencies in the sequence and realize the prediction of action recognition. This bidirectional structure enables Bi-GRU to effectively learn from the encoded hidden sequence patterns.
[0055] The forward and reverse GRU layers of Bi-GRU are composed of GRU units, each of which consists of a reset gate γ and an update gate μ, with two activation functions sigmoid and than; the reset gate determines whether the GRU needs to forget or retain the part of the information according to its value; when the output value of the reset gate is close to 0, the reset gate forgets the previous part of the sequence, and if the value of the reset gate is close to 1, the reset gate retains the previous part of the sequence; the update gate determines the amount of information retained from the previous hidden state to the current hidden state according to its value; when the value of the update gate is close to 0, the update gate only forgets part of the information of the previous hidden state, and retains part of the information from the previous hidden state to the current hidden state when the value is close to 1. The operations of these gates can be expressed as follows:
[0056] r t =σ(ω r ·x t +u r ·h t-1 ) (5)
[0057] μ t =σ(ω μ ·x t +u μ ·h t-1 ) (6)
[0058]
[0059]
[0060] y t =σ(ω0·h t ) (9)
[0061] Among them, r t and μ t They represent the reset gate and update gate respectively, and their values are between 0 and 1; w and u are weight variables, x t is the input value of the GRU layer; ω0 is the output layer of the weight variable of the input and intermediate layers; y t Represents the output layer node at time t; is the candidate hidden state of the current node; h t is the current hidden state; h t-1 is the hidden state of the previous node.
[0062] Experimental process:
[0063] Due to the particularity of the oil station environment, there are few studies on the correlation of some abnormal behavior recognition of oil station workers in videos, and there is a lack of worker behavior data sets that can be directly used in real well site work scenarios; the abnormal behavior data of construction workers shot at a certain oil station site were obtained through a video monitoring platform, including three abnormal behaviors of illegal use of mobile phones, smoking, and accidental falls, and one normal behavior of touching the face, so that each short video contains at least one behavior; after processing, a total of 892 videos were obtained, with a video length of 4-10 seconds and a frame rate of 30f / s; the videos were placed in corresponding folders according to the different behaviors, and the folders were named after the behavior type; the produced videos included: samples of different time periods and different lighting conditions, which ensured the adaptability of the method of the present invention to all-weather environments and the richness of training data; the obtained videos were divided into training set and verification set in a ratio of 8:2, and there was another test set containing 20 clips of each of the four types of behavior for testing, and the video data set part is as follows Figure 5 .
[0064] Table 1. Details of the abnormal behavior dataset of construction personnel at oil stations
[0065]
[0066] The input image size of the GTB-ResNet network of the present invention is 112 pixels×112 pixels during training, the initial learning rate of the network is 0.01, and the epoch decay strategy is used. The parameter value is updated once every 10 iterations by dividing the learning rate by 10, and the optimizer is Adam; the training batch size is set to 8; the experiment adopts the transfer learning method, and the Resnet network pre-trained on the Kinetics-400 data is used for parameter initialization, and some model parameters are fixed after the training is completed. The training is carried out on the data set of construction personnel of the oil station through transfer learning, the hidden layer dimension is set to 512, the feature vector sequence extracted by Bi-GRU is input to the fully connected layer with a dimension of 1024, and the output dimension is a vector of 5; the parameter dim of the last Softmax layer of the network is set to 1. The number of iterations performed in the experiment is 200.
[0067] The experiment is based on the Python 3.8 programming language and the PyTorch 1.10.0 deep learning framework to build the model. The specific experimental environment is: Intel (R) Xeon (R) Platinum 8375C processor, NVIDIA A100-SXM graphics card and 91G running memory. The operating system is 63-bit Ubuntu 20.04 and the CUDA version is 11.1.
[0068] Considering that the model of the present invention is mainly used in edge devices, in addition to precision, recall and recognition accuracy, frame rate, floating-point operations and parameter quantity are also the main indicators for evaluating the model size.
[0069] Video action recognition is a multi-class classification task. For a multi-classification task, assuming that the total number of samples is N and the total number of classes is M, in the prediction results, the r in the prediction of the i-th class is i The sample predictions are correct.
[0070] For data sets with more behavior categories, some types have higher similarity, and Top-1 and Top-5 accuracy are used as evaluation indicators at the same time; Top5 (f(x)) represents the 5 most likely types in the predicted results; the number of categories in all data sets used in the present invention is very small, and the differences between categories are large, so the Top-1 classification accuracy is used in subsequent experiments.
[0071] The training process of the network model is as follows Figure 6 As shown in the figure, a total of 200 iterations were run. From the figure, it can be seen that in the first 80 iterations of training, although the recognition accuracy and loss value of the model showed obvious oscillations, they showed an overall trend of rapid increase and rapid decrease respectively; from 80 to 200 iterations, the rising trend of the recognition accuracy of the model gradually slowed down, and the decreasing trend of the loss value also gradually slowed down until it stabilized.
[0072] 80 videos in the test set were input into the trained GTB-ResNet network for behavior recognition. When inputting videos, a sliding window sampling mechanism was used to input continuous video frame images into the GTB-ResNet network. One frame was extracted every three frames, and 16 frames were extracted continuously as a sliding window. Table 2 shows the recognition results of the test set. It can be seen from Table 2 that the recognition accuracy of smoking and touching the face is low, and the recognition accuracy of falling is the highest, which is 98.7%. From the perspective of recall rate, the same pattern is shown. The comprehensive recognition accuracy of the network model reached 96.7%, which shows that the GTB-ResNet network has a good recognition effect on behaviors with obvious motion characteristics.
[0073] Table 2 Precision and recall of different behaviors
[0074]
[0075]
[0076] The visual effects of the test experiment are as follows Figure 7 As shown, in Figure 8In the figure, representative frames and ground truths of the predicted action clips are presented, and the actions predicted by the model are based on the confidence scores on the probability prediction bar chart; from the presented visual effects, it can be seen that the action recognition rate predicted by the GTB-ResNet model is very good, with a confidence level of about 95%; for some cases where the motion features are less obvious, there will be a non-zero probability of erroneous actions, but the probabilities of these erroneous action categories are still low and do not affect the prediction of the actual action category; therefore, the obtained qualitative visual results verify the effectiveness of the actual use of the model of the present invention in different vision-based human action recognition and monitoring environments.
[0077] Figure 7 Visual recognition results of the GTB-ResNet model with predicted categories and their confidence scores for the test videos obtained from the dataset.
[0078] The superiority and effectiveness of the proposed method are verified on the abnormal behavior dataset of oil station construction workers. The most advanced testing methods are used, including C3D, I3D, MobileNetV2, ShuffleNetV2 and X3D. The C3D loss is calculated by categorical cross entropy, the optimizer uses rmsprop, the initial learning rate is 0.001, and the epoch is 200 rounds. Different architectures are selected according to the complexity of the data. The MobileNetV2 learning strategy is 1.0x and the ShuffleNetV2 is 2.0x. The optimization algorithm is SGD, the momentum factor is 0.9, the weight decay factor is set to 0.001, the initial learning rate is 0.01, and it is reduced twice after the 30th and 45th epochs, the factor is 0.1, and the optimization is completed after 15 epochs. The total training epoch is 200 rounds. The initial learning rate of I3D and X3D-M is set to 0.01, and it is increased to the set base learning rate of 0.1 after the 35th epoch. A half-period cosine schedule of learning rate decay is adopted, the momentum factor is 0.9, and the weight decay of the former is set to 1×10 -4 , the latter is set to 5×10 -5 , a total of 200 epochs were trained.
[0079] Table 3 Comparison of different detection methods on the abnormal behavior dataset of construction personnel in oil stations. Pre means pre-training with the dataset: Im., S.1M, K.400, K.600, and K.700 represent ImageNet,
[0080] Table 3Sport1M, Kinetics400, Kinetics600, Kinetics700
[0081]
[0082] As shown in Table 3, the I3D network based on the combination of dual-stream network and 3D convolutional neural network has the lowest recognition accuracy of 67.2%; because the I3D network needs to extract optical flow information, the outdoor light intensity makes it more difficult to extract optical flow information, and the ability to capture information about small target people with small movements is poor; therefore, the recognition effect is poor when the movement amplitude of outdoor people is not obvious; C3D has the largest number of parameters, and the number of C3D network layers is shallow, which is easy to overfit. Therefore, it is not suitable for deployment on the edge devices of oil stations; MobileNetV22.0x has a recognition accuracy of 76.7% and ShuffleNetV22.0x has a recognition accuracy of 80.1%. In order to save computational effort, MobileNetV2 changes input to low-dimensional features, but reducing the dimensionality is not enough to retain sufficient useful information. Similar behaviors in the data set cannot be well distinguished in low-dimensional features, and information about subtle differences between behaviors is easily lost. Similarly, ShuffleNetV2 also suffers from feature loss. The network may lose some important features or details when transmitting information, and the distinction between similar behaviors in the data set is not obvious. X3D improves the efficiency of 3D convolution by progressive expansion, and the accuracy can reach 90.4%, but requires a large amount of memory resources over a large time window. The model of the present invention utilizes the residual structure of the Resnet network to maintain the utilization of shallow features. Ghost convolution uses cheap linear transformations to ensure the retention of feature information for redundant features, rather than simply utilizing low-dimensional information to save computational effort, and can alleviate the problem of large 3D convolution parameters and large computational effort in the residual module. At the same time, Bi-GRU is used to solve the long-distance dependency on time series, thereby improving the recognition accuracy of the model for similar behaviors. Triplet attention is used to enhance the perception relationship between channels and spatial position information, thereby improving the positioning and attention to significant features of behavior recognition. In addition, the method of the present invention also achieves the highest accuracy.
[0083] like Figure 8 As shown in the figure, the experimental results on the abnormal behavior dataset of construction workers at oil stations include accuracy and loss rate; Figure 8 Including C3D, ShuffleNetV2, X3D and the original 3D ResNet and the improved 3D ResNet. It can be seen from the figure that after 200 iterations, the average loss function and average accuracy of each round tend to be stable, and the final loss value drops to about 0.1. The convergence effect of the loss function is also better than other models. The recognition accuracy of the GTB-ResNet model is finally stabilized at 96.7%. In addition, the GTB-ResNet network model has almost no large oscillations during the entire training process. It can be analyzed that the training results of the proposed model are relatively ideal and have a high recognition accuracy.
[0084] In order to further verify the feasibility of the GTB-ResNet model and the different effects of each module on the recognition results, an ablation experiment was carried out on each module on the abnormal behavior dataset of oil station construction personnel. 3D ResNet50 was used as the baseline. Table 4 shows the model performance before and after the introduction of Ghost Bottleneck, Triplet, and Bi-GRU.
[0085] Table 4 Ablation experiment
[0086]
[0087] As shown in Table 4, after replacing the ordinary convolution with Ghost convolution, the lightweight residual structure GhostBottleneck is constructed. The memory usage of the network model is reduced by 90.4%, GLOPs is reduced by 61.7%, and the frame rate reaches 67.34f / s, and the execution speed is increased by 14.19%, laying the foundation for subsequent application to edge devices; the improved model accuracy can reach 95.6%, proving that the improved network model can still effectively reduce the number of parameters for model training while maintaining a high accuracy, which brings the possibility of lightweight; after introducing the Triplet attention mechanism alone, the model accuracy is increased by 2.8 percentage points to 95.1%. Although the frame rate drops by 7.22f / s, the introduction of floating point The amount of computation and model parameters changed little, which strengthened the focus on key channel and spatial information; after Bi-GRU was introduced into the model, the model accuracy reached 94.8%, an increase of 2.5%. After the introduction, the amount of floating-point computation and model parameters changed little, indicating that the Bi-GRU structure has better performance for the network feature fusion of long time series; after the introduction of GhostBottleneck, Triplet, and Bi-GRU, the model accuracy increased by 4.4%, and the frame rate reached 61.64f / s, which is higher than the video frame rate of 30.64f / s, and can meet the requirements of real-time recognition; in addition, the floating-point computation and parameter volume of the model decreased by 61.5% and 90.4% respectively, achieving high-precision behavior recognition with minimal computational overhead.
[0088] For the recognition efficiency of similar actions, the CV accuracy confusion matrix is drawn according to the four types of actions, such as Fig. 9 As shown, Fig. 9 It shows that the confusion rate of illegal phone calls, smoking and touching the face is only between 0.01-0.2, which shows that the model of the present invention has a good recognition effect. Therefore, the GTB-ResNet model improves the accuracy of behavior recognition, while effectively reducing the amount of network parameters, accelerating the training speed and shortening the training time.
[0089] The present invention integrates Ghost Bottleneck, Triplet, and Bi-GRU to design a lightweight three-dimensional residual network GTB-ResNet for behavior recognition of oil station workers. Ghost Bottleneck is designed to replace the residual module in the three-dimensional residual network, which can greatly reduce the model calculation complexity and model parameter quantity, and the behavior recognition accuracy on the oil station construction personnel data set is 96.7%; GTB-ResNet has lower computational complexity and smaller model parameter quantity compared with 3DResNet, the floating-point operation quantity is 3.90G, and the parameter quantity is 4.46M. Compared with other lightweight networks, it has the highest recognition accuracy and is suitable for deployment on edge devices.
[0090] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. The GTB-ResNet recognition method for abnormal behavior of oil station personnel is characterized by: The following steps are involved: Step 1: Collect abnormal behavior images of construction personnel at oil stations; Step 2: Build the GTB-ResNet network. The backbone network of the GTB-ResNet network includes stage 1 to stage 4. Stage 1 is connected by 3 Ghost Bottleneck modules with a stride of 1 and 1 Triplet attention module; stage 2 includes 4 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage 3 includes 6 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; stage 4 includes 3 Ghost Bottleneck modules with a stride of 2 and 1 Triplet attention module; Step 3: Input the abnormal behavior image into the GTB-ResNet network for abnormal behavior recognition.
2. The GTB-ResNet recognition method for abnormal behavior of oil station personnel according to claim 1 is characterized in that: The Ghost Bottleneck module with stride 1 includes: the first Ghost_Conv3D, BN+Relu, the second Ghost_Conv3D and BN, and is residually connected with the first Ghost_conv3D.
3. The GTB-ResNet method for identifying abnormal behavior of oil station personnel according to claim 2 is characterized in that: The Ghost Bottleneck module with stride 2 includes: the third Ghost_Conv3D, BN+Relu, DWConv, BN, the fourth Ghost_Conv3D and BN, and is residually connected with the third Ghost_Conv3D.
4. The GTB-ResNet method for identifying abnormal behavior of oil station personnel according to claim 3 is characterized in that: The output of the backbone network of the GTB-ResNet network is connected to the BI-GRU network.
5. The GTB-ResNet recognition method for abnormal behavior of oil station personnel according to claim 1 is characterized in that: Abnormal behaviors include: using mobile phones, smoking and accidental falls.
6. The GTB-ResNet method for identifying abnormal behavior of oil station personnel according to claim 1 is characterized in that: The GTB-ResNet network is evaluated using precision, recall, recognition accuracy, frame rate, floating-point operations, and parameter count.
7. The GTB-ResNet method for identifying abnormal behavior of oil station personnel according to claim 1 is characterized in that: The confusion matrix is used to calculate the confusion rate between abnormal behaviors and normal actions.
8. The GTB-ResNet recognition system for abnormal behavior of oil station personnel is characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the GTB-ResNet method for identifying abnormal behavior of personnel at a petroleum station as described in any one of claims 1 to 7.
9. A computer readable medium storing a computer program code, characterized in that: The computer program code, when executed by a processor, implements the GTB-ResNet method for identifying abnormal behavior of oil station personnel as described in any one of claims 1 to 7.
Citation Information
Cited By
Natural gas pipeline anomaly detection method based on DN-A and MFF
CN120015054A
