A method for identifying abnormal behavior in electric vehicle charging stations

Abstract

A method for identifying abnormal behavior in electric vehicle charging stations, based on the constraints of edge computing resources, comprises the following steps: First, images related to abnormal behavior are acquired at the electric vehicle charging station and compiled into a sample dataset. Second, an architecture for the recognition model is built using the YOLO model structure as a framework. Third, a lightweight convolutional module is constructed. Fourth, a lightweight abnormal behavior recognition component is built. Fifth, the backbone of the YOLO framework is replaced with the lightweight abnormal behavior recognition component to obtain the final recognition model. Sixth, the recognition model is trained using the sample dataset for identifying abnormal behavior in electric vehicle charging stations. This invention can achieve rapid identification of abnormal behaviors in charging stations, such as people falling to the ground and vehicles queuing, using relatively few edge computing resources.

Description

Technical Field

[0001] This invention belongs to the field of computer artificial intelligence technology, involving theories such as convolutional neural networks, artificial intelligence, and deep learning. Based on the limitations of edge computing resources, it is a method for identifying abnormal behavior in electric vehicle charging sites. Background Technology

[0002] The rapid development of artificial intelligence and neural networks has provided a more efficient method for image recognition, and also offered a feasible path for image-based abnormal behavior recognition in electric vehicle charging stations. Currently, neural network algorithms such as YOLO have been maturely applied in the field of object detection. However, unlike other object detection fields, charging station environments are complex, with a large number of images and high speed requirements for recognition. Traditional neural network methods not only have high computational resource requirements but also relatively slow operating speeds, making it difficult to meet the requirements of charging stations for real-time alarms for abnormal behavior.

[0003] Most charging stations have limited computing resources at the edge. To better identify abnormal behavior in electric vehicle charging stations under these constraints, developing lightweight target detection models is essential. Simultaneously, generating redundant features plays a crucial role in reducing computational costs. In existing technologies, the number of model parameters is considered the primary goal of model lightweighting. However, existing research shows that the number of parameters is not positively correlated with model speed. In recent years, more attention has been paid to the number of floating-point operations per second (FLOPs). Although the actual running speed of a model is still affected by the framework, operating system, and implementation details, FLOPs are still widely recognized as the most theoretically sound metric. This invention aims to research a new lightweight model with fewer FLOPs to adapt to the limitations of computing resources in edge computing. Summary of the Invention

[0004] The technical problem this invention aims to solve is: given the limited computing resources at the edge of charging sites and the high requirements for abnormal behavior recognition speed, how can we achieve the recognition of abnormal behavior in charging sites at a lower cost and with higher efficiency?

[0005] The technical solution of this invention is: a method for identifying abnormal behavior in electric vehicle charging sites, which involves constructing an identification model in edge computing to automatically identify abnormal behavior in charging sites, including the following steps:

[0006] The first step is to acquire images related to abnormal behavior at electric vehicle charging stations and organize the images into a sample dataset.

[0007] The second step is to build the architecture of the recognition model based on the YOLO model structure.

[0008] The third step is to construct a lightweight convolutional module to generate features related to abnormal behavior recognition from the input image. The input abnormal behavior recognition image samples are divided into two branches for processing. One branch generates a semi-redundant feature map, called the "skip branch". The other branch is called the "lightweight main branch", which generates the other half of the redundant feature map through the lightweight network. Then the two are connected.

[0009] The fourth step is to construct a lightweight abnormal behavior recognition component to identify the features generated in the third step; the lightweight abnormal behavior recognition component consists of several lightweight convolutional modules.

[0010] The fifth step is to replace the main part of the YOLO framework with a lightweight abnormal behavior recognition component to obtain the final recognition model.

[0011] The sixth step involves training a recognition model using the sample dataset to identify abnormal behavior at electric vehicle charging stations.

[0012] Furthermore, the lightweight main branch includes an extension module and a spatial feature processing module. The output of the spatial feature processing module is passed through a 1×1 convolutional layer to output the feature map of the branch. The skip branch includes a 1×1 convolutional layer, which is then passed through average pooling to output the feature map of the branch. In the lightweight main branch, samples are first extracted for detail features by average pooling, and then concatenated with the feature map output by the skip branch in the extension module. The samples are then input into the spatial feature processing module for spatial feature extraction. In the spatial feature processing module, the encoding network and the prediction network first output spatial features and determine whether undersampling is needed. If so, the samples are input into a shrinking module and then passed through the Mish activation function before being output. Otherwise, the samples are directly passed through the Mish activation function before being output. The encoding network and the prediction network are two-layer recurrent convolutional structures. The output of the Mish activation function is passed through a 1×1 convolutional layer and then concatenated with the feature map output by the skip branch to obtain the abnormal behavior feature map.

[0013] Furthermore, the extension module includes multi-layer deep convolution operations and the Mish activation function. The feature maps of the skipped branches are extended using deep convolution with an extension ratio of 2. The extension is then concatenated with the detailed features of the lightweight main branch and output after passing through the Mish activation function.

[0014] Furthermore, the lightweight abnormal behavior recognition component includes a 3×3 convolutional layer with a stride of 2, a 2× lightweight convolutional module, a 4× lightweight convolutional module, a first 6× lightweight convolutional module, a second 6× lightweight convolutional module, and an 8× lightweight convolutional module. Pooling layers are added to the 2× lightweight convolutional module, the 4× lightweight convolutional module, the first 6× lightweight convolutional module, and the 8× lightweight convolutional module to perform undersampling. The outputs of the 4× lightweight convolutional module, the second 6× lightweight convolutional module, and the 8× lightweight convolutional module are three different scaled feature map outputs.

[0015] The technical features of this invention are as follows:

[0016] 1. In view of the actual characteristics of electric vehicle charging site management, such as limited computing resources and high requirements for the accuracy and speed of abnormal behavior identification, an abnormal behavior identification method based on the improved YOLO algorithm for charging sites was designed.

[0017] 2. An improvement has been made to the YOLO algorithm commonly used in traditional target monitoring, overcoming the shortcomings of the YOLO algorithm, such as high computational resource requirements and slow computation speed.

[0018] 3. A lightweight convolutional module was constructed, and a lightweight abnormal behavior recognition component was built based on the lightweight convolutional module. This component can generate more redundant feature maps with less floating-point calculation based on the actual situation of the charging site, and realize the abnormal behavior recognition of electric vehicle charging sites more efficiently.

[0019] 4. Abnormal behaviors in electric vehicle charging stations are diverse and difficult to identify. To address the challenges of identifying complex spatial and detailed features, this invention introduces a shrinking module, average pooling, and recurrent convolution operations. This enables the effective capture of abnormal features with fewer resources, overcoming the problem of failing to extract certain detailed and spatial features in previous research and inventions, thus significantly improving the accuracy and effectiveness of identification.

[0020] The beneficial effects of this invention include the following two aspects:

[0021] 1. To fully adapt to the resource constraints of edge computing in electric vehicle charging stations, a new algorithm framework was constructed under limited resource conditions. The model can be run with fewer FLOPs, i.e. fewer computing resources are used, which plays an important role in promoting the identification of abnormal behavior in electric vehicle charging stations.

[0022] 2. This invention greatly improves the running speed of the algorithm, overcomes the shortcomings of the traditional YOLO algorithm in terms of slow running speed, and can more quickly identify abnormal behavior in electric vehicle charging stations, which helps to improve the safety management level of charging stations. Attached Figure Description

[0023] Figure 1 This is a flowchart of the present invention.

[0024] Figure 2 This is the YOLO framework diagram for the second step of this invention.

[0025] Figure 3 This is the overall architecture diagram of the lightweight convolution module in the third step of this invention.

[0026] Figure 4 This is a schematic diagram of the lightweight abnormal behavior recognition component in the fourth step of the present invention. Detailed Implementation

[0027] This invention primarily relies on image classification for abnormal behavior recognition. It constructs an algorithm framework for abnormal behavior recognition in charging stations, addressing the limitations of edge computing resources and the security requirement for rapid abnormal behavior identification. This invention utilizes edge computing to build a recognition model for automatic identification of abnormal behavior in charging stations, comprising the following steps:

[0028] The first step is to acquire images related to abnormal behavior at electric vehicle charging stations and organize the images into a sample dataset.

[0029] The second step is to build the architecture of the recognition model based on the YOLO model structure.

[0030] The third step is to construct a lightweight convolutional module to generate features related to abnormal behavior recognition from the input image. The input abnormal behavior recognition image samples are divided into two branches for processing. One branch generates a semi-redundant feature map, called the "skip branch". The other branch is called the "lightweight main branch", which generates the other half of the redundant feature map through the lightweight network. Then the two are connected.

[0031] The fourth step is to construct a lightweight abnormal behavior recognition component to identify the features generated in the third step; the lightweight abnormal behavior recognition component consists of several lightweight convolutional modules.

[0032] The fifth step is to replace the main part of the YOLO framework with a lightweight abnormal behavior recognition component to obtain the final recognition model.

[0033] The sixth step involves training a recognition model using the sample dataset to identify abnormal behavior at electric vehicle charging stations.

[0034] To make the purpose, technical solution, and advantages of the invention more apparent and clear, the invention will be described in detail below with accompanying drawings and specific examples.

[0035] like Figure 1 As shown, the present invention mainly consists of 6 steps in actual operation.

[0036] Step 1: Acquire images related to abnormal behavior at electric vehicle charging stations and organize the images into a sample dataset.

[0037] Cameras connected to the electric vehicle charging station capture images of the station's status. These captured images are denoted as samples X, {x1, x2, ..., x...}. n}∈X. This invention will develop a method for identifying abnormal behaviors in electric vehicle charging stations under edge computing resource constraints, accurately identifying abnormal behaviors such as people falling to the ground and vehicles queuing in different samples.

[0038] Step 2: Build the YOLO model framework.

[0039] The YOLO algorithm focuses on fast target recognition, and the method proposed in this invention is an update within the YOLO model framework. The YOLO model selected in this invention contains 24 convolutional layers and 2 fully connected layers. Specific framework details are as follows... Figure 2 As shown, the model framework consists of three parts: CSP-Darknet53 as the backbone, SPP and PANet as the neck, and a head identical to the YOLOv4 algorithm. The backbone includes the Bottleneck CSP (BottleNeck Local Branch Network) and Spatial Pyramid Pooling; the neck includes convolutional layers, fully connected layers, upsampling, and the Bottleneck CSP; the head mainly consists of some convolutional layers. In this framework, the initial 24 convolutional layers are responsible for extracting abnormal behavior features from the charging site image, while the fully connected layers predict the output probability and coordinates. This constitutes the basic method for identifying abnormal behavior in charging sites using the YOLO algorithm, and the YOLO model framework constructed in this invention is also the usual arrangement and practice of the YOLO model algorithm. However, the basic YOLO framework suffers from slow model execution speed and high computational resource requirements. In the next step, this invention will introduce a lightweight convolution module into the basic YOLO framework to improve the computation speed, so that it can better meet the requirements of identifying abnormal behavior in charging fields.

[0040] Step 3: Build a lightweight convolutional module.

[0041] To make the charging station abnormal behavior recognition model framework more lightweight, a lightweight convolutional module will be introduced in this step. This module can generate redundant feature maps related to abnormal behavior recognition with less computation, thereby significantly reducing FLOPs, lowering computational resource usage, and improving running speed. Existing technologies have conducted extensive research on how to achieve lightweighting of the YOLO algorithm, such as introducing the Ghost module. The operation of this module mainly consists of three steps: 1. Using a 1×1 convolutional kernel, obtain the necessary feature condensation of the input charging station abnormal behavior image features; 2. Using depthwise separable convolution to obtain similar feature maps of feature condensation; 3. Concatenating the generated abnormal behavior feature maps in the channels to achieve the ideal output effect. The Ghost module is more suitable for the scenario of abnormal behavior recognition in electric vehicle charging stations. However, the main operation of generating valuable redundant feature maps in the Ghost module still consumes a lot of computational resources for edge computing in charging stations. Considering that most charging stations have scarce edge computing resources, further optimization is still needed.

[0042] The lightweight convolution module design proposed in this invention is as follows: Figure 3 As shown, the input abnormal behavior feature mapping, i.e., the abnormal behavior recognition image sample, is divided into two branches. The first branch, like the Ghost module, generates a semi-redundant feature map through simple operations. In this invention, the first branch is called the "skip branch"; the other branch is called the "lightweight main branch," which generates the other half of the redundant feature map through a lightweight network and then connects them. The lightweight main branch includes an extension module and a spatial feature processing module. The output of the spatial feature processing module is passed through a 1×1 convolutional layer to output the feature map of the branch. The skip branch includes a 1×1 convolutional layer, which is then averaged before outputting the feature map of the branch. In the lightweight main branch, samples are first averaged to extract detailed features, and then concatenated with the feature map output from the skip branch in the extension module. The expanded feature map is then input into the spatial feature processing module for spatial feature extraction. In the spatial feature processing module, the encoding and prediction networks first output spatial features and determine whether undersampling is needed. If so, the features are input into a shrinking module and then passed through the Mish activation function before being output. Otherwise, the features are directly passed through the Mish activation function before being output. The encoding and prediction networks are two-layer recurrent convolutional structures. The output of the Mish activation function is passed through a 1×1 convolutional layer and then concatenated with the feature map output from the skip branch to obtain the abnormal behavior feature map.

[0043] The specific design details of the lightweight main branch are as follows.

[0044] First, considering the practical situation of abnormal behavior recognition in electric vehicle charging stations, a lightweight convolutional module architecture is designed to meet the computational resource and speed requirements of abnormal behavior recognition in charging stations. This architecture includes an extension module, a spatial feature processing module, and multiple 1×1 convolutional layers and depthwise convolutional layers. 1×1 convolutions and depthwise convolutions effectively reduce the algorithm's FLOPs, decrease the resources required for edge computing in abnormal behavior recognition in charging stations, and improve the running speed of this branch. In this invention, Mish is selected as the activation function to perform nonlinear transformations on the sample data.

[0045] Second, the present invention designs the following in the main branch: Figure 3 One of the extension modules shown uses the input and output feature maps of the skipped branches to generate intermediate candidate feature maps through deep convolution. This module includes multiple layers of deep convolution operations and the Mish activation function. Existing research and inventions have shown that Mish can deliver superior performance in deep neural network models, enabling operations such as rotating samples to better alter the linear relationship of the previous input sample data, thereby improving the model's expressive and classification capabilities. In the extension module, 1×1 convolutions and other methods are used to expand the feature maps of the skipped branches to obtain more effective abnormal behavior recognition features. The parameter t represents the feature expansion ratio. Considering the training conditions on the charging field data and actual needs, this invention sets t to 2 or 3.

[0046] Third, there are numerous features for identifying abnormal behavior in charging stations, and different categories of abnormal behavior may have overlapping features. Therefore, a lightweight convolutional module is needed to accurately extract the detailed features for identifying abnormal behavior. To enable the developed lightweight convolutional module to have this capability, this invention introduces an attention mechanism, that is, in the model such as... Figure 3 Adding average pooling operations at appropriate locations as shown enables the module to identify more abnormal behavior features.

[0047] Fourth, the environmental characteristics of charging stations are spatial and three-dimensional. For identifying abnormal behaviors such as vehicle queuing and people falling, considering only planar, textured features may lead to inaccurate identification due to physical obstruction by vehicles and charging piles. Therefore, this module needs to be able to capture spatial details. This invention designs a spatial feature processing submodule, which is added to the lightweight convolutional module. To maintain a lightweight design while extracting spatial features, the spatial feature processing module consists of only two spatial feature extraction networks and activation functions. If undersampling is required, a shrinking module, i.e., the SE module (Squeeze-and-Excitation Networks), can be added. The two spatial feature extraction networks are an encoding network and a prediction network. The initial state and output of the prediction network are copied from the final state of the encoding network. Both networks are formed by stacking several left and right recurrent convolutional layers. Due to the need to maintain a lightweight design, the network mainly extracts spatial information of important concepts such as people, electric vehicles, and charging piles; therefore, each network generally has two recurrent convolutional layers. Since the prediction target of this invention has the same dimension as the input, all states in the prediction network are connected together, and after passing through a shrinking module and an activation function, they are input into a 1×1 convolutional layer to generate the final prediction. By introducing a spatial feature processing submodule into the lightweight convolutional module, features related to the spatial state of the charging site can be better captured, improving the accuracy of predictions such as vehicle queuing and people falling to the ground.

[0048] Fifth, there are numerous abnormal behaviors in charging sites. For example, charging gun misplacement and personnel not wearing safety helmets are relatively common, while perimeter intrusion into core equipment and smoke alarms in the charging site are relatively rare. This can lead to class imbalance in the input samples for identifying abnormal behaviors in charging sites, which can be addressed through undersampling. When undersampling is required after the expansion module, this invention will use a shrinking module in the spatial feature processing submodule of the lightweight convolution module to better address the identification accuracy issues of different categories of abnormal behaviors and better learn the identification features of various abnormal behaviors, including spatial features.

[0049] In the lightweight convolutional module, the input sample size is H×W×C, and the output feature map size is H×W×N. The FLOPs of the convolution are H'×W'×C×K. 2 ×N. Where H' represents the height of the output, W' represents the width of the output, H represents the height of the input, W represents the width of the input, C represents the number of input channels, N represents the number of output channels, and K represents the kernel size of the convolution.

[0050] Step 4: Build lightweight components.

[0051] To meet the need for identifying abnormal behavior in charging stations, this invention proposes to construct a lightweight abnormal behavior identification component, the structure of which is as follows: Figure 4 As shown, the model includes a 3×3 convolutional layer with a stride of 2, a 2× lightweight convolutional module, a 4× lightweight convolutional module, a first 6× lightweight convolutional module, a second 6× lightweight convolutional module, and an 8× lightweight convolutional module. Pooling layers are added to the 2×, 4×, 6×, and 8× lightweight convolutional modules for undersampling. The outputs of the 4×, 6×, and 8× lightweight convolutional modules are three different scaled feature maps. The main function of the lightweight anomaly detection component is to extract features from the input charging field image. Compared with other model components with the same function, it has fewer FLOPs.

[0052] The lightweight abnormal behavior recognition component proposed in this invention consists of several lightweight convolutional modules. A shrinking module is integrated into the first lightweight convolutional module in each group to enhance the feature extraction capability of the entire group. Furthermore, this invention inserts pooling layers at appropriate locations for undersampling to obtain high-level semantic features, enabling more effective identification of difficult-to-detect abnormal behaviors such as vehicle queuing. Finally, the lightweight abnormal behavior recognition component outputs three different scale feature maps to better identify edge, corner, or texture features in charging site abnormal behavior recognition samples, making the learned features more comprehensive and targeted. Testing shows that the lightweight abnormal behavior recognition component developed in this invention exhibits good performance and operating efficiency.

[0053] Step 5: Replace the backbone of the YOLO architecture with a lightweight abnormal behavior detection component.

[0054] To meet the computational resource and time requirements for identifying abnormal behavior in electric vehicle charging stations, a lightweight abnormal behavior recognition component was used to replace the backbone of the YOLO architecture, resulting in a new algorithm framework adaptable to conditions with limited edge computing resources. Under this framework, better performance can be achieved with fewer FLOPs, reaching the desired effect.

[0055] Step 6: Use the improved algorithm to identify abnormal behavior in electric vehicle charging stations.

[0056] The collected samples were re-input into the improved algorithm, and the training data yielded features related to abnormal behavior identification, such as people falling to the ground, vehicles queuing, and perimeter intrusion. Based on real-time images of the charging site obtained from the charging site monitoring facilities, the algorithm quickly and efficiently identifies and detects abnormal behaviors, and outputs labels for the real-time status of the charging site (normal / abnormal) and the types of abnormal behaviors (people falling to the ground, vehicles queuing, etc.).

[0057] In this invention, a novel lightweight convolutional module is designed using FLOPs as a guide. This module generates more redundant features with less computation. In the intermediate expansion stage, pointwise convolutions are replaced with depthwise convolutions to generate candidate features. The proposed lightweight convolutional module achieves fitting capabilities comparable to 3x3 convolutions, achieving similar results to neural network methods such as CNNs under limited resources. This module can be used to construct a lightweight abnormal behavior recognition component. Finally, this invention uses this component to replace the backbone of the YOLO algorithm, constructing an improved lightweight detection algorithm for YOLO, which achieves better performance in detecting abnormal behavior in charging fields while reducing costs.

[0058] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for identifying abnormal behavior in electric vehicle charging stations. Its key feature is the construction of an identification model in edge computing to automatically identify abnormal behavior in charging fields, including the following steps: The first step is to acquire images related to abnormal behavior at electric vehicle charging stations and organize the images into a sample dataset. The second step is to build the architecture of the recognition model based on the YOLO model structure. The third step involves constructing a lightweight convolutional module to generate features related to abnormal behavior recognition from the input image. The input abnormal behavior recognition image samples are divided into two branches for processing. One branch generates a semi-redundant feature map, called the "skip branch." The other branch, called the "lightweight main branch," generates the remaining half of the redundant feature map through a lightweight network, and then the two are concatenated. The lightweight main branch includes an extension module and a spatial feature processing module. The output of the spatial feature processing module is passed through a 1×1 convolutional layer to output the branch's feature map. The skip branch includes a 1×1 convolutional layer, which outputs the branch's feature map after average pooling. In the main branch, the sample is first extracted for detailed features by average pooling, and then concatenated with the feature map output from the skip branch in the expansion module. Then it is input into the spatial feature processing module for spatial feature extraction. In the spatial feature processing module, the encoding network and the prediction network output spatial features and determine whether undersampling is needed. If so, it is input into a shrinking module and then output after passing through the Mish activation function. Otherwise, it is directly output after passing through the Mish activation function. The encoding network and the prediction network are two-layer recurrent convolutional structures. The output of the Mish activation function is concatenated with the feature map output from the skip branch after passing through a 1×1 convolutional layer to obtain the abnormal behavior feature map. The fourth step involves constructing a lightweight abnormal behavior recognition component to identify the features generated in the third step. This component consists of several lightweight convolutional modules, including a 3×3 convolutional layer with a stride of 2, a 2× lightweight convolutional module, a 4× lightweight convolutional module, a first 6× lightweight convolutional module, a second 6× lightweight convolutional module, and an 8× lightweight convolutional module. Pooling layers are added to the 2×, 4×, 6×, and 8× lightweight convolutional modules for undersampling. The outputs of the 4×, 6×, and 8× lightweight convolutional modules are used as feature maps with three different scales. The fifth step is to replace the main part of the YOLO framework with a lightweight abnormal behavior recognition component to obtain the final recognition model. The sixth step involves training a recognition model using the sample dataset to identify abnormal behavior at electric vehicle charging stations.

2. The method for identifying abnormal behavior in electric vehicle charging stations according to claim 1, characterized in that the extension module includes multi-layer deep convolution operations and the activation function Mish, which expands the feature map of the skipped branch using deep convolution with an expansion ratio of 2, concatenates it with the detailed features of the lightweight main branch, and then outputs it after passing through the Mish activation function.

Images