A method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning

Abstract

A method for identifying abnormal behaviors in electric vehicle charging stations based on few-shot learning is proposed. This method extracts knowledge information from common abnormal behaviors in electric vehicle charging stations through meta-learning and combines it with information on rare abnormal behaviors. Subsequently, a neural dictionary is constructed to query the characteristics of rare abnormal behaviors in charging stations, enabling effective identification and classification of these behaviors even with limited samples. This invention focuses on identifying rare abnormal behaviors in charging stations, such as people falling to the ground or smoke alarms. By defining a sample set and parameter information, introducing meta-learning concepts, a neural dictionary is constructed to query the characteristics of rare abnormal behaviors in electric vehicle charging stations, and a rare abnormal behavior identification model is built to achieve accurate and effective identification of rare abnormal behaviors in electric vehicle charging stations.

Description

Technical Field

[0001] This invention belongs to the field of computer technology and relates to image recognition. It is a method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning. Background Technology

[0002] Electric vehicle charging stations often exhibit various abnormal behaviors, such as vehicle queues and people falling to the ground, requiring monitoring. Current technologies utilize neural networks based on image classification algorithms to identify these abnormal behaviors. Traditional image classification algorithms primarily rely on manually constructed features, such as Speed-Up Robust Features (SURF), Scale-Invariant Feature Transform (SIFT), and Histogram of Oriented Gradients (HOG). With the development of deep learning, Convolutional Neural Networks (CNNs) have also been applied to image recognition. However, for abnormal behaviors at charging stations, traditional neural networks have limitations due to the diverse range of abnormal behavior images involved. Their performance in practical applications is not ideal; models trained on the dataset often fail to recognize new image categories, requiring retraining after different abnormal behaviors occur. Furthermore, different abnormal behaviors have varying probabilities of occurrence, leading to class imbalance issues in the dataset, such as rare occurrences of specific behaviors. Even data augmentation or supervised learning cannot solve these training challenges.

[0003] In practical applications, while traditional neural networks can play a role in image recognition, underfitting can occur when the sample dataset is too small or when there is an imbalance in the number of categories, such as rare special behaviors like people falling down or smoke alarms. During the recognition process, an insufficient dataset can lead to a large difference between the overall data and the true distribution, resulting in low model classification accuracy and posing challenges to the model in practical applications. Even data augmentation or supervised learning are difficult to effectively solve these problems. Summary of the Invention

[0004] The problem this invention aims to solve is that existing neural network technology based on image classification has limited ability to identify abnormal behaviors in electric vehicle charging stations, especially for rare abnormal behaviors that occur infrequently in charging stations. Due to the small sample size problem, the classification accuracy of the trained abnormal behavior image classifier is low.

[0005] The technical solution of this invention is as follows: a method for identifying abnormal behavior in electric vehicle charging stations based on few-sample learning. Abnormal behavior in electric vehicle charging stations is divided into common abnormal behavior and rare abnormal behavior, where the frequency of rare abnormal behavior is lower than that of common abnormal behavior. Rare abnormal behavior is used as a small sample, and common abnormal behavior is used as a large sample to construct and train an abnormal behavior identification model for electric vehicle charging stations, thereby achieving automatic detection of abnormal behavior images at charging stations. The method includes the following steps:

[0006] Step 1: Collect image samples related to abnormal behavior in the charging field, transform the information of small samples through adversarial neural networks to increase the number of small sample data instances, and define the sample set and parameter information for the collected abnormal behavior images to obtain the abnormal behavior dataset.

[0007] Step 2: Apply meta-learning methods to few-shot learning, build a common abnormal behavior recognition model for electric vehicle charging stations based on neural networks, extract features from common abnormal behavior images, combine rare abnormal behavior images that need to be identified and predicted, and build a classification and recognition model for rare abnormal behavior learning in electric vehicle charging stations.

[0008] Step 3: Construct a neural dictionary based on meta-learning to query rare abnormal behavior features in electric vehicle charging stations. The neural dictionary contains various behavior recognition features in the charging station, as well as shared features of common abnormal behaviors and rare abnormal behaviors. Step 4: Construct a rare abnormal behavior recognition model for electric vehicle charging stations using meta-learning and the neural dictionary.

[0009] Step 5: Utilize the rare abnormal behavior recognition model of electric vehicle charging sites to automatically identify abnormal behaviors in the collected images of electric vehicle charging sites.

[0010] Furthermore, in step 1, image samples of abnormal behavior at charging stations are collected. For small sample information, a generative adversarial neural network (GAN) is used to expand the sample size: the small sample image information is divided into various factors, including key factors and environmental factors. Key factors refer to the objects where the abnormal behavior occurs. These factors are input into the GAN, and by performing feature transformations on the environmental factors of the image information, new images of rare abnormal behavior at electric vehicle charging stations are generated after manual processing. This increases the sample instances of rare abnormal behavior at electric vehicle charging stations.

[0011] Then, an abnormal behavior dataset is constructed using image samples containing both common and rare abnormal behaviors in the charging station. This dataset assumes N types of abnormal behaviors in the charging station that need to be identified and monitored, with K sample instances for each type. The training set is defined as follows: in It learns by optimizing sample instances, y i ∈{0,1} N It is its label; similarly, the sample test set is defined as... Among them, test set samples Distribution of training set samples same.

[0012] Furthermore, in step 2, during the construction and training of the classification and recognition model for rare abnormal behaviors in electric vehicle charging stations, a training set for rare abnormal behaviors in electric vehicle charging stations is established. The function f in the equation aims to minimize its expected error value, as shown in the formula below:

[0013]

[0014] Among them, function f is a training set of rare and anomalous behaviors of electric vehicle charging sites. and a set of class descriptors containing information about common abnormal behaviors They are combined as input information and used to build a classifier. f is instantiated as an embedded function. and a linear classifier f(x i )=φ(x i ) T Θ, the class descriptor set of classifier Θ is a set of vectors summarizing the features of the target class, for a known class. We define class descriptors as the set of weight vectors in classifier Θ, and the loss function l(·,·) is used to represent the difference between the prediction and the true label. and These represent datasets with common abnormal behavior and datasets with rare abnormal behavior, respectively.

[0015] Furthermore, in step 3, the neural dictionary is defined as a pair of learnable key information embedded in the model, where each key information is associated with a set of neural foundations designed to encode shared features that make up the recognition models for common and rare abnormal behaviors in charging fields. The neural foundations consist of two sets of elements:

[0016] B = B share ∪B specific (2)

[0017] Among them, B share It contains a series of elements {b1, b2, ..., b} for learning. |Bshare|}, When combining different classification and recognition tasks, it represents common knowledge information in the neural dictionary, B. specific ={P c |c∈S}∪Θ S B specific As special knowledge information in the neural dictionary, namely the rare anomalous behavior matrix P of electric vehicle charging sites c and the class descriptor set Θ containing information about common abnormal behaviors SThe knowledge information was then analyzed, and the co-correlation coefficient α of the knowledge information in the neural dictionary was obtained by measuring the matching degree between common knowledge information and specific knowledge information in the neural dictionary. c α c The calculation method is as follows:

[0018]

[0019] Wherein, the synergy coefficient α c After standardizing the matching degree of the acquired knowledge information, it is used in the subsequent classification and recognition processing of the model.

[0020] Furthermore, in step 4, a rare abnormal behavior recognition model for electric vehicle charging stations is constructed using a neural dictionary based on cross-category common features. This model is then combined with a common abnormal behavior recognition model for electric vehicle charging stations via end-to-end transmission. Samples are taken from the common abnormal behavior recognition information, and the known identified objects within these sample instances are assumed to be unknown categories. This basic knowledge of rare abnormal behavior recognition is incorporated into the neural dictionary. Subsequently, knowledge information is retrieved from the rare abnormal behavior information by querying the neural dictionary and applied to the constructed rare abnormal behavior recognition model for electric vehicle charging stations. Then, by combining the common abnormal behavior information of electric vehicle charging stations and the rare abnormal behavior information of electric vehicle charging stations, the expected errors of the common and rare abnormal behaviors are compared. Finally, by re-invoking the parameter information from the common behavior recognition model for electric vehicle charging stations, the prediction results of the rare abnormal behavior recognition model for electric vehicle charging stations are calibrated, resulting in improved prediction performance of the classification and recognition model.

[0021] This invention studies the abnormal behavior of electric vehicle charging stations from the perspective of "few-shot learning". Through meta-learning methods, knowledge information is extracted from common abnormal behaviors of electric vehicle charging stations and combined with information on rare abnormal behaviors. Subsequently, a neural dictionary that can be used to query the characteristics of rare abnormal behaviors of electric vehicle charging stations is constructed, and the knowledge information output by the dictionary is used for the identification of rare abnormal behaviors of electric vehicle charging stations, so as to effectively identify and classify rare abnormal behaviors of electric vehicle charging stations under the condition of small sample size. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention.

[0023] Figure 2 This is a flowchart of the abnormal behavior identification process for electric vehicle charging sites in this invention.

[0024] Figure 3This is a schematic diagram of a neural dictionary built based on the meta-learning concept, which can be used to query rare and abnormal behavioral features of electric vehicle charging sites. Detailed Implementation

[0025] This invention categorizes abnormal behaviors in electric vehicle charging stations into common and rare abnormal behaviors. Rare abnormal behaviors occur less frequently than common abnormal behaviors, and are used as a small sample, such as people falling to the ground or smoke alarms. Common abnormal behaviors are used as a large sample, such as charging gun misalignment or vehicles queuing for charging. The specific categorization can be determined based on the frequency of the abnormal behaviors and detection requirements. After defining the sample set, an abnormal behavior recognition model for electric vehicle charging stations is constructed and trained to achieve automatic detection of abnormal behavior images at charging stations. The process includes the following steps:

[0026] Step 1: Collect image samples related to abnormal behavior in the charging field, transform the information of small samples through adversarial neural networks to increase the number of small sample data instances, and define the sample set and parameter information for the collected abnormal behavior images to obtain the abnormal behavior dataset.

[0027] Step 2: Apply meta-learning methods to few-shot learning, build a common abnormal behavior recognition model for electric vehicle charging stations based on neural networks, extract features from common abnormal behavior images, combine rare abnormal behavior images that need to be identified and predicted, and build a classification and recognition model for rare abnormal behavior learning in electric vehicle charging stations.

[0028] Step 3: Construct a neural dictionary based on meta-learning to query rare abnormal behavior features in electric vehicle charging stations. The neural dictionary contains various behavior recognition features in the charging station, as well as shared features of common abnormal behaviors and rare abnormal behaviors. Step 4: Construct a rare abnormal behavior recognition model for electric vehicle charging stations using meta-learning and the neural dictionary.

[0029] Step 5: Utilize the rare abnormal behavior recognition model of electric vehicle charging sites to automatically identify abnormal behaviors in the collected images of electric vehicle charging sites.

[0030] The implementation of this invention is described in detail below.

[0031] In step 1, the charging station's built-in cameras capture real-time charging station status information, extracting video frames related to abnormal behavior as samples. For small sample information—images of rare abnormal behavior of electric vehicle charging vehicles—the sample size can be partially expanded using a generative adversarial neural network (GAN Awesome Applications). Specifically, this invention categorizes image information from electric vehicle charging stations into various factors, such as key factors including information about vehicles, charging piles, and people; and environmental factors including weather and lighting. These are input into the adversarial neural network, which modifies the environmental factors of the image information, such as lighting and weather, without altering the key factors affecting abnormal behavior identification. New images of rare abnormal behavior in electric vehicle charging stations are generated after manual processing, thereby increasing the sample instances of rare abnormal behavior in electric vehicle charging stations. However, simply using an adversarial neural network can only improve the model's accuracy in identifying abnormal behavior under different environments; it cannot improve the breadth and depth of the model's learning of key features of rare abnormal behavior. To address this issue, this invention uses image samples containing both common and rare abnormal behaviors in charging stations as a test set to test the model's recognition performance. Suppose there are N types of abnormal behaviors in a charging station, such as people falling to the ground, smoke alarms, etc., that need to be identified and monitored, with K sample instances for each type of abnormal behavior. Let the sample training set be defined as follows. in It learns by optimizing sample instances, y i ∈{0,1} N It is its label. Similarly, the sample test set is defined as... Among them, test set samples Distribution of training set samples same.

[0032] In step 2, due to limitations in camera shooting angles and light obstruction in real-world electric vehicle charging stations, the collection of data on certain categories of abnormal charging station behaviors is often very limited. In this "small sample" scenario, effective learning methods are often required for refined identification of abnormal charging station behaviors. Meta-learning methods, as an important learning method in neural networks, can achieve the same level of model classification accuracy as training with massive amounts of data through training with a small amount of data. The main idea is to update the existing classification model parameters by learning from existing knowledge, such as K-shot N-way task parameters, and after learning from dataset preprocessing, model initialization parameter transformations, and model network structure selection. This allows the model to be used to solve different small sample classification learning tasks.

[0033] This invention integrates meta-learning concepts into the learning of abnormal behaviors in charging stations. A common abnormal behavior recognition model for electric vehicle charging stations is constructed based on a neural network. Features are extracted from collected images of common abnormal behaviors in electric vehicle charging stations (this is existing technology and will not be detailed further). A classification and recognition model for "learning rare abnormal behaviors in electric vehicle charging stations" is constructed by combining the collected images of abnormal behaviors with images of rare abnormal behaviors that need to be identified and predicted. In this model, a training set for rare abnormal behaviors in electric vehicle charging stations is defined. The function f in the equation aims to minimize its expected error value, as shown in the formula below:

[0034]

[0035] Among them, function f is a training set of rare and anomalous behaviors of electric vehicle charging sites. and a set of class descriptors containing information about common abnormal behaviors They are combined as input information and used to build a classifier. Here, f is usually instantiated as an embedded function. and a linear classifier f(x i )=φ(x i ) T Θ. The classifier Θ's "class descriptor" set is a set of vectors summarizing the features of the target class, for a known class. Set the class descriptor as the classifier Θ s A set of weight vectors. The loss function l(·,·) is used to represent the difference between the prediction and the true label. and These represent datasets with common abnormal behavior and datasets with rare abnormal behavior, respectively.

[0036] In step 3, the neural dictionary is a module composed of a series of knowledge information B. Based on the similarity between these knowledge information items, this information is transformed into a set of weighted information, which serves as input information for the rare abnormal behavior recognition model in electric vehicle charging stations. For example... Figure 2 As shown, this invention defines a neural dictionary as a pair of learnable key information embedded in a model, where each key information is associated with a set of neural foundations designed to encode shared features that constitute the recognition models for common and rare abnormal behaviors in charging fields. Formally, the neural foundations consist of two sets of elements:

[0037] B = B share ∪B specific (2)

[0038] Among them, Bshare It contains a range of elements that can be learned. When combining different classification and recognition tasks, it represents common knowledge information that can be obtained from the neural dictionary. specific ={P c |c∈S}∪Θ S B specific As a special type of knowledge information that can be obtained from the neural dictionary, the typical rare abnormal behavior matrix P of electric vehicle charging sites c and the "class descriptor" set Θ S The invention then obtains the co-operation coefficient α of knowledge information in the neural dictionary by measuring the matching degree between common knowledge information and specific knowledge information in the neural dictionary. c α c The calculation method is as follows:

[0039]

[0040] Wherein, the synergy coefficient α c After standardizing the matching degree of the acquired knowledge information, it is used in the subsequent classification and recognition processing of the model.

[0041] In step 4, based on the above steps, a rare abnormal behavior identification model for electric vehicle charging stations is constructed. This model can be built using a neural dictionary based on cross-category common features. The process for identifying rare abnormal behaviors in electric vehicle charging stations according to this invention is as follows: Figure 3 As shown, the rare abnormal behavior recognition model for electric vehicle charging stations and the common abnormal behavior recognition model for electric vehicle charging stations capable of handling massive amounts of data are combined through end-to-end transmission. A series of massive data, namely the common abnormal behavior recognition information for charging stations, are sampled, and the general concepts, such as the identification of people, vehicles, and charging piles, are extracted from these sample instances and incorporated into the neural dictionary as the basic knowledge of rare abnormal behavior recognition for electric vehicle charging stations, representing unknown categories. Subsequently, knowledge information, such as people falling to the ground, charging station smoke alarms, and perimeter intrusion of key equipment, is obtained from the rare abnormal behavior information for electric vehicle charging stations by querying the neural dictionary, and applied to the constructed rare abnormal behavior recognition model for electric vehicle charging stations. Then, using formula (1), the expected errors of common and rare abnormal behaviors are compared by combining the common abnormal behavior and rare abnormal behavior information for electric vehicle charging stations. By re-invoking the parameter information in the common behavior recognition model for electric vehicle charging stations, the prediction results of the rare abnormal behavior recognition model for electric vehicle charging stations are calibrated, making the prediction performance of the classification recognition model better.

[0042] In step 5, the electric vehicle rare abnormal behavior recognition model from step 4 is applied to charging stations to observe its recognition efficiency and accuracy. Based on the spatial distribution of different stations, edge computing resources, and the model's performance in the charging station, the recognition model parameters are continuously optimized so that while it can quickly and efficiently recognize common abnormal behaviors, it can ultimately achieve accurate recognition of rare abnormal behaviors such as people falling to the ground or smoke alarms.

[0043] This invention proposes an image classification method for abnormal behavior based on few-shot learning. By defining the sample set and parameter information, introducing the meta-learning concept, constructing a neural dictionary based on the meta-learning concept that can be used to query the features of rare abnormal behaviors in electric vehicle charging stations, and constructing a rare abnormal behavior recognition model for electric vehicle charging stations, the method achieves accurate and effective recognition of rare abnormal behaviors in electric vehicle charging stations.

Claims

1. A method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning, characterized by: Abnormal behaviors at electric vehicle charging stations are categorized into common and rare abnormal behaviors, with rare behaviors occurring less frequently than common abnormal behaviors. Rare abnormal behaviors are used as a small sample, and common abnormal behaviors as a large sample. An abnormal behavior recognition model for electric vehicle charging stations is constructed and trained to achieve automatic detection of abnormal behavior images at charging stations. The process includes the following steps: Step 1: Collect image samples related to abnormal behavior in the charging field, transform the information of small samples through adversarial neural networks to increase the number of small sample data instances, and define the sample set and parameter information for the collected abnormal behavior images to obtain the abnormal behavior dataset. Step 2: Apply meta-learning methods to few-shot learning, build a common abnormal behavior recognition model for electric vehicle charging stations based on neural networks, extract features from common abnormal behavior images, combine rare abnormal behavior images that need to be identified and predicted, and build a classification and recognition model for rare abnormal behavior learning in electric vehicle charging stations. Step 3: Construct a neural dictionary based on the meta-learning concept for querying rare abnormal behavior features in electric vehicle charging stations. The neural dictionary contains various behavior recognition features in the charging station, as well as shared features of common abnormal behaviors and rare abnormal behaviors. Step 4: Construct a rare abnormal behavior recognition model for electric vehicle charging stations using meta-learning and neural dictionaries; Step 5: Utilize the rare abnormal behavior recognition model of electric vehicle charging sites to automatically identify abnormal behaviors in the collected images of electric vehicle charging sites.

2. The method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning according to claim 1, characterized in that in step 1, image samples of abnormal behavior in charging stations are collected. For the few-shot information, the sample size is expanded using a generative adversarial neural network: the image information of the few samples is divided into various factors, including key factors and environmental factors. Key factors refer to the objects where the abnormal behavior occurs. These factors are input into the adversarial neural network. By performing feature changes on the environmental factors of the image information, new images of rare abnormal behavior in electric vehicle charging stations are generated after manual processing, thereby increasing the sample instances of rare abnormal behavior in electric vehicle charging stations. Then, an abnormal behavior dataset is constructed using image samples containing both common and rare abnormal behaviors in the charging station. This dataset assumes N types of abnormal behaviors in the charging station that need to be identified and monitored, with K sample instances for each type. The training set is defined as follows: in It learns by optimizing sample instances, y i ∈{0,1} N It is its label; similarly, the sample test set is defined as... in, Test set samples Distribution of training set samples same.

3. The method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning according to claim 1, characterized in that: In step 2, during the construction and training of the classification and recognition model for rare abnormal behaviors in electric vehicle charging stations, a training set for rare abnormal behaviors in electric vehicle charging stations is established. The function f in the equation aims to minimize its expected error value, as shown in the formula below: Among them, function f is a training set of rare and anomalous behaviors of electric vehicle charging sites. and a set of class descriptors containing information about common abnormal behaviors They are combined as input information and used to build a classifier. f is instantiated as an embedded function. and a linear classifier f(x i )=φ(x i ) T Θ, the class descriptor set of classifier Θ is a set of vectors summarizing the features of the target class. For a known class S, the class descriptors are defined as the set of weight vectors in classifier Θ. The loss function l(·,·) is used to represent the difference between the prediction and the true label. and These represent datasets with common abnormal behavior and datasets with rare abnormal behavior, respectively.

4. The method for identifying abnormal behavior in electric vehicle charging stations based on few-shot learning according to claim 1, characterized in that in step 3, the neural dictionary is defined as a pair of learnable key information embedded in the model, wherein each key information is associated with a set of neural bases, which are designed to encode shared features constituting the identification models for common and rare abnormal behaviors in charging stations. The neural bases contain two sets of elements: B=B share ∪B specific (2) in, B share It contains a range of learning elements. When combining different classification and recognition tasks, it represents common knowledge information in the neural dictionary, B. specific ={P c |c∈S}∪Θ S B specific As special knowledge information in the neural dictionary, namely the rare anomalous behavior matrix P of electric vehicle charging sites c and the class descriptor set Θ containing information about common abnormal behaviors S The knowledge information was then analyzed, and the co-correlation coefficient α of the knowledge information in the neural dictionary was obtained by measuring the matching degree between common knowledge information and specific knowledge information in the neural dictionary. c α c The calculation method is as follows: Wherein, the synergy coefficient α c After standardizing the matching degree of the acquired knowledge information, it is used in the subsequent classification and recognition processing of the model.

5. The method for identifying abnormal behavior in electric vehicle charging stations based on few-sample learning according to claim 1, characterized in that in step 4, a rare abnormal behavior identification model for electric vehicle charging stations is constructed by using a neural dictionary based on cross-category common features. The rare abnormal behavior identification model and the common abnormal behavior identification model for electric vehicle charging stations are combined through end-to-end transmission. Samples are taken from the common abnormal behavior identification information of the charging stations, and the known identification objects among these sample instances are assumed to be unknown categories. That is, the basic knowledge of rare abnormal behavior identification is incorporated into the neural dictionary. Subsequently, knowledge information is obtained from the rare abnormal behavior information by querying the neural dictionary and applied to the constructed rare abnormal behavior identification model for electric vehicle charging stations. Then, by combining the common abnormal behavior of electric vehicle charging stations and the rare abnormal behavior information of electric vehicle charging stations, the expected errors of the common and rare abnormal behaviors are compared. By re-calling the parameter information in the common behavior identification model of electric vehicle charging stations, the prediction results of the rare abnormal behavior identification model for electric vehicle charging stations are calibrated, thereby improving the prediction performance of the classification and identification model.

Images