New energy station illegal behavior intelligent identification system and method based on machine vision

Abstract

The invention relates to the technical field of deep learning, in particular to a new energy station illegal behavior intelligent recognition system and method based on machine vision. The method comprises the steps of obtaining new energy station image data, performing preprocessing and data enhancement, and creating a behavior image data set; a CNN-based behavior recognition model is constructed, and training is carried out through the data set; meanwhile, a clustering algorithm based on comparative learning is constructed for judging whether the unknown behavior belongs to a new violation type or not; if a new violation type is found, transmitting a sample of the new violation type to a behavior recognition model for incremental learning, and expanding the number of the sample; and through a knowledge distillation technology, knowledge of the trained CNN behavior recognition model is migrated to a miniaturized student model, and lightweight deployment is realized. According to the method, the illegal behaviors of the new energy station can be automatically identified, new illegal behavior types can be continuously found, the model is dynamically updated, and the adaptive capacity and the real-time performance of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and specifically to a system and method for intelligently identifying illegal behaviors at new energy stations based on machine vision. Background Art

[0002] With the rapid development of the new energy industry, the number of new energy stations is increasing. New energy stations are usually built in remote areas with sparse personnel and weak security forces. They face problems such as difficult security management and frequent violations. The traditional manual inspection method has defects such as untimely response, limited inspection scope, and easy fatigue omissions. In order to ensure the safe and stable operation of new energy stations, it is urgent to introduce advanced intelligent security technology.

[0003] At present, machine vision technology has been widely used in the field of smart security. Traditional machine vision methods mainly rely on manually designed features, have poor generalization ability, and are difficult to cope with complex and changing scenes. In recent years, deep learning, especially convolutional neural networks (CNN), has made breakthrough progress in the field of image recognition. Applying CNN to video surveillance analysis can realize automatic detection, tracking and recognition of targets such as people and vehicles in the picture, and further analyze the behavior patterns of the targets, detect abnormal behaviors in time and issue warnings.

[0004] The complexity and variability of the real environment make it difficult for pre-trained deep learning models to cover all situations, and the generalization ability is insufficient when facing new unknown situations. Incremental learning allows continuous learning of new knowledge on the basis of retaining the knowledge already learned, and continuously expands and updates the model, so that the intelligent system has the ability to continuously evolve. Currently, commonly used incremental learning methods include fine-tuning, knowledge distillation, parameter configuration network, etc. Although deep learning has shown superior performance in intelligent security, the huge model size and computing power restrict its real-time deployment and application on the edge side.

[0005] Although smart security technology has made great progress, there are still some problems in its application in the specific scenario of new energy stations: lack of targeted behavior definition. Unlike general public places, the behavior of people inside new energy stations is relatively simple; insufficient environmental adaptability. Affected by factors such as climate and light, the internal environment of new energy stations changes greatly, and the robustness of existing general algorithm models needs to be improved to adapt to the actual application needs of this field; lack of ability to detect new types of violations. Faced with the emergence of new types of violations, the update and iteration mechanism of existing models is still imperfect, and the dynamic response capability needs to be strengthened; the degree of lightweight needs to be improved. Considering that new energy stations are located in remote areas and have limited network conditions, it is necessary to further compress the model size, reduce computing resource consumption, and achieve efficient localized deployment.

[0006] In order to solve the above problems, the present invention proposes a system and method for intelligently identifying violations of new energy stations based on machine vision. Summary of the invention

[0007] To achieve the above purpose, the present invention provides a system and method for intelligently identifying illegal behaviors of new energy stations based on machine vision. The specific technical solutions are as follows:

[0008] Acquire image data from new energy stations, preprocess and enhance the acquired data, and create a behavioral image dataset;

[0009] Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset;

[0010] Construct a clustering algorithm based on contrastive learning. When unknown behaviors are obtained in the image data of new energy stations, the clustering algorithm is used to determine whether the unknown behaviors belong to new types of violations.

[0011] If the clustering algorithm determines that the unknown behavior is a new type of violation, the new type of violation is transmitted to the behavior recognition model as a data sample, and the number of samples of the new type of violation is expanded and inserted into the original behavior image data set, so that the behavior recognition model can perform incremental learning;

[0012] When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

[0013] Preferably, surveillance cameras and patrol drones are deployed in the new energy station to obtain video images in the new energy station, and frame extraction is performed on the obtained video images, and one frame of image is extracted as image data at a specific interval;

[0014] Preprocessing the acquired new energy station image data, wherein the preprocessing includes image size normalization and image denoising;

[0015] Perform data enhancement on the pre-processed image data in the new energy station to enhance the data features of the image data;

[0016] Manually label samples of illegal behavior images, non-illegal behavior images, and unknown behavior images in existing new energy stations; randomly divide the labeled image samples into training sets, validation sets, and test sets according to set ratios.

[0017] Preferably, a convolutional neural network (CNN) model is used as the basic network of the behavior recognition model, and the initial parameters of the CNN model are configured. An adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a feature vector of a fixed size.

[0018] Add a Softmax activation function after the fully connected layer in the CNN model to map the feature vector to the probability distribution of the violation category;

[0019] Define the cross entropy loss function of the CNN model to measure the difference between the model prediction results and the true labels;

[0020] Select Adam optimizer, set initial learning rate and batch size;

[0021] Load the training set data and input the image data and labels into the model for training;

[0022] At the end of each training cycle, the validation set is used to evaluate the performance of the model and the hyperparameters of the model are adjusted based on the validation results.

[0023] The behavior recognition model training steps include taking the training set as the input of the CNN model, the CNN model taking the proportion of illegal behavior image samples, non-illegal behavior image samples and unknown behavior image samples as output, selecting cross entropy as the loss function, using the Adam optimizer for training, and stopping training when the accuracy on the validation set no longer improves.

[0024] Preferably, the SimCLR algorithm is used to perform comparative learning on the image data of the illegal behavior and the non-illegal behavior, and the comparative learning step includes:

[0025] Perform two random data augmentations on the violation behavior images and non-violation behavior images respectively to obtain a pair of positive samples;

[0026] Use the pre-trained residual network ResNet-50 model to extract feature representations of violation images and non-violation images;

[0027] After representing the image data features, a projection head is added to map the features into a low-dimensional space;

[0028] Calculate the contrast loss for the feature representation of the positive sample pair, maximize the similarity of the positive sample pair, and minimize the similarity of the negative sample pair;

[0029] Train the residual network ResNet-50 model and projection head using the Adam optimizer and contrastive loss function until the contrastive loss function converges;

[0030] Remove the projection head from the trained residual network ResNet-50 model, and use the residual network ResNet-50 model to extract feature representations of violation images and non-violation images;

[0031] The feature representation of the violation image and the feature representation of the non-violation image are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes cluster centers of different types of violations.

[0032] Preferably, for the image data collected in the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When the image data is identified to contain illegal behavior, the illegal behavior is output; when an unknown behavior is identified, an unknown behavior image is output;

[0033] The unknown behavior image features are extracted by training the residual network ResNet-50 model, and the Euclidean distance between the unknown behavior features and all cluster centers is calculated;

[0034] Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into a violation or non-violation behavior;

[0035] If the distance between the unknown behavior feature and any cluster center is less than the preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs;

[0036] When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center in the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type.

[0037] Preferably, allowing the behavior recognition model to perform incremental learning includes, when a new type of violation behavior is discovered through comparison of the clustering algorithm, adding sample images of the new type of violation behavior to the original behavior image data set, and performing data enhancement on the newly added sample images to expand the number of samples of the new category;

[0038] In the output layer of the original violation recognition model, add a new neuron corresponding to the new violation category;

[0039] Using the expanded training set, the violation identification model is updated and trained. The steps of the update training are as follows:

[0040] Update the parameters of the last convolutional block and the fully connected layer in the CNN model, and reset the learning rate of the CNN model; use the Adam optimizer and the cross entropy loss function for training until the performance of the model on the validation set does not change;

[0041] After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of new violation types are extracted, the sample image features of the new violation types are added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

[0042] Preferably, knowledge distillation is performed on the trained behavior recognition model based on the teacher-student network to achieve compression and lightweighting of the original model:

[0043] Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model;

[0044] The step of knowledge distillation includes: applying a Softmax activation function to the outputs of the teacher model and the student model to obtain soft labels;

[0045] Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the prediction results of the student model and the true labels as the classification loss; sum the distillation loss and classification loss to get the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges;

[0046] The performance of the student model is evaluated on the test set, and the knowledge distillation process is completed; the lightweight student model is deployed on the edge device for real-time violation identification and alerting.

[0047] A machine vision-based intelligent identification system for illegal behaviors of new energy stations, which is used to implement the machine vision-based intelligent identification method for illegal behaviors of new energy stations, including: a data acquisition module, a behavior recognition module, a new behavior discrimination module, an incremental learning module and a knowledge distillation module;

[0048] The data acquisition module is used to acquire image data in the new energy station, preprocess and enhance the acquired data, and create a behavior image data set;

[0049] The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model, and train the behavior recognition model through a behavior image dataset;

[0050] The new behavior identification module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of illegal behavior;

[0051] The incremental learning module, when the clustering algorithm determines the unknown behavior as a new type of violation, transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image data set, so that the behavior recognition model can perform incremental learning;

[0052] The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through the knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

[0053] An electronic device comprises: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the intelligent identification method for illegal behavior of new energy stations based on machine vision by calling the computer program stored in the memory.

[0054] A storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method for intelligently identifying illegal behaviors of new energy stations based on machine vision.

[0055] Beneficial effects of the present invention: By acquiring actual image data and performing preprocessing and data enhancement, the present invention can construct a behavioral image dataset that is highly compatible with the application scenario. High-quality and diverse datasets are the basis for training high-performance behavior recognition models, which helps to improve the generalization ability and robustness of the model.

[0056] The present invention applies CNN to behavior recognition, and can learn behavior patterns end-to-end from the original image, without the need for manual feature design, thus simplifying the development process. Through training, the CNN model can learn the discriminative features for identifying different types of behaviors, thus forming a powerful behavior recognition capability.

[0057] The present invention introduces a clustering algorithm of contrastive learning to discover new types of illegal behaviors and expand the existing behavior type system. This unsupervised automatic discovery mechanism can improve the system's adaptability to new situations and reduce the limitations of manual definition.

[0058] Through incremental learning, the behavior recognition model can be updated and iterated in real time, continuously learn new knowledge, and expand the recognition categories. This dynamic update mechanism can help the model quickly respond to environmental changes, actively adapt to new illegal behavior patterns, and improve the real-time performance and accuracy of the system.

[0059] Through knowledge distillation, the present invention can transfer the knowledge of complex models to miniaturized student models. The distilled student model can significantly reduce the amount of calculation, realize lightweight deployment of behavior recognition algorithms, improve the real-time processing capability of the system, and facilitate application in edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A flow chart of the method for intelligently identifying illegal behaviors of new energy stations based on machine vision provided by the present invention;

[0061] Figure 2 This is a structural diagram of the intelligent identification system for illegal behaviors of new energy stations based on machine vision provided by the present invention;

[0062] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0063] Figure 4 It is a schematic diagram of the storage medium structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to better understand the present invention, a more detailed description will be made of various aspects of the present invention with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0065] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The accompanying drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in the present invention, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0066] It should also be understood that expressions such as "comprises", "including", "having", "includes" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present invention, "may" is used to mean "one or more embodiments of the present invention". And, the term "exemplary" is intended to refer to an example or illustration.

[0067] Unless otherwise defined, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which the present invention belongs. It should also be understood that unless there is a clear explanation in the present invention, the words defined in the commonly used dictionary should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0068] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0069] Example 1

[0070] Reference Figure 1 , which is the first embodiment of the present invention, provides an intelligent identification method for illegal behaviors in new energy stations based on machine vision.

[0071] S1: Acquire image data from new energy stations, preprocess and enhance the acquired data, and create a behavioral image dataset.

[0072] Surveillance cameras and patrol drones are deployed in the new energy stations to obtain video images in the new energy stations, and the obtained video images are processed by frame extraction, and one frame of image is extracted as image data at a specific time interval.

[0073] The acquired new energy station image data is preprocessed, and the preprocessing includes image size normalization and image denoising.

[0074] The image size normalization includes scaling the image to a uniform size (eg, 224×224 pixels) to facilitate subsequent feature extraction and model input.

[0075] The image denoising includes using median filtering and bilateral filtering methods to remove noise in the image and improve image quality.

[0076] The pre-processed image data in the new energy station is enhanced to enhance the data features of the image data.

[0077] The data enhancement includes using histogram equalization and contrast adjustment methods to enhance the visual effect of the image and highlight key information in the image data.

[0078] Manually label the samples D of illegal behavior images, non-illegal behavior images and unknown behavior images in the existing new energy stations, D = {P j ,y j}, j = 1, 2, ..., n, y j =0, 1, 2, ..., K, where P j is the jth image sample, n is the number of image samples, y j is the label of the jth image sample, K is the number of image labels; y j = 0 means that the label of the image sample is a non-violation image, y j =1, ..., K-1, it means that the labels of the image samples are different types of violation images; j =K, it means that the label of the image sample is an unknown behavior image.

[0079] The labeled image samples are randomly divided into training set, validation set and test set according to the set ratio.

[0080] By deploying surveillance cameras and drones, video image data in new energy stations can be collected from all directions and angles, providing a data basis for subsequent behavior recognition; pre-processing operations such as size normalization and image denoising on the original image data can eliminate irrelevant interference in the image, improve image quality, and prepare for feature extraction; expanding training samples through data enhancement technology can improve the generalization ability and robustness of the model and reduce the risk of overfitting; the collected image data can be manually annotated and divided into training sets, validation sets, and test sets to provide a standard data set for the training, parameter adjustment, and performance evaluation of the behavior recognition model.

[0081] S2: Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset.

[0082] The convolutional neural network (CNN) model is used as the basic network of the behavior recognition model, and the initial parameters of the CNN model are configured. An adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a feature vector of a fixed size.

[0083] A Softmax activation function is added after the fully connected layer in the CNN model to map the feature vector into the probability distribution of the violation category.

[0084] Construct a convolutional neural network CNN model, and assume that the input of the lth convolutional layer is: The convolution kernel is: Bias is The activation function is f(·), then the output of the l-th convolution layer is:

[0085] X (l) =f(W (l) *X (l-1) +b (l) )

[0086] Among them, * represents the convolution operation. The output of the convolution layer passes through the pooling layer and the fully connected layer, and then passes through the Softmax function to obtain the final classification probability.

[0087] Define the cross entropy loss function of the CNN model to measure the difference between the model prediction results and the true labels; select the Adam optimizer, set the initial learning rate and batch size; load the training set data, and input the image data and labels into the model for training.

[0088] At the end of each training cycle, the performance of the model is evaluated using the validation set and the model's hyperparameters are adjusted based on the validation results.

[0089] The behavior recognition model training steps include taking the training set as the input of the CNN model, the CNN model taking the proportion of illegal behavior image samples, non-illegal behavior image samples and unknown behavior image samples as output, selecting cross entropy as the loss function, using the Adam optimizer for training, and stopping training when the accuracy on the validation set no longer improves.

[0090] The behavior recognition model built based on CNN can automatically extract high-level semantic features in the image, which has stronger expressive power than manually designed features. By adding an adaptive pooling layer and a fully connected layer at the end of the CNN network, feature maps of different sizes can be mapped into feature vectors of fixed length, which is convenient for classifier processing. By connecting the Softmax activation function after the fully connected layer, the feature space is mapped into the probability distribution of the violation category, and intuitive classification results are obtained. The mature classifier training paradigm is used, cross entropy is used as the loss function, and the Adam optimizer is used to train the CNN model, which can efficiently fit the sample data. After each round of iteration, the validation set is used to evaluate the current model performance, and the hyperparameters are adjusted in time to avoid overfitting, so as to improve the generalization performance of the model.

[0091] S3: Construct a clustering algorithm based on contrastive learning. When unknown behaviors are obtained in the image data of new energy stations, the clustering algorithm is used to determine whether the unknown behaviors belong to new types of violations.

[0092] The SimCLR algorithm is used to perform comparative learning on the image data of the violation behavior and the non-violation behavior, and the comparative learning steps include:

[0093] Random data augmentation is performed twice on the violation images and non-violation images respectively to obtain a pair of positive samples.

[0094] The pre-trained residual network ResNet-50 model is used to extract feature representations of violation images and non-violation images.

[0095] After representing the image data features, a projection head is added to map the features into a low-dimensional space.

[0096] The contrast loss is calculated for the feature representation of the positive sample pairs, maximizing the similarity of the positive sample pairs and minimizing the similarity of the negative sample pairs.

[0097] The residual network ResNet-50 model and the projection head are trained using the Adam optimizer and the contrastive loss function until the contrastive loss function converges.

[0098] The projection head is removed from the trained residual network ResNet-50 model, and the residual network ResNet-50 model is used to extract feature representations of violation images and non-violation images.

[0099] The feature representation of the violation image and the feature representation of the non-violation image are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes K-1 cluster centers of different types of violations.

[0100] By introducing contrastive learning, more discriminative image feature representations can be learned by comparing positive samples of the same category, paving the way for downstream clustering tasks; contrastive learning maps features to low-dimensional space, reducing the computational overhead of clustering tasks while retaining important semantic information; using unsupervised clustering algorithms such as K-means, unknown behaviors with similar features can be automatically divided into a cluster to facilitate the analysis of their commonalities; by comparing the features of unknown behaviors with the cluster centers of known violations, it can be determined whether the unknown behavior is close to a known violation, providing a reference for the classification of unknown behaviors; if the features of the unknown behavior are far away from all known cluster centers, it can be preliminarily determined to be a new type of violation, providing a basis for timely updating the violation category.

[0101] S4: If the clustering algorithm determines the unknown behavior as a new type of violation, the new type of violation will be transmitted to the behavior recognition model as a data sample, and data enhancement technology will be used to expand the number of samples of the new type of violation and insert them into the original behavior image data set, allowing the behavior recognition model to perform incremental learning.

[0102] For the image data collected in the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When it is identified that the image data contains illegal behavior, the output is that the illegal behavior is identified; when an unknown behavior is identified, the unknown behavior image is output.

[0103] The residual network ResNet-50 model is trained to extract the output unknown behavior image features and calculate the Euclidean distance between the unknown behavior features and all cluster centers.

[0104] Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into violation behavior or non-violation behavior.

[0105] If the distance between the unknown behavior feature and any cluster center is less than a preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs.

[0106] When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center in the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type.

[0107] Allowing the behavior recognition model to perform incremental learning includes, when a new type of violation is discovered through comparison of the clustering algorithm, adding sample images of the new type of violation to the original behavior image dataset, and performing data enhancement on the newly added sample images to expand the number of samples of the new category.

[0108] In the output layer of the original violation recognition model, a new neuron is added to correspond to the new violation category.

[0109] Using the expanded training set, the violation identification model is updated and trained. The steps of the update training are as follows:

[0110] Update the parameters of the last convolutional block and the fully connected layer in the CNN model, reset the learning rate of the CNN model; use the Adam optimizer and the cross entropy loss function for training until the performance of the model on the validation set does not change.

[0111] After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of new violation types are extracted, the sample image features of the new violation types are added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

[0112] When the clustering algorithm discovers a new type of violation, its samples are added to the original training set in a timely manner, which can expand the category coverage of the behavior recognition model and enhance the model's ability to cope with new situations. Data enhancement is performed on the newly inserted violation category samples, which can alleviate the problem of insufficient sample size of the new category to a certain extent, so that the new category can be learned by the model more quickly. On the basis of the original CNN model structure, only the terminal convolution block and the fully connected layer parameters are updated, which can adapt to the new category while inheriting the original model's knowledge of known categories to achieve incremental learning. By re-clustering and updating the cluster center, the impact of the new violation type on the original category division can be included, making the clustering result more accurate.

[0113] The entire incremental learning process can dynamically expand the types of violations without training from scratch, improving the system's real-time response and sustainable optimization capabilities.

[0114] S5: When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through the knowledge distillation technology.

[0115] Based on the teacher-student network, knowledge distillation is performed on the trained behavior recognition model to achieve compression and lightweight of the original model.

[0116] Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model;

[0117] The step of knowledge distillation includes: applying a Softmax activation function to the outputs of the teacher model and the student model to obtain soft labels;

[0118] Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the prediction results of the student model and the true labels as the classification loss; sum the distillation loss and classification loss to get the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges.

[0119] Calculate the distillation loss in knowledge distillation, assuming that the soft label output of the teacher model is p T , the soft label output of the student model is p S , the temperature coefficient is τ, then the distillation loss based on KL divergence is:

[0120]

[0121] Among them, p T (a) and p S(a) represents the soft label output of the teacher model and the student model for the ath category. The temperature coefficient τ is used to control the smoothness of the soft label, and its value is usually greater than 1.

[0122] The performance of the student model is evaluated on the test set, and the knowledge distillation process is completed; the trained student model is deployed on the edge device for real-time violation identification and alerting.

[0123] Through knowledge distillation, the knowledge of a trained complex teacher model can be compressed into a lighter student model, reducing the storage and computing overhead of the model while maintaining high performance. There is no need to retrain lightweight models, which simplifies the process of model update iteration and greatly shortens the development cycle of new models. The distillation process softens the output of the teacher model through the soft label temperature, which can reflect the correlation between different categories and provide more useful information for the student model. The soft label distillation loss and the hard label classification loss are weighted and combined, so that the student model can imitate the soft label output of the teacher model while trying to fit the original hard label to ensure the performance after distillation.

[0124] The lightweight student model is more suitable for deployment in edge computing devices. It can process and collect monitoring data locally in real time and issue timely warnings, thereby improving the efficiency of detecting and handling violations.

[0125] Example 2

[0126] Reference Figure 2 , which is the second embodiment of the present invention, provides an intelligent identification system for violations of new energy stations based on machine vision.

[0127] The system includes: a data acquisition module, a behavior recognition module, a new behavior discrimination module, an incremental learning module and a knowledge distillation module.

[0128] The data acquisition module is used to acquire image data in the new energy station, preprocess and enhance the acquired data, and create a behavioral image data set.

[0129] The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model, and train the behavior recognition model through a behavior image dataset.

[0130] The new behavior identification module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data in the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of violation.

[0131] When the clustering algorithm determines that an unknown behavior is a new type of violation, the incremental learning module transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image data set, allowing the behavior recognition model to perform incremental learning.

[0132] The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through the knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

[0133] Example 3

[0134] Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, according to another aspect of the present invention, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the above-mentioned intelligent identification method for illegal behaviors of new energy stations based on machine vision can be executed.

[0135] The method or system according to the embodiment of the present invention can also be used by Figure 3 The architecture of the electronic device shown is implemented.

[0136] like Figure 3 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, and the like.

[0137] The storage device in the electronic device 500, such as ROM 503 or hard disk 507, can store the intelligent identification method of illegal behaviors of new energy stations based on machine vision provided by the present invention.

[0138] The invention discloses an intelligent identification method for illegal behaviors in new energy stations based on machine vision, comprising the steps of obtaining image data in new energy stations, preprocessing and enhancing the obtained data, and creating a behavior image data set; constructing a behavior recognition model based on a convolutional neural network (CNN) model, and training the behavior recognition model through a behavior image data set; constructing a clustering algorithm based on contrastive learning, and when unknown behaviors are obtained in image data in new energy stations, determining whether the unknown behaviors belong to new types of illegal behaviors through a clustering algorithm; if the clustering algorithm determines that the unknown behaviors are new types of illegal behaviors, transmitting the new types of illegal behaviors as data samples to the behavior recognition model, expanding the number of samples of the new types of illegal behaviors, and inserting them into the original behavior image data set, so that the behavior recognition model can perform incremental learning; when the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model is transferred to the student model through the knowledge distillation technology.

[0139] Furthermore, the electronic device 500 may also include a user interface 508. Figure 3 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 3 One or more components of an electronic device are shown.

[0140] Example 4

[0141] Figure 4 It is a schematic diagram of the storage medium structure provided by an embodiment of the present invention.

[0142] like Figure 4 As shown is a storage medium 600 according to one embodiment of the present invention.

[0143] The storage medium 600 has computer readable instructions stored thereon.

[0144] When the computer-readable instructions are executed by the processor, the method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to the embodiments of the present invention described with reference to the above drawings can be executed.

[0145] The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. In addition, according to an embodiment of the present invention, the process described above with reference to the flowchart may be implemented as a computer software program.

[0146] For example, the present invention provides a non-temporary machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present invention, for example: obtaining image data in a new energy station, preprocessing and data enhancement of the obtained data, and creating a behavior image data set; constructing a behavior recognition model based on a convolutional neural network CNN model, and training the behavior recognition model through a behavior image data set; constructing a clustering algorithm based on contrastive learning, and when an unknown behavior is obtained in the image data in the new energy station, using a clustering algorithm to determine whether the unknown behavior belongs to a new type of violation; if the clustering algorithm determines the unknown behavior as a new type of violation, the new type of violation is transmitted as a data sample to the behavior recognition model, and the number of samples of the new type of violation is expanded and inserted into the original behavior image data set, so that the behavior recognition model performs incremental learning; when the behavior recognition model training is completed and parameter updates occur, the knowledge of the behavior recognition model constructed based on the convolutional neural network CNN model is transferred to the student model through knowledge distillation technology.

[0147] When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present invention are performed. The method, apparatus, and device of the present invention may be implemented in many ways. For example, the method, apparatus, and device of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware.

[0148] The above sequence for the steps of the method is for illustration only, and the steps of the method of the present invention are not limited to the sequence specifically described above unless otherwise specifically stated.

[0149] In addition, in some embodiments, the present invention can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Therefore, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0150] In addition, the parts of the above technical solutions provided in the embodiments of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0151] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The intelligent identification method of illegal behaviors in new energy stations based on machine vision is characterized by: include: Acquire image data from new energy stations, preprocess and enhance the acquired data, and create a behavioral image dataset; Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset; Construct a clustering algorithm based on contrastive learning. When unknown behaviors are obtained in the image data of new energy stations, the clustering algorithm is used to determine whether the unknown behaviors belong to new types of violations. If the clustering algorithm determines that the unknown behavior is a new type of violation, the new type of violation is transmitted to the behavior recognition model as a data sample, and the number of samples of the new type of violation is expanded and inserted into the original behavior image data set, so that the behavior recognition model can perform incremental learning; When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

2. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 1 is characterized in that: Deploy surveillance cameras and patrol drones in the new energy stations to obtain video images in the new energy stations, perform frame extraction on the obtained video images, and extract a frame of image as image data at a specific interval; Preprocessing the acquired new energy station image data, wherein the preprocessing includes image size normalization and image denoising; Perform data enhancement on the pre-processed image data in the new energy station to enhance the data features of the image data; Manually label samples of illegal behavior images, non-illegal behavior images, and unknown behavior images in existing new energy stations; randomly divide the labeled image samples into training sets, validation sets, and test sets according to set ratios.

3. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 2 is characterized in that: The convolutional neural network (CNN) model is used as the basic network of the behavior recognition model, and the initial parameters of the CNN model are configured. An adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a feature vector of a fixed size. Add a Softmax activation function after the fully connected layer in the CNN model to map the feature vector to the probability distribution of the violation category; Define the cross entropy loss function of the CNN model to measure the difference between the model prediction results and the true labels; Select Adam optimizer, set initial learning rate and batch size; Load the training set data and input the image data and labels into the model for training; At the end of each training cycle, the validation set is used to evaluate the performance of the model and the hyperparameters of the model are adjusted based on the validation results. The behavior recognition model training steps include taking the training set as the input of the CNN model, the CNN model taking the proportion of illegal behavior image samples, non-illegal behavior image samples and unknown behavior image samples as output, selecting cross entropy as the loss function, using the Adam optimizer for training, and stopping training when the accuracy on the validation set no longer improves.

4. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 3 is characterized in that: The SimCLR algorithm is used to perform comparative learning on the image data of the violation behavior and the non-violation behavior, and the comparative learning steps include: Perform two random data augmentations on the violation behavior images and non-violation behavior images respectively to obtain a pair of positive samples; Use the pre-trained residual network ResNet-50 model to extract feature representations of violation images and non-violation images; After representing the image data features, a projection head is added to map the features into a low-dimensional space; Calculate the contrast loss for the feature representation of the positive sample pair, maximize the similarity of the positive sample pair, and minimize the similarity of the negative sample pair; Train the residual network ResNet-50 model and projection head using the Adam optimizer and contrastive loss function until the contrastive loss function converges; Remove the projection head from the trained residual network ResNet-50 model, and use the residual network ResNet-50 model to extract feature representations of violation images and non-violation images; The feature representation of the violation image and the feature representation of the non-violation image are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes cluster centers of different types of violations.

5. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 4 is characterized in that: For the image data collected in the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When the image data is identified to contain illegal behavior, the illegal behavior is output; When an unknown behavior is identified, an unknown behavior image is output; The unknown behavior image features are extracted by training the residual network ResNet-50 model, and the Euclidean distance between the unknown behavior features and all cluster centers is calculated; Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into a violation or non-violation behavior; If the distance between the unknown behavior feature and any cluster center is less than the preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs; When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center in the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type.

6. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 5 is characterized in that: The behavior recognition model is allowed to perform incremental learning, including adding sample images of the new violation type to the original behavior image dataset when a new violation type is discovered through the comparison of the clustering algorithm, and performing data enhancement on the newly added sample images to expand the number of samples of the new category; In the output layer of the original violation recognition model, add a new neuron corresponding to the new violation category; Using the expanded training set, the violation identification model is updated and trained. The steps of the update training are as follows: Update the parameters of the last convolutional block and the fully connected layer in the CNN model, and reset the learning rate of the CNN model; use the Adam optimizer and the cross entropy loss function for training until the performance of the model on the validation set does not change; After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of new violation types are extracted, the sample image features of the new violation types are added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

7. The method for intelligently identifying illegal behaviors of new energy stations based on machine vision according to claim 5 is characterized in that: Based on the teacher-student network, knowledge distillation is performed on the trained behavior recognition model to achieve compression and lightweight of the original model: Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model; The step of knowledge distillation includes: applying a Softmax activation function to the outputs of the teacher model and the student model to obtain soft labels; Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the prediction results of the student model and the true labels as the classification loss; sum the distillation loss and classification loss to get the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges; The performance of the student model is evaluated on the test set, and the knowledge distillation process is completed; the lightweight student model is deployed on the edge device for real-time violation identification and alerting.

8. A system for intelligently identifying violations of new energy stations based on machine vision, which is used to implement the method for intelligently identifying violations of new energy stations based on machine vision as described in any one of claims 1 to 7, characterized in that: include: Data acquisition module, behavior recognition module, new behavior identification module, incremental learning module and knowledge distillation module; The data acquisition module is used to acquire image data in the new energy station, preprocess and enhance the acquired data, and create a behavior image data set; The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model, and train the behavior recognition model through a behavior image dataset; The new behavior identification module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of illegal behavior; The incremental learning module, when the clustering algorithm determines the unknown behavior as a new type of violation, transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image data set, so that the behavior recognition model can perform incremental learning; The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through the knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the machine vision-based intelligent identification method for illegal behaviors in new energy stations as described in any one of claims 1 to 7 by calling the computer program stored in the memory.

10. A storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer executes the method for intelligently identifying illegal behaviors of new energy stations based on machine vision as described in any one of claims 1 to 7.

Images