Intelligent Identification System and Method for Illegal Behaviors in New Energy Power Stations Based on Machine Vision

Abstract

The present invention relates to the field of deep learning technology, and specifically to a system and method for intelligently identifying violations at new energy stations based on machine vision. The method comprises: acquiring image data of new energy stations, performing preprocessing and data enhancement, and creating a behavioral image dataset; constructing a CNN-based behavior recognition model, and training it through the dataset; simultaneously constructing a clustering algorithm based on contrastive learning to determine whether an unknown behavior belongs to a new violation type; if a new violation type is found, its sample is transferred to the behavior recognition model for incremental learning to expand the number of samples; through knowledge distillation technology, the knowledge of the trained CNN behavior recognition model is transferred to a miniaturized student model to achieve lightweight deployment. The method of the present invention can not only automatically identify violations at new energy stations, but also continuously discover new types of violations and dynamically update the model to improve the adaptability and real-time performance of the system.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and specifically to a system and method for intelligently identifying violations at new energy stations based on machine vision. Background Art

[0002] With the rapid development of the new energy industry, the number of new energy stations is increasing. These stations are often located in remote areas, with sparse staff and weak security. This presents challenges such as difficult security management and frequent violations. Traditional manual inspections suffer from shortcomings such as delayed response, limited inspection coverage, and the tendency for fatigue and omissions. To ensure the safe and stable operation of these stations, the introduction of advanced intelligent security technologies is urgently needed.

[0003] Machine vision technology is currently widely used in the field of smart security. Traditional machine vision methods rely primarily on handcrafted features, resulting in poor generalization and difficulty adapting to complex and changing scenarios. In recent years, deep learning, particularly convolutional neural networks (CNNs), has achieved breakthroughs in image recognition. Applying CNNs to video surveillance analysis enables automatic detection, tracking, and recognition of objects such as people and vehicles in the footage. Furthermore, CNNs can analyze target behavior patterns, detect abnormal behavior, and issue early warnings.

[0004] The complexity and variability of real-world environments make it difficult for pre-trained deep learning models to cover all situations, resulting in insufficient generalization capabilities when faced with new and unknown scenarios. Incremental learning allows for the continuous acquisition of new knowledge while preserving existing knowledge, continuously expanding and updating models, enabling intelligent systems to continuously evolve. Currently, common incremental learning methods include fine-tuning, knowledge distillation, and parameter configuration networks. While deep learning has demonstrated superior performance in intelligent security, the sheer size of the models and computational complexity restricts its real-time deployment and application at the edge.

[0005] Although intelligent security technology has made great progress, there are still some problems in its application in the specific scenario of new energy stations: lack of targeted behavioral definitions. Unlike general public places, the behavior of people inside new energy stations is relatively simple; insufficient environmental adaptability. Affected by factors such as climate and light, the internal environment of new energy stations changes greatly. The robustness of existing general algorithm models needs to be improved to adapt to the actual application needs of this field; lack of ability to detect new types of violations. Faced with the emergence of new types of violations, the update and iteration mechanism of existing models is still imperfect, and the dynamic response capability needs to be strengthened; the degree of lightweight needs to be improved. Considering that new energy stations are located in remote areas and have limited network conditions, it is necessary to further compress the model volume, reduce computing resource consumption, and achieve efficient localized deployment.

[0006] To solve the above problems, the present invention proposes a system and method for intelligent identification of violations at new energy stations based on machine vision. Summary of the Invention

[0007] To achieve the above objectives, the present invention provides a system and method for intelligently identifying violations at new energy stations based on machine vision. The specific technical solutions are as follows:

[0008] Acquire image data from new energy stations, perform preprocessing and data enhancement on the acquired data, and create a behavioral image dataset;

[0009] Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset;

[0010] Construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior is a new type of violation.

[0011] If the clustering algorithm identifies an unknown behavior as a new type of violation, the new type of violation is transmitted as a data sample to the behavior recognition model. The number of samples of the new type of violation is expanded and inserted into the original behavior image dataset, allowing the behavior recognition model to perform incremental learning.

[0012] When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

[0013] Preferably, surveillance cameras and patrol drones are deployed in the new energy station to obtain video images in the new energy station, and the obtained video images are subjected to frame extraction processing, and a frame of image is extracted as image data at a specific interval;

[0014] Preprocessing the acquired new energy station image data, wherein the preprocessing includes image size normalization and image denoising;

[0015] Perform data enhancement on the pre-processed image data within the new energy station to enhance the data features of the image data;

[0016] Manually label samples of illegal behavior images, non-illegal behavior images, and unknown behavior images in existing new energy stations; randomly divide the labeled image samples into training sets, validation sets, and test sets according to the set ratio.

[0017] Preferably, a convolutional neural network (CNN) model is used as the basic network of the behavior recognition model, and the initial parameters of the CNN model are configured. An adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a feature vector of a fixed size.

[0018] Adding a Softmax activation function after the fully connected layer in the CNN model to map the feature vector into a probability distribution of the violation category;

[0019] Define the cross entropy loss function of the CNN model to measure the difference between the model prediction results and the actual labels;

[0020] Select Adam optimizer, set initial learning rate and batch size;

[0021] Load the training set data and input the image data and labels into the model for training;

[0022] At the end of each training cycle, the performance of the model is evaluated using the validation set, and the hyperparameters of the model are adjusted based on the validation results.

[0023] The behavior recognition model training steps include using the training set as the input of the CNN model. The CNN model outputs the ratio of violation image samples, non-violation image samples, and unknown behavior image samples. Cross entropy is selected as the loss function, and the Adam optimizer is used for training. Training is stopped when the accuracy on the validation set no longer improves.

[0024] Preferably, the SimCLR algorithm is used to perform comparative learning on the image data of the illegal behavior and the non-illegal behavior, and the comparative learning step includes:

[0025] Perform two random data augmentations on the violation images and non-violation images respectively to obtain a pair of positive samples;

[0026] Use the pre-trained residual network ResNet-50 model to extract feature representations of violation images and non-violation images;

[0027] After representing the image data features, a projection head is added to map the features into a low-dimensional space;

[0028] Compute the contrast loss for the feature representation of the positive sample pairs, maximize the similarity of the positive sample pairs, and minimize the similarity of the negative sample pairs;

[0029] The residual network ResNet-50 model and projection head are trained using the Adam optimizer and contrastive loss function until the contrastive loss function converges;

[0030] Remove the projection head from the trained ResNet-50 model and use it to extract feature representations of violation images and non-violation images.

[0031] The feature representations of the violation images and the feature representations of the non-violation images are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes cluster centers of different types of violations.

[0032] Preferably, for the image data collected from the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When the image data is identified to contain illegal behavior, the illegal behavior is output; when an unknown behavior is identified, the unknown behavior image is output;

[0033] The residual network ResNet-50 model is trained to extract the output unknown behavior image features and calculate the Euclidean distance between the unknown behavior features and all cluster centers;

[0034] Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into violation or non-violation behavior;

[0035] If the distance between the unknown behavior feature and any cluster center is less than the preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs;

[0036] When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center in the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type.

[0037] Preferably, allowing the behavior recognition model to perform incremental learning includes, when a new violation type is discovered through comparison of the clustering algorithm, adding sample images of the new violation type to the original behavior image dataset, and performing data enhancement on the newly added sample images to expand the number of samples of the new category;

[0038] In the output layer of the original violation recognition model, a new neuron is added to correspond to the new violation category;

[0039] The expanded training set is used to update the violation identification model. The steps of the update training are as follows:

[0040] Update the parameters of the last convolutional block and the fully connected layer in the CNN model, reset the learning rate of the CNN model; use the Adam optimizer and the cross-entropy loss function to train until the performance of the model on the validation set does not change;

[0041] After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of the new violation type are extracted, added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

[0042] Preferably, knowledge distillation is performed on the trained behavior recognition model based on the teacher-student network to achieve compression and lightweighting of the original model:

[0043] Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model;

[0044] The knowledge distillation step includes: applying a softmax activation function to the outputs of the teacher model and the student model to obtain soft labels;

[0045] Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the student model's prediction results and the true labels as the classification loss; sum the distillation loss and classification loss to obtain the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges;

[0046] The performance of the student model is evaluated on the test set, and the knowledge distillation process is completed; the lightweight student model is deployed on the edge device for real-time violation identification and alerting.

[0047] A machine vision-based intelligent identification system for violations at new energy stations, which is used to implement the machine vision-based intelligent identification method for violations at new energy stations, includes: a data acquisition module, a behavior recognition module, a new behavior discrimination module, an incremental learning module, and a knowledge distillation module;

[0048] The data acquisition module is used to acquire image data from the new energy station, preprocess and enhance the acquired data, and create a behavioral image dataset;

[0049] The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset;

[0050] The new behavior discrimination module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of violation;

[0051] When the clustering algorithm identifies an unknown behavior as a new type of violation, the incremental learning module transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image dataset, allowing the behavior recognition model to perform incremental learning;

[0052] The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

[0053] An electronic device comprises: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the machine vision-based intelligent identification method for violations in new energy stations by calling the computer program stored in the memory.

[0054] A storage medium stores instructions. When the instructions are run on a computer, the computer executes the method for intelligently identifying violations at new energy stations based on machine vision.

[0055] The present invention has the following beneficial effects: By acquiring real-world image data and performing preprocessing and data enhancement, it can construct a behavioral image dataset that is highly relevant to the application scenario. High-quality, diverse datasets are the foundation for training high-performance behavior recognition models, helping to improve their generalization and robustness.

[0056] This paper applies CNN to behavior recognition, enabling end-to-end learning of behavior patterns starting from raw images, eliminating the need for manual feature design and streamlining the development process. Through training, the CNN model can learn discriminative features for different behavior types, developing powerful behavior recognition capabilities.

[0057] This invention introduces a clustering algorithm based on contrastive learning to discover new types of violations and expand the existing behavior classification system. This unsupervised automatic discovery mechanism can improve the system's adaptability to new situations and reduce the limitations of manual definition.

[0058] Through incremental learning, the behavior recognition model can be updated and iterated in real time, continuously learning new knowledge and expanding the recognition categories. This dynamic update mechanism can help the model quickly respond to environmental changes and proactively adapt to new violation patterns, improving the real-time performance and accuracy of the system.

[0059] Through knowledge distillation, this method can transfer the knowledge of complex models into a miniaturized student model. The distilled student model can significantly reduce the computational workload, enabling lightweight deployment of behavior recognition algorithms, improving the system's real-time processing capabilities, and facilitating its application in edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of the method for intelligently identifying violations at new energy stations based on machine vision provided by the present invention;

[0061] Figure 2 This is a structural diagram of the machine vision-based intelligent identification system for new energy stations and illegal behaviors provided by the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of an electronic device provided by one embodiment of the present invention;

[0063] Figure 4 It is a schematic diagram of the storage medium structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] For a better understanding of the present invention, various aspects of the present invention will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely descriptions of exemplary embodiments of the present invention and are not intended to limit the scope of the present invention in any way. Throughout this specification, like reference numerals refer to like elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0065] In the accompanying drawings, the size, dimensions, and shapes of the elements have been slightly adjusted for ease of illustration. The accompanying drawings are for illustration purposes only and are not drawn strictly to scale. As used herein, the terms "substantially," "approximately," and similar terms are used to indicate approximate values, not degrees, and are intended to illustrate inherent deviations in measurements or calculations that would be recognized by a person of ordinary skill in the art. In addition, in the present invention, the order in which the steps are described does not necessarily represent the order in which these steps occur in actual operation, unless otherwise specified or inferred from the context.

[0066] It should also be understood that expressions such as "comprises," "including," "having," "includes," and / or "comprising" are open rather than closed expressions in this specification, indicating the presence of the stated features, elements, and / or components, but do not exclude the presence of one or more other features, elements, components, and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present invention, "may" is used to mean "one or more embodiments of the present invention." And, the term "exemplary" is intended to refer to an example or illustration.

[0067] Unless otherwise defined, all terms used herein (including engineering terms and scientific and technological terms) have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. It should also be understood that, unless otherwise expressly stated in the present invention, words defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0068] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0069] Example 1

[0070] Reference Figure 1 , which is the first embodiment of the present invention, provides an intelligent identification method for violations at new energy stations based on machine vision.

[0071] S1: Acquire image data from new energy stations, preprocess and enhance the acquired data, and create a behavioral image dataset.

[0072] Surveillance cameras and patrol drones are deployed in new energy stations to obtain video images inside the new energy stations, and the obtained video images are subjected to frame extraction processing, with one frame of image being extracted as image data at specific time intervals.

[0073] The acquired new energy station image data is preprocessed, and the preprocessing includes image size normalization and image denoising.

[0074] The image size normalization includes scaling the image to a uniform size (eg, 224×224 pixels) to facilitate subsequent feature extraction and model input.

[0075] The image denoising includes using median filtering and bilateral filtering methods to remove noise in the image and improve image quality.

[0076] The pre-processed image data in the new energy station is enhanced to enhance the data features of the image data.

[0077] The data enhancement includes using histogram equalization and contrast adjustment methods to enhance the visual effect of the image and highlight the key information in the image data.

[0078] Manually label the samples D of illegal behavior images, non-illegal behavior images and unknown behavior images in the existing new energy stations, D = {P j ,y j}, j = 1, 2, ..., n, y j =0, 1, 2, ..., K, where P j is the jth image sample, n is the number of image samples, y j is the label of the jth image sample, K is the number of image labels; y j =0 means the label of the image sample is a non-violation image, y j =1,...,K-1, it means that the labels of the image samples are different types of violation images; j =K, it means that the label of the image sample is an unknown behavior image.

[0079] The labeled image samples are randomly divided into training set, validation set and test set according to the set ratio.

[0080] By deploying surveillance cameras and drones, video image data within new energy stations can be collected from all directions and angles, providing a data basis for subsequent behavior recognition; pre-processing operations such as size normalization and image denoising on the original image data can eliminate irrelevant interference in the image, improve image quality, and prepare for feature extraction; expanding training samples through data enhancement technology can improve the generalization ability and robustness of the model and reduce the risk of overfitting; the collected image data is manually labeled and divided into training sets, validation sets, and test sets to provide a standard data set for the training, parameter adjustment, and performance evaluation of the behavior recognition model.

[0081] S2: Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset.

[0082] The convolutional neural network (CNN) model is used as the basic network of the behavior recognition model. The initial parameters of the CNN model are configured, and an adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a feature vector of a fixed size.

[0083] A Softmax activation function is added after the fully connected layer in the CNN model to map the feature vector into the probability distribution of the violation category.

[0084] Construct a convolutional neural network CNN model, and assume that the input of the lth convolutional layer is: The convolution kernel is: Bias The activation function is f(·), then the output of the l-th layer convolution is:

[0085] X (l) =f(W (l) *X (l-1) +b (l) )

[0086] Among them, * represents the convolution operation. The output of the convolution layer passes through the pooling layer and the fully connected layer, and then passes through the Softmax function to obtain the final classification probability.

[0087] Define the cross-entropy loss function of the CNN model to measure the difference between the model's predictions and the true labels; select the Adam optimizer, set the initial learning rate and batch size; load the training set data, and input the image data and labels into the model for training.

[0088] At the end of each training cycle, the performance of the model is evaluated using the validation set, and the model's hyperparameters are adjusted based on the validation results.

[0089] The behavior recognition model training steps include using the training set as the input of the CNN model. The CNN model outputs the ratio of violation image samples, non-violation image samples, and unknown behavior image samples. Cross entropy is selected as the loss function, and the Adam optimizer is used for training. Training is stopped when the accuracy on the validation set no longer improves.

[0090] Building a behavior recognition model based on CNN can automatically extract high-level semantic features in images, which has stronger expressive power than manually designed features. Adding adaptive pooling layers and fully connected layers at the end of the CNN network can map feature maps of different sizes into feature vectors of fixed length, which facilitates classifier processing. By connecting the Softmax activation function after the fully connected layer, the feature space is mapped into the probability distribution of the violation category, obtaining intuitive classification results. Using a mature classifier training paradigm, cross entropy as the loss function, and the Adam optimizer to train the CNN model can efficiently fit the sample data. After each round of iteration, the current model performance is evaluated using a validation set, and hyperparameters are adjusted in a timely manner to avoid overfitting and improve the model's generalization performance.

[0091] S3: Construct a clustering algorithm based on contrastive learning. When unknown behaviors are obtained in the image data of new energy stations, the clustering algorithm is used to determine whether the unknown behaviors belong to new types of violations.

[0092] The SimCLR algorithm is used to perform comparative learning on image data of violations and non-violations, wherein the comparative learning steps include:

[0093] Random data augmentation is performed twice on the violation images and non-violation images respectively to obtain a pair of positive samples.

[0094] The pre-trained residual network ResNet-50 model is used to extract feature representations of violation images and non-violation images.

[0095] After representing the image data features, a projection head is added to map the features into a low-dimensional space.

[0096] The contrast loss is calculated for the feature representation of the positive sample pairs, maximizing the similarity of the positive sample pairs and minimizing the similarity of the negative sample pairs.

[0097] The residual network ResNet-50 model and projection head are trained using the Adam optimizer and the contrastive loss function until the contrastive loss function converges.

[0098] The projection head is removed from the trained residual network ResNet-50 model, and the residual network ResNet-50 model is used to extract feature representations of violation images and non-violation images.

[0099] The feature representations of the violation images and the feature representations of the non-violation images are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes K-1 cluster centers of different types of violations.

[0100] By introducing contrastive learning, more discriminative image feature representations can be learned by comparing positive samples of the same category, paving the way for downstream clustering tasks. Contrastive learning maps features to a low-dimensional space, reducing the computational overhead of clustering tasks while retaining important semantic information. Unsupervised clustering algorithms such as K-means can be used to automatically group unknown behaviors with similar features into clusters, facilitating the analysis of their commonalities. By comparing the features of unknown behaviors with the cluster centers of known violations, it can be determined whether the unknown behavior is close to a known violation, providing a reference for the classification of unknown behaviors. If the features of the unknown behavior are far away from all known cluster centers, it can be preliminarily determined to be a new type of violation, providing a basis for timely updating the violation category.

[0101] S4: If the clustering algorithm determines that the unknown behavior is a new type of violation, the new type of violation will be transmitted to the behavior recognition model as a data sample, and data enhancement technology will be used to expand the number of samples of the new type of violation and insert them into the original behavior image dataset, allowing the behavior recognition model to perform incremental learning.

[0102] For the image data collected in the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When the image data is identified to contain illegal behavior, the output is that the illegal behavior is identified; when an unknown behavior is identified, the unknown behavior image is output.

[0103] The residual network ResNet-50 model is trained to extract the output unknown behavior image features and calculate the Euclidean distance between the unknown behavior features and all cluster centers.

[0104] Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into violation or non-violation.

[0105] If the distance between the unknown behavior feature and any cluster center is less than a preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs.

[0106] When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center in the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type.

[0107] Allowing the behavior recognition model to perform incremental learning includes adding sample images of the new violation type to the original behavior image dataset when a new violation type is discovered through comparison of the clustering algorithm, and performing data enhancement on the newly added sample images to expand the number of samples of the new category.

[0108] In the output layer of the original violation recognition model, a new neuron is added to correspond to the new violation category.

[0109] The expanded training set is used to update the violation identification model. The steps of the update training are as follows:

[0110] Update the parameters of the last convolutional block and the fully connected layer in the CNN model, reset the learning rate of the CNN model; use the Adam optimizer and the cross-entropy loss function to train until the performance of the model on the validation set does not change.

[0111] After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of the new violation type are extracted, added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

[0112] When the clustering algorithm discovers a new type of violation, its samples are promptly added to the original training set, which can expand the category coverage of the behavior recognition model and enhance the model's ability to cope with new situations. Data enhancement is performed on the newly inserted violation category samples, which can alleviate the problem of insufficient sample size of the new category to a certain extent, so that the new category can be learned by the model more quickly. Based on the original CNN model structure, only the terminal convolution block and fully connected layer parameters are updated. While adapting to the new category, it can also inherit the original model's knowledge of known categories and achieve incremental learning. By re-clustering and updating the cluster center, the impact of the new violation type on the original category division can be included, making the clustering results more accurate.

[0113] The entire incremental learning process can dynamically expand the types of violations without training from scratch, improving the system's real-time response and sustainable optimization capabilities.

[0114] S5: When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

[0115] Based on the teacher-student network, knowledge distillation is performed on the trained behavior recognition model to achieve compression and lightweighting of the original model.

[0116] Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model;

[0117] The knowledge distillation step includes: applying a softmax activation function to the outputs of the teacher model and the student model to obtain soft labels;

[0118] Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the prediction results of the student model and the true labels as the classification loss; sum the weighted distillation loss and classification loss to obtain the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges.

[0119] Calculate the distillation loss in knowledge distillation, assuming the soft label output of the teacher model is p T , the soft label output of the student model is p S , the temperature coefficient is τ, then the distillation loss based on KL divergence is:

[0120]

[0121] Among them, p T (a) and p S(a) represents the soft label output of the teacher model and the student model for the a-th category. The temperature coefficient τ is used to control the smoothness of the soft label and is usually greater than 1.

[0122] The performance of the student model is evaluated on the test set, and the knowledge distillation process is completed; the trained student model is deployed on the edge device for real-time violation identification and alerting.

[0123] Through knowledge distillation, the knowledge of a trained complex teacher model can be compressed into a lighter student model, reducing the model's storage and computational overhead while maintaining high performance. This eliminates the need to retrain lightweight models, simplifying the model update and iteration process and significantly shortening the development cycle of new models. The distillation process softens the output of the teacher model through soft label temperature, which can reflect the correlation between different categories and provide more useful information for the student model. The soft label distillation loss and hard label classification loss are weighted together, so that the student model can imitate the soft label output of the teacher model while also trying to fit the original hard label to ensure performance after distillation.

[0124] The lightweight student model is more suitable for deployment in edge computing devices. It can process and collect monitoring data locally in real time and issue timely warnings, thereby improving the efficiency of detecting and handling violations.

[0125] Example 2

[0126] Reference Figure 2 , which is the second embodiment of the present invention, provides an intelligent identification system for violations at new energy stations based on machine vision.

[0127] The system includes: a data acquisition module, a behavior recognition module, a new behavior discrimination module, an incremental learning module and a knowledge distillation module.

[0128] The data acquisition module is used to acquire image data from new energy stations, preprocess and enhance the acquired data, and create a behavioral image dataset.

[0129] The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset.

[0130] The new behavior discrimination module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of violation.

[0131] When the clustering algorithm determines that an unknown behavior is a new type of violation, the incremental learning module transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image data set, allowing the behavior recognition model to perform incremental learning.

[0132] The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

[0133] Example 3

[0134] Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 According to another aspect of the present invention, an electronic device 500 is provided. The electronic device 500 may include one or more processors and one or more memories. The memories may store computer-readable code that, when executed by the one or more processors, may execute the aforementioned method for intelligently identifying violations at new energy stations based on machine vision.

[0135] The method or system according to an embodiment of the present invention can also be used by Figure 3 The electronic device architecture shown is implemented.

[0136] like Figure 3 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, and the like.

[0137] The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, can store the intelligent identification method for illegal behaviors in new energy stations based on machine vision provided by the present invention.

[0138] An intelligent identification method for violations at new energy stations based on machine vision includes: acquiring image data in new energy stations, preprocessing and data enhancement of the acquired data, and creating a behavior image dataset; constructing a behavior recognition model based on a convolutional neural network (CNN) model, and training the behavior recognition model through a behavior image dataset; constructing a clustering algorithm based on contrastive learning, and when unknown behaviors are acquired in image data in new energy stations, determining whether the unknown behaviors belong to new types of violations through a clustering algorithm; if the clustering algorithm determines that the unknown behaviors are new types of violations, transmitting the new types of violations as data samples to the behavior recognition model, expanding the number of samples of the new types of violations, and inserting them into the original behavior image dataset, so that the behavior recognition model can perform incremental learning; when the behavior recognition model training is completed and parameters are updated, the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

[0139] Furthermore, the electronic device 500 may further include a user interface 508. Figure 3 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 3 One or more components of an electronic device are shown.

[0140] Example 4

[0141] Figure 4 It is a schematic diagram of the storage medium structure provided by an embodiment of the present invention.

[0142] like Figure 4 FIG. 6 shows a storage medium 600 according to an embodiment of the present invention.

[0143] The storage medium 600 has computer-readable instructions stored thereon.

[0144] When the computer-readable instructions are executed by the processor, the method for intelligently identifying violations in new energy stations based on machine vision according to the embodiments of the present invention described with reference to the above drawings can be executed.

[0145] The storage medium 600 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. In addition, according to an embodiment of the present invention, the process described above with reference to the flowchart may be implemented as a computer software program.

[0146] For example, the present invention provides a non-temporary machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present invention, such as: obtaining image data in a new energy station, preprocessing and data enhancement of the obtained data, and creating a behavior image dataset; constructing a behavior recognition model based on a convolutional neural network (CNN) model, and training the behavior recognition model through a behavior image dataset; constructing a clustering algorithm based on contrastive learning, and when an unknown behavior is obtained in the image data in the new energy station, determining whether the unknown behavior belongs to a new type of violation through a clustering algorithm; if the clustering algorithm determines that the unknown behavior is a new type of violation, the new type of violation is transmitted as a data sample to the behavior recognition model, and the number of samples of the new type of violation is expanded and inserted into the original behavior image dataset, so that the behavior recognition model performs incremental learning; when the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology.

[0147] When the computer program is executed by a central processing unit (CPU), the functions defined in the method of the present invention are performed. The method, apparatus, and device of the present invention may be implemented in many ways. For example, the method, apparatus, and device of the present invention may be implemented using software, hardware, firmware, or any combination of software, hardware, and firmware.

[0148] The above sequence of steps for the method is for illustration only, and the steps of the method of the present invention are not limited to the sequence specifically described above unless otherwise specifically stated.

[0149] In addition, in some embodiments, the present invention can also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Therefore, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0150] In addition, the parts of the above technical solutions provided in the embodiments of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0151] The above-described specific embodiments further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. The intelligent identification method of illegal behaviors in new energy stations based on machine vision is characterized by: include: Acquire image data from new energy stations, perform preprocessing and data enhancement on the acquired data, and create a behavioral image dataset; Build a behavior recognition model based on the convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset; Construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior is a new type of violation. For the image data collected from the new energy station, the trained behavior recognition model is used to identify the behavior type in the image data. When the image data is identified to contain illegal behavior, the illegal behavior is output; When an unknown behavior is identified, an unknown behavior image is output; The residual network ResNet-50 model is trained to extract the output unknown behavior image features and calculate the Euclidean distance between the unknown behavior features and all cluster centers; Find the nearest cluster center and classify the unknown behavior into the category to which the nearest cluster center belongs, that is, classify the unknown behavior into violation or non-violation behavior; If the distance between the unknown behavior feature and any cluster center is less than the preset threshold, the unknown behavior is marked as the behavior type represented by the cluster center to which it belongs; When an unknown behavior feature is determined to be a violation, if the distance between the unknown behavior feature and any cluster center of the violation is less than a preset threshold, the unknown behavior is classified as the violation type represented by the cluster center; otherwise, the unknown behavior is marked as a new violation type; If the clustering algorithm identifies an unknown behavior as a new type of violation, the new type of violation is transmitted as a data sample to the behavior recognition model. The number of samples of the new type of violation is expanded and inserted into the original behavior image dataset, allowing the behavior recognition model to perform incremental learning. When the behavior recognition model training is completed and the parameters are updated, the knowledge of the behavior recognition model built based on the convolutional neural network (CNN) model is transferred to the student model through knowledge distillation technology; Select a lightweight CNN model as the student model, use the trained behavior recognition model as the teacher model, and perform knowledge distillation on the student model; The knowledge distillation step includes: applying a softmax activation function to the outputs of the teacher model and the student model to obtain soft labels; Calculate the KL divergence between the soft labels of the student model and the soft labels of the teacher model as the distillation loss; calculate the cross entropy loss between the student model's prediction results and the true labels as the classification loss; sum the distillation loss and classification loss to obtain the total loss function; use the Adam optimizer and the total loss function to train the student model until the cross entropy loss function converges; Evaluate the performance of the student model on the test set and complete the process of knowledge distillation.

2. The method for intelligently identifying violations at new energy stations based on machine vision according to claim 1 is characterized in that: Deploy surveillance cameras and patrol drones in new energy stations to capture video images within the stations, perform frame extraction on the captured video images, and extract a frame of image as image data at specific intervals; Preprocessing the acquired new energy station image data, wherein the preprocessing includes image size normalization and image denoising; Perform data enhancement on the pre-processed image data within the new energy station to enhance the data features of the image data; Manually label samples of images of violations, non-violations, and unknown behaviors in existing new energy stations; randomly divide the labeled image samples into training sets, validation sets, and test sets according to the set ratio.

3. The method for intelligently identifying violations at new energy stations based on machine vision according to claim 2 is characterized in that: The convolutional neural network (CNN) model is used as the basic network of the behavior recognition model. The initial parameters of the CNN model are configured. An adaptive pooling layer and a fully connected layer are added at the end of the CNN model to convert the feature map into a fixed-size feature vector. Adding a Softmax activation function after the fully connected layer in the CNN model to map the feature vector into a probability distribution of the violation category; Define the cross entropy loss function of the CNN model to measure the difference between the model prediction results and the true labels; Select Adam optimizer, set initial learning rate and batch size; Load the training set data and input the image data and labels into the model for training; At the end of each training cycle, the performance of the model is evaluated using the validation set, and the model's hyperparameters are adjusted based on the validation results. The behavior recognition model training steps include using the training set as the input of the CNN model. The CNN model outputs the ratio of violation image samples, non-violation image samples, and unknown behavior image samples. Cross entropy is selected as the loss function, and the Adam optimizer is used for training. Training is stopped when the accuracy on the validation set no longer improves.

4. The method for intelligently identifying violations at new energy stations based on machine vision according to claim 3 is characterized in that: The SimCLR algorithm is used to perform comparative learning on image data of violations and non-violations, wherein the comparative learning steps include: Perform two random data augmentations on violation images and non-violation images respectively to obtain a pair of positive samples; Use the pre-trained residual network ResNet-50 model to extract feature representations of violation images and non-violation images; After representing the image data features, a projection head is added to map the features into a low-dimensional space; Compute the contrast loss for the feature representation of the positive sample pairs, maximize the similarity of the positive sample pairs, and minimize the similarity of the negative sample pairs; Use the Adam optimizer and contrastive loss function to train the residual network ResNet-50 model and projection head until the contrastive loss function converges; Remove the projection head from the trained ResNet-50 model and use it to extract feature representations of violation images and non-violation images. The feature representations of the violation images and the feature representations of the non-violation images are respectively input into the K-means clustering algorithm to obtain the violation cluster center and the non-violation cluster center respectively; the violation cluster center includes cluster centers of different types of violations.

5. The method for intelligently identifying violations at new energy stations based on machine vision according to claim 4 is characterized in that: The behavior recognition model performs incremental learning. When a new violation type is discovered through clustering algorithm comparison, sample images of the new violation type are added to the original behavior image dataset. The newly added sample images are then augmented to increase the number of samples of the new category. In the output layer of the original violation recognition model, a new neuron is added to correspond to the new violation category; The expanded training set is used to update the violation identification model. The steps of the update training are as follows: Update the parameters of the last convolutional block and the fully connected layer in the CNN model, reset the learning rate of the CNN model; use the Adam optimizer and the cross-entropy loss function to train until the performance of the model on the validation set does not change; After updating and training the CNN model, the cluster centers of violations and non-violations in the K-means clustering algorithm are updated: sample image features of the new violation type are extracted, added to the original violation feature set, and the K-means clustering algorithm is rerun to update the cluster centers of violations.

6. The method for intelligently identifying violations at new energy stations based on machine vision according to claim 5 is characterized in that: Based on the teacher-student network, knowledge distillation is performed on the trained behavior recognition model to achieve compression and lightweighting of the original model: Deploy the lightweight student model to edge devices for real-time violation identification and alerting.

7. A machine vision-based intelligent identification system for violations at new energy stations, which is used to implement the machine vision-based intelligent identification method for violations at new energy stations according to any one of claims 1 to 6, characterized in that: include: Data acquisition module, behavior recognition module, new behavior discrimination module, incremental learning module and knowledge distillation module; The data acquisition module is used to acquire image data from the new energy station, preprocess and enhance the acquired data, and create a behavioral image dataset; The behavior recognition module is used to build a behavior recognition model based on a convolutional neural network (CNN) model and train the behavior recognition model using a behavior image dataset; The new behavior discrimination module is used to construct a clustering algorithm based on contrastive learning. When an unknown behavior is obtained in the image data of the new energy station, the clustering algorithm is used to determine whether the unknown behavior belongs to a new type of violation; When the clustering algorithm identifies an unknown behavior as a new type of violation, the incremental learning module transmits the new type of violation as a data sample to the behavior recognition model, expands the number of samples of the new type of violation, and inserts them into the original behavior image dataset, allowing the behavior recognition model to perform incremental learning; The knowledge distillation module transfers the knowledge of the behavior recognition model constructed based on the convolutional neural network (CNN) model to the student model through knowledge distillation technology when the behavior recognition model training is completed and the parameters are updated.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the machine vision-based intelligent identification method for violations in new energy stations as described in any one of claims 1 to 6 by calling the computer program stored in the memory.

9. A storage medium, characterized in that: Instructions are stored, and when the instructions are run on a computer, the computer executes the method for intelligently identifying violations of new energy stations based on machine vision as described in any one of claims 1 to 6.

Images