Power equipment intelligent identification method and system based on deep learning

Abstract

The invention provides a power equipment intelligent identification method and system based on deep learning, and relates to the technical field of equipment identification, and the method comprises the steps: firstly collecting a real-time image and a historical image of power equipment, splicing the historical image into a spliced image, randomly intercepting a sub-image with the same size as the historical image from the spliced image, and storing the sub-image in a database; adding training labels for the historical images and the sub-images to obtain training data; the method comprises the following steps: constructing a classification model by using a deep learning algorithm, training the classification model by using training data, inputting a real-time image into the trained classification model, and outputting a classification result, and can improve the accuracy of a power equipment identification result under the condition of limited training samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device recognition, and in particular to a power equipment intelligent recognition method and system based on deep learning. BACKGROUND

[0002] In the daily operation and management of the power system, accurate identification of power equipment is a key link to ensure the safe and stable operation of the power grid. With the continuous expansion and complication of the power network, the types and quantities of power equipment are increasing, and the distribution range is becoming more extensive. The traditional manual inspection method is not only inefficient, but also requires a lot of manpower, material resources and time. Moreover, due to the subjectivity and limitations of manual judgment, it is easy to miss or misjudge, which cannot meet the high requirements of modern power systems on the accuracy and timeliness of equipment identification. Therefore, it is an important problem to be solved in the power industry to realize efficient and accurate intelligent identification of power equipment, which is of great significance to improve the operation and management level of the power system and ensure the reliability of power supply.

[0003] To solve the problem of power equipment identification, a variety of technical means have been proposed and applied. Among them, the traditional method based on image processing is a common technology. This kind of method mainly extracts features such as color, shape and texture from power equipment images, and then uses pattern recognition algorithms to analyze and classify the extracted features, so as to realize the identification of power equipment. In addition, with the rapid development of artificial intelligence technology, methods based on machine learning have also been gradually applied in the field of power equipment identification. Some researches use traditional machine learning algorithms such as support vector machine (SVM) and decision tree to build classification models for classification and identification of power equipment images.

[0004] In recent years, the rise of deep learning technology has brought new ideas and methods to power equipment identification. Deep learning algorithms have strong feature learning and expression capabilities, and can automatically learn effective feature representations from a large number of image data. However, there are deficiencies in training data acquisition and processing, and it is difficult to obtain a large amount of high-quality and accurately labeled power equipment image data, which increases the cost and time of data collection. SUMMARY

[0005] In order to improve the accuracy of power equipment identification results under the condition of limited training samples, the present application provides a power equipment intelligent recognition method and system based on deep learning.

[0006] In the first aspect, the present application provides a power equipment intelligent recognition method based on deep learning, which adopts the following technical scheme: A power equipment intelligent recognition method based on deep learning, comprising the following steps: The real-time image and the historical image of the power equipment are collected, the historical image is spliced to obtain a spliced image, a sub-image with the same size as the historical image is randomly cropped in the spliced image, training labels are added to the historical image and the sub-image to obtain training data. A classification model is constructed by using a deep learning algorithm, the classification model is trained by using the training data to obtain a trained classification model, and the real-time image is input into the trained classification model to output a classification result.

[0007] The application effectively expands the training data set, increases the diversity and richness of the data, helps the classification model to learn more comprehensive image features, improves the classification accuracy and generalization ability of the classification model, and enables the classification model to accurately classify when facing different scenes and different states of power equipment images. The classification model constructed by the application can quickly classify the collected real-time images of the power equipment, realizing real-time monitoring of the state of the power equipment. By timely outputting the classification result, the application can help the operation and maintenance personnel to quickly understand the operation status of the power equipment, timely discover potential problems, provide strong support for the maintenance and management of the power equipment, and help to improve the stability and reliability of the power system.

[0008] Optionally, the power equipment includes a plurality of equipment components, after the sub-image with the same size as the historical image is randomly cropped in the spliced image, the method further includes: The overall bounding box of the power equipment in the spliced image is labeled by using a pre-constructed YOLOv8s model to obtain a spliced image labeled with an overall bounding box, the spliced image labeled with the overall bounding box is subjected to semantic segmentation processing to obtain a component segmentation mask, and the component contour is extracted in the component segmentation mask, and the minimum bounding rectangle is calculated according to the component contour. A coordinate system is constructed to determine the coordinate range of the minimum bounding rectangle and the coordinate range of the sub-image, and based on the coordinate range of the minimum bounding rectangle and the coordinate range of the sub-image, the sub-image containing the incomplete minimum bounding rectangle is deleted.

[0009] The application retains the sub-image containing the complete equipment component, so that the training data focuses more on the effective equipment component, which helps the classification model to better learn the feature pattern of the equipment component and improves the training efficiency and classification accuracy of the classification model. After deleting part of the sub-image, the amount of data to be processed in the training process of the classification model is reduced, the consumption of computing resources is reduced, and the training speed is improved.

[0010] Optionally, the YOLOv8s model comprises a backbone network for feature extraction, a neck network for feature fusion, and a detection head for predicting overall bounding boxes, before labeling the overall bounding boxes of the power equipment contained in the spliced image using the pre-constructed YOLOv8s model, the method further comprises: Generating labels: obtaining images of power equipment contained in the ImageNet-X dataset, denoted as target images, using a labeling tool to label the overall bounding boxes of the power equipment contained in the target images, and generating YOLO format labels; Pre-training: pre-training the pre-constructed YOLOv8s model using the target images with YOLO format labels to obtain a pre-trained YOLOv8s model; Fine-tuning: freezing the parameters of the first m layers of the backbone network and the parameters of the down-sampling path of the neck network in the pre-trained YOLOv8s model, obtaining n spliced images labeled with overall bounding boxes as a training set, and fine-tuning the parameters of the unfrozen part of the pre-trained YOLOv8s model using the training set to obtain a migration trained YOLOv8s model; Model updating: updating the migration trained YOLOv8s model to the pre-constructed YOLOv8s model.

[0011] The present application uses a labeling tool to label the overall bounding boxes of the power equipment images in the ImageNet-X dataset and generate YOLO format labels, providing clear target information for model training. The YOLO format label is simple and easy for the YOLOv8s model to analyze, enabling the YOLOv8s model to quickly understand the location of the power equipment in the image, thereby learning the mapping relationship between the features and the bounding box position in a targeted manner, which helps the YOLOv8s model to quickly converge in the early stage of training. The ImageNet-X dataset contains a large number of power equipment images under different scenarios, different angles, and different lighting conditions. By generating YOLO format labels using this dataset, the YOLOv8s model can be exposed to a variety of power equipment image samples, learn more extensive feature representations, enhance the adaptability of the YOLOv8s model to various complex situations, and improve the generalization of the YOLOv8s model.

[0012] The pre-trained YOLOv8s model has already learned some basic features and patterns of power equipment before formal training, so that the YOLOv8s model can learn more efficiently in the subsequent training process, reduce the number of training iterations, speed up the convergence speed, save training time and computing resources. In the fine-tuning step, the YOLOv8s model can learn the unique features and boundary box distribution of the power equipment in the spliced image by using the spliced image labeled with the overall boundary box as the training set, thereby improving the labeling accuracy of the model on the spliced image.

[0013] Optionally, after performing the step of generating labels, before performing the step of pre-training, the method further comprises: performing SimCLR style enhancement processing on each target image to obtain an enhanced image pair for each target image; constructing a contrast learning model, the contrast learning model comprising an encoder and a projection head, the encoder being a backbone network of the YOLOv8s model, the projection head comprising 2 layers of MLP, iteratively training the contrast learning model using the enhanced image pair until a preset stopping condition is met, stopping training, outputting parameters of the backbone network, and updating the backbone network parameters to the pre-constructed YOLOv8s model to perform the step of pre-training.

[0014] The present application performs SimCLR style enhancement processing on each target image to generate an enhanced image pair, thereby creating new images similar to the original images but with differences. In practical applications, the images of power equipment may be affected by various factors, such as changes in lighting, different shooting angles, and equipment surface stains. SimCLR style enhancement processing can simulate these changes in real-world scenarios, allowing the YOLOv8s model to be exposed to more image samples similar to actual situations during training. By generating an enhanced image pair, the present application increases the diversity of data, which helps the YOLOv8s model to learn more generalizable features and avoid overfitting to specific image features as much as possible, so that the YOLOv8s model can better adapt to power equipment images under different scenarios and conditions.

[0015] The contrast learning model constructed in the application takes the backbone network of the YOLOv8s model as an encoder. By training the contrast learning model, the backbone network can learn more discriminative feature representations. Contrast learning is an unsupervised learning method that does not require manual annotation of additional information. It only needs to train the similarity and dissimilarity between pairs of enhanced images, which reduces the workload and cost of manual annotation to a certain extent. The application uses a large number of unlabeled image data of power equipment to train, further enriches the learning samples of the contrast learning model, and improves the performance of the contrast learning model.

[0016] The application improves the feature extraction capability of the backbone network through contrast learning, and further provides higher quality feature input for the bounding box labeling task of the entire YOLOv8s model. In the subsequent pre-training and fine-tuning process, the YOLOv8s model can better utilize these features for bounding box prediction, thereby improving the accuracy and robustness of the YOLOv8s model in the overall bounding box task of the power equipment in the labeled splicing image.

[0017] Optionally, the method further comprises: after updating the parameters of the backbone network to the pre-constructed YOLOv8s model, freezing the parameters of the backbone network in the pre-constructed YOLOv8s model when performing the pre-training step.

[0018] The application trains the backbone network through SimCLR style contrast learning, so that it learns feature representations with good discrimination. Freezing the parameters of the backbone network can prevent the high-quality features obtained in the contrast learning from being destroyed or changed by subsequent training processes. Freezing the parameters of the backbone network means that only the parameters of the neck network and the detection head need to be updated during the pre-training phase, reducing the number of parameters that need to be trained, accelerating the convergence speed of pre-training, saving training time and computing resources. Moreover, during the backpropagation process of pre-training, the gradient only needs to be transmitted to the neck network and the detection head, and the gradient propagation path is more stable and direct, which helps the YOLOv8s model to adjust the parameters of the neck network and the detection head more quickly, so that the YOLOv8s model can better adapt to the pre-training dataset and accelerate the convergence process of the entire YOLOv8s model.

[0019] Optionally, after determining the coordinate range of the minimum bounding rectangle and the coordinate range of the sub-image, the method further comprises: setting an initial confidence for each minimum bounding rectangle, performing image segmentation on the splicing image labeled with the overall bounding box using a region segmentation algorithm, screening image blocks containing device components, and obtaining the coordinate range of each image block; determining whether the coordinate range of the i-th image block falls into the coordinate range of the corresponding minimum bounding rectangle, if yes, increasing the confidence of the minimum bounding rectangle to which the i-th image block belongs; if no, decreasing the confidence of the minimum bounding rectangle to which the i-th image block belongs, and deleting the minimum bounding rectangle whose confidence is less than a preset confidence threshold.

[0020] The present application adopts a region segmentation algorithm to segment the spliced image labeled with an overall bounding box, and further screens out an image block containing a device component. The overall bounding box region which may originally contain multiple device components or background interference is further divided into more accurate image blocks where the device components are located.

[0021] The present application dynamically increases or decreases the confidence of the minimum bounding rectangle by judging the relationship between the coordinate range of the image block and the coordinate range of the minimum bounding rectangle. When the coordinate range of the image block falls into the minimum bounding rectangle, it indicates that the minimum bounding rectangle may accurately contain the device component, and the confidence thereof is increased. Conversely, the confidence is decreased. The above scheme can screen and optimize the minimum bounding rectangle according to the actual image segmentation result, delete the minimum bounding rectangle whose confidence is less than a preset confidence threshold, and thus remove those bounding boxes which may contain false labeling or irrelevant regions, thereby improving the accuracy of the final labeling result.

[0022] Optionally, if there are overlapping minimum bounding rectangles in the subgraph, the method further comprises: retaining the minimum bounding rectangle with the highest confidence, and updating the remaining minimum bounding rectangles into the shape of the component contour.

[0023] When the minimum bounding rectangles of two components overlap, false labeling information may be generated, which interferes with the identification and analysis of the target component. The present application retains the rectangle box with high confidence, effectively reducing the possibility of false labeling. The overlapping rectangle box may make the definition of the target range become ambiguous, affecting the accurate understanding of the content of the subgraph. After reducing the overlap, the minimum bounding rectangle of each target can be more accurately determined, so as to accurately judge whether the subgraph contains these complete target ranges, because judging whether the subgraph contains the complete minimum bounding rectangle needs a series of calculation and comparison operations, such as judging whether the vertex coordinates of the rectangle box are within the range of the subgraph. When there are a large number of overlapping rectangle boxes, the calculation process will become complex and time-consuming. After reducing the overlapping rectangle boxes, the number of rectangle boxes to be processed is reduced, and the calculation amount is also correspondingly reduced, thereby improving the judgment efficiency.

[0024] Optionally, the historical image contained in the spliced image is recorded as a first image, and the method further comprises: extracting key points and SIFT descriptors in the first image using the SIFT algorithm, generating initial matching point pairs based on the key points and SIFT descriptors through a nearest neighbor matching strategy; The RANSAC algorithm is used to screen inliers in the initial matching point pairs, a homography matrix between the first images is obtained, the first images are subjected to perspective transformation according to the homography matrix, a new first image is obtained, and the new first image is subjected to splicing processing to obtain a new spliced image.

[0025] The SIFT algorithm can detect key points in the image with scale invariance and rotation invariance, and the SIFT descriptor can effectively describe the local feature information around the key points. Since the SIFT descriptor has high uniqueness, the occurrence of false matching can be reduced, and the accuracy of the initial matching point pairs can be improved. The nearest neighbor matching strategy quickly finds the most similar matching point pair by calculating the distance between the key point descriptors, so that potential matching points can be quickly screened from a large number of key points. The RANSAC algorithm can accurately estimate the homography matrix between the first images by means of random sampling and model verification, thereby improving the accuracy of geometric transformation. The first images are subjected to perspective transformation according to the homography matrix estimated by the RANSAC algorithm, so that the images taken from different perspectives can be accurately aligned. The spliced image generated by the above scheme is more natural in vision and has no obvious splicing traces, and can preserve the details and features of the original first images.

[0026] Optionally, the method further comprises: The gradient field of the new first image is calculated by using the Sobel operator, and a Poisson equation is constructed by taking the gradient field as a constraint condition, and the pixel values of the splicing boundary region are optimized by using the least square method.

[0027] The Sobel operator can effectively calculate the gradient field of the image. It obtains the gradient information of the new first image in two directions by respectively performing convolution operations on the image in the horizontal and vertical directions, and then constructs a complete gradient field. In the present application, the Poisson equation is constructed by taking the gradient field as a constraint, and the pixel values of the splicing boundary region are optimized by using the least square method, so that the pixel values on both sides of the splicing boundary can be smoothly transitioned.

[0028] In a second aspect, the present application provides a power equipment intelligent identification system based on deep learning, which adopts the following technical scheme: A power equipment intelligent identification system based on deep learning, comprising a memory and a processor, The memory stores a computer readable storage medium; The processor processes the computer program stored on the computer readable storage medium to implement the method of the first aspect.

[0029] In summary, the present application has at least one of the following beneficial technical effects: 1. The present application effectively expands the training data set by stitching historical images to obtain stitched images, and performing operations such as cropping sub-images in the stitched images, effectively increasing the diversity and richness of the data, helping the classification model to learn more comprehensive image features, improving the classification accuracy and generalization ability of the classification model, so that it can make accurate classification when facing different scenes and different states of power equipment images.

[0030] 2. The classification model constructed by the present application can quickly classify the collected images of power equipment, realizing real-time monitoring of the state of the power equipment. By timely outputting the classification result, the present application can help the operation and maintenance personnel to quickly understand the running status of the equipment, timely discover potential problems, and provide strong support for the maintenance and management of the power equipment, which helps to improve the stability and reliability of the power system.

[0031] 3. The present application performs SimCLR style enhancement processing on each target image to generate enhanced image pairs, thereby creating new images similar to the original images but with differences. In practical applications, the images of power equipment may be affected by various factors, such as changes in lighting, different shooting angles, and stains on the surface of the equipment. SimCLR style enhancement processing can simulate these changes in real scenarios, allowing the YOLOv8s model to be exposed to more image samples similar to actual situations during the training process. The present application generates enhanced image pairs, increasing the diversity of the data, which helps the YOLOv8s model to learn more generalizable features, and tries to avoid the YOLOv8s model from overfitting to specific image features, so that the YOLOv8s model can better adapt to images under different scenes and conditions. The present application improves the feature extraction capability of the backbone network through contrastive learning, thereby providing better feature input for the bounding box labeling task of the entire YOLOv8s model. In the subsequent pre-training and fine-tuning process, the YOLOv8s model can better utilize these features for bounding box prediction, thereby improving the accuracy and robustness of the YOLOv8s model in labeling the overall bounding box of the power equipment in the stitched image. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 is a flowchart of embodiment 1 of the present application; Figure 2 is a flowchart of embodiment 2 of the present application; Figure 3 is a flowchart of embodiment 3 of the present application; Figure 4 is a flowchart of embodiment 4 of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described in detail below. Figures 1 to 4 ​

[0034] Embodiment 1: The embodiment discloses a power equipment intelligent identification method based on deep learning, referring to Figure 1 , the method comprises: S11 data acquisition and processing, S12 classification, first, the real-time image and the historical image of the power equipment are collected, the historical image is spliced into a spliced image, a sub-image with the same size as the historical image is randomly intercepted in the spliced image, a training label is added to the historical image and the sub-image to obtain training data; then a classification model is constructed by using a deep learning algorithm, after training the classification model with the training data, the real-time image is input into the trained classification model to output a classification result, the execution process of each step of the embodiment is as follows: S11 data acquisition and processing, the real-time image of the power equipment is acquired through the AR device worn by the maintenance personnel, and the historical image of the power equipment / the type of the power equipment is recalled from the database based on the unique identifier on the power equipment.

[0035] In the embodiment, all the historical images are of the same size, and the historical images are subjected to arbitrary splicing processing to obtain a spliced image, the splicing processing includes left-right splicing, up-down splicing, and square splicing.

[0036] A sub-image with the same size as the historical image is randomly intercepted in the spliced image by using the Monte Carlo sampling method, and a training label is sequentially added to the historical image and the sub-image, and the historical image with the added training label and the sub-image with the added training label are used as training data.

[0037] The training label includes normal, rust, oil leakage, loose wiring, abnormal instrument, and abnormal switch position.

[0038] In other embodiments, a sub-image can also be intercepted in the spliced image according to a fixed step sliding window.

[0039] S12 classification, a classification model is constructed by using a deep learning algorithm, and the classification model can be a CNN model, a ResNet model, etc.

[0040] The training data in S11 data acquisition and processing is divided into a training set, a validation set, and a test set according to a ratio of 7:2:1, the training set is used to train the classification model, the validation set is used to verify whether the classification result of the trained classification model is correct, and the test set is used to test the training effect of the classification model.

[0041] In the training process, the Adam optimizer is used in the embodiment, the initial learning rate is set to 0.001, the learning rate is decayed to avoid overfitting, the cross-entropy loss function is used as the loss function, the training rounds are set to 50-100 rounds, and the stopping criterion is that the validation accuracy no longer improves or reaches 100 iterations, to obtain the trained classification model.

[0042] Input the real-time image into the trained classification model, and display the classification result on the interactive interface of the AR device.

[0043] The embodiment realizes the function of expanding the training sample set by converting the limited historical images into massive diversified samples through splicing and sampling, so that the classification model can fully learn the common features and individual difference features of the power equipment, and the classification result accuracy is improved.

[0044] Embodiment 2: Refer to Figure 2 The difference between the embodiment and embodiment 1 is that the power equipment includes multiple equipment components, and after randomly intercepting a sub-image with the same size as the historical image in the spliced image, the method further includes: S21 constructs a YOLOv8s model, constructs a YOLOv8s model including a backbone network, a neck network and a detection head, and the structures of each part are introduced as follows: The backbone network of the YOLOv8s model adopts the CSPDarknet structure, which realizes efficient feature extraction through cross-stage local connection design, includes 5 convolution modules and 4 CSP modules, and gradually down-samples the input image to 8x8 size to generate feature maps (P3, P4, P5) of different scales, which correspond to the feature information of small, medium and large targets, respectively.

[0045] The neck network of the YOLOv8s model adopts the PAN-FPN (Path Aggregation Network - FeaturePyramid Network) structure, which realizes multi-scale feature fusion, transmits high-level semantic information through the top-down feature pyramid (FPN), and supplements low-level positioning details through the bottom-up path aggregation (PAN), and finally outputs three fused feature maps of different scales.

[0046] The detection head of the YOLOv8s model adopts the decoupled head design, which separates the classification and regression tasks into two parallel branches, each branch includes 3 convolution layers, and outputs target class probability, bounding box coordinate offset and target confidence, respectively.

[0047] S22 generates a label, acquires an image of a power equipment included in the ImageNet-X dataset, denoted as a target image, uses a labeling tool (LabelImg or LabelStudio tool) to label the overall bounding box of the power equipment included in the target image, and generates a YOLO format label, which includes: normalized class ID, normalized horizontal coordinate of center coordinate, normalized vertical coordinate of center coordinate, normalized width, and normalized height.

[0048] For example, the bounding box of the transformer (category ID = 0) in the historical image has the upper-left corner coordinates (100, 150) and the lower-right corner coordinates (500, 400), and the image size is 640x640. The YOLO format label is: 0, 0.4688, 0.4375, 0.6250, 0.3906.

[0049] S23 integrates the data set, and generates two related but not identical enhanced images for each target image, that is, an enhanced image pair. The enhancement operation includes: Geometric transformation: random cropping, random horizontal or vertical flipping, random rotation; Pixel transformation: randomly adjusting brightness (±20%), contrast (±20%), saturation (±20%), and randomly adding Gaussian noise (σ≤0.1); Professional enhancement: simulating common disturbances in the environment of power equipment, such as adding dust blur, light spot shielding, and device partial reflection effect.

[0050] All enhanced image pairs are named according to the original image ID_enhancement method 1 and the original image ID_enhancement method 2, and are uniformly stored in the contrast learning data set to form positive sample pairs (i.e., different enhanced images of the same power equipment) and implicit negative sample pairs (i.e., enhanced images of different power equipment).

[0051] S24 constructs a contrast learning model. The contrast learning model includes an encoder and a projection head. The encoder is the backbone network of the YOLOv8s model, and the encoder is responsible for converting the input image into a 512-dimensional feature vector. The projection head includes two layers of MLP. The first layer of MLP maps the 512-dimensional feature to 256-dimensional, and the second layer of MLP further maps to 128-dimensional, and outputs a feature vector used to calculate the contrast loss.

[0052] The enhanced image pair is used to iteratively train the contrast learning model until the preset stopping condition is met, and the training is stopped. The parameters of the backbone network are updated to the pre-constructed YOLOv8s model, and S25 pre-training is performed.

[0053] The preset stopping condition includes: the number of training times reaches a preset number threshold or the accuracy of the contrast learning model reaches a preset threshold.

[0054] S25 pre-trains. The parameters of the backbone network in the pre-constructed YOLOv8s model are frozen. The target image pair with the YOLO format label is used to pre-train the YOLOv8s model with the frozen backbone network parameters to obtain the pre-trained YOLOv8s model.

[0055] S26 fine-tuning, freezing the parameters of the first m layers of the backbone network in the pre-trained YOLOv8s model (in this embodiment, the value of m is 10) and the parameters of the downsampling path of the neck network, obtaining n spliced images labeled with overall bounding boxes as a training set, which covers various combination modes such as left-right splicing, top-bottom splicing, and cross splicing, simulating the scene of multiple devices in the same frame in actual maintenance.

[0056] In this embodiment, a part of the spliced images are labeled with overall bounding boxes, and they are integrated into the training set, that is, n is less than the total number of spliced images.

[0057] The parameters of the unfrozen part of the pre-trained YOLOv8s model are fine-tuned using the training set, and a migration-trained YOLOv8s model is obtained. The bounding box positioning accuracy (IoU, intersection over union) of the migration-trained YOLOv8s model for power equipment in historical images can reach more than 85%.

[0058] S27 model update, updating the migration-trained YOLOv8s model to the pre-constructed YOLOv8s model.

[0059] S28 calculate the minimum bounding rectangle, label the overall bounding box of the power equipment in the spliced image using the migration-trained YOLOv8s model in S27 model update, and obtain the spliced image labeled with the overall bounding box.

[0060] The region within the overall bounding box labeled by the migration-trained YOLOv8s model is subjected to semantic segmentation processing, and the segmentation mask of the device components is obtained. The component contour is extracted in the segmentation mask using the Canny edge detection algorithm, and the minimum bounding rectangle of the contour is calculated using the Rotating Calipers algorithm. The four vertex coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) and the rotation angle (i.e. the angle with the horizontal axis) of the minimum bounding rectangle are output.

[0061] S29 delete subgraph, establish a pixel-level coordinate system with the upper left corner of the spliced image as the origin (0, 0), the horizontal axis to the right as the horizontal axis, and the vertical axis downward as the vertical axis. The coordinates of the bounding box, the segmentation mask, and the vertex coordinates of the minimum bounding rectangle are mapped to this coordinate system, and the position range of the subgraph in this coordinate system is determined. For the subgraphs generated by Monte Carlo sampling in S11 data collection and processing, it is judged whether all the minimum bounding rectangles of the device components contained therein are complete, and all subgraphs containing incomplete minimum bounding rectangles are deleted.

[0062] The standard for all the minimum bounding rectangles of the device components contained in the subgraph to be complete is that the four vertex coordinates of the minimum bounding rectangle all fall within the coordinate range of the subgraph.

[0063] By adopting the above scheme, the subgraph is screened in the embodiment, the quality of the subgraph is improved, the occupation of the calculation resources in the subsequent steps is reduced, and the training efficiency and accuracy of the classification model are improved.

[0064] Embodiment 3: Refer to Figure 3 The difference between the embodiment and the embodiment 2 is that, after the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph are determined, the method further includes the following steps: S31 image segmentation, setting an initial confidence for each minimum bounding rectangle, in the embodiment, the initial confidence of each minimum bounding rectangle is the same, which is 0.8.

[0065] In other embodiments, the initial confidence of the minimum bounding rectangle can also be set by the following scheme: If the semantic segmentation IoU of a certain device component is greater than or equal to 90%, the initial confidence is set to any value in the interval of 0.8-0.9; If the semantic segmentation IoU of a certain device component is between 80%-90% (including 80% but not including 90%), the initial confidence is set to any value in the interval of 0.6-0.7; If the semantic segmentation IoU of a certain device component is less than 80%, the initial confidence is set to any value in the interval of 0.4-0.5.

[0066] The region segmentation algorithm is adopted to perform image segmentation on the spliced image labeled with the overall bounding box, the image block containing the device component is screened out, the pixel-level coordinate system constructed in the subgraph is deleted based on S29, and the coordinate range of each image block is obtained.

[0067] S32 updating the confidence, judging whether the coordinate range of the i-th image block falls into the coordinate range of the corresponding minimum bounding rectangle, if the coordinate range of the i-th image block completely falls into the coordinate range of a minimum bounding rectangle, increasing the confidence of the minimum bounding rectangle to which the i-th image block belongs, and the increasing amplitude is 5%; otherwise, decreasing the confidence of the minimum bounding rectangle to which the i-th image block belongs, and the decreasing amplitude is 10%, and deleting the minimum bounding rectangle with a confidence less than a pre-set confidence threshold.

[0068] In the subgraph, the minimum bounding rectangles of multiple device components may overlap due to the dense distribution of the device components (if the semantic segmentation IoU of two minimum bounding rectangles is greater than or equal to 30%, it is considered that the two minimum bounding rectangles are in an overlapping state), if directly retained, it will cause repeated labeling or feature confusion, the embodiment will retain the minimum bounding rectangle with the maximum confidence, and update the remaining minimum bounding rectangles to the shape of the component contour.

[0069] The embodiment removes the rectangle generated by the semantic segmentation misjudgment through region segmentation and confidence adjustment, and improves the accuracy of device component positioning. The high-confidence rectangle retained in the embodiment and the modified component contour can more accurately determine whether a subgraph contains a complete component, thereby improving the quality of samples in the classification model training process.

[0070] Embodiment 4: Reference Figure 4 The embodiment differs from embodiment 1 in that the historical images contained in the spliced image are denoted as first images, and the method further includes the following steps: S41 matching, SIFT algorithm is used to extract key points and SIFT descriptors in the first image. For the kth SIFT descriptor in a certain first image, the Euclidean distance of the kth SIFT descriptor from all SIFT descriptors of another first image is calculated, the descriptor with the smallest Euclidean distance in the other first image is taken as the nearest neighbor of the kth SIFT descriptor, and the corresponding two key points constitute a set of initial matching point pairs.

[0071] Four groups of non-collinear points are randomly selected from the initial matching point pairs, and a homography matrix is solved using a DLT (Direct Linear Transform) algorithm. The projection error of all matching points after perspective transformation is calculated, and the points with projection error less than a preset error threshold are regarded as inliers.

[0072] S42 iteration, the operations contained in S41 matching are performed on all matching points to obtain all inliers. The position of a certain first image in another first image is calculated using the homography matrix, the transformed first image is generated using reverse mapping and bilinear interpolation, and the transformed first image is denoted as a new first image. The new first image is spliced to obtain a new spliced image.

[0073] S43 optimizing pixel value, the gradient field of the new first image is calculated using a Sobel operator, and a Poisson equation is constructed using the gradient field as a constraint condition. The Poisson equation is converted into a sparse linear system, and the pixel value of the splicing boundary region is solved by a least square method. The pixel value of the splicing boundary region in the spliced image is replaced by the solved pixel value.

[0074] The embodiment can realize natural transition of the splicing boundary and improve the visual effect while maintaining the calculation efficiency by unifying the coordinate values of the first images and optimizing the pixel values at the splicing positions.

[0075] Embodiment 5: The embodiment discloses an intelligent power equipment identification system based on deep learning, which comprises a memory and a processor, The memory stores a computer readable storage medium; The processor processes a computer program stored on the computer readable storage medium to implement the deep learning-based power equipment intelligent identification method.

[0076] The above are preferred embodiments of the present application, and do not limit the protection scope of the present application, so: all equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for intelligent identification of power equipment based on deep learning, characterized in that, include: Real-time and historical images of power equipment are collected. The historical images are stitched together to obtain a stitched image. A sub-image of the same size as the historical image is randomly cropped from the stitched image. Training labels are added to the historical image and the sub-image to obtain training data. A classification model is built using deep learning algorithms. The model is trained using training data to obtain the trained classification model. Real-time images are input into the trained classification model to output the classification results.

2. The intelligent identification method for power equipment based on deep learning according to claim 1, characterized in that, The power equipment comprises multiple equipment components. After randomly selecting a sub-image of the same size as a historical image from the stitched image, the method further includes: The overall bounding boxes of power equipment in the stitched image are labeled using a pre-built YOLOv8s model to obtain a stitched image with labeled overall bounding boxes. Semantic segmentation processing is performed on the stitched image with labeled overall bounding boxes to obtain the segmentation mask of the equipment components. The component contours are extracted from the segmentation mask, and the minimum bounding rectangle is calculated based on the component contours. Construct a coordinate system, determine the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, and delete the subgraphs containing incomplete minimum bounding rectangles based on the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraphs.

3. The intelligent identification method for power equipment based on deep learning according to claim 2, characterized in that, The YOLOv8s model includes a backbone network for feature extraction, a neck network for feature fusion, and a detection head for predicting overall bounding boxes. Before annotating the overall bounding boxes of electrical equipment contained in the stitched image using a pre-built YOLOv8s model, the method further includes: Generate labels: Obtain images of electrical equipment contained in the ImageNet-X dataset, denote them as target images, use annotation tools to annotate the overall bounding boxes of electrical equipment contained in the target images, and generate YOLO format labels; Pre-training: The pre-built YOLOv8s model is pre-trained using target images labeled with YOLO format to obtain the pre-trained YOLOv8s model; Fine-tuning: Freeze the parameters of the first m layers of the backbone network and the parameters of the downsampling path of the neck network in the pre-trained YOLOv8s model, obtain n stitched images labeled with the overall bounding box as the training set, and use the training set to fine-tune the parameters of the unfrozen part of the pre-trained YOLOv8s model to obtain the YOLOv8s model after transfer training. Model update: Update the transfer-trained YOLOv8s model to a pre-built YOLOv8s model.

4. The intelligent identification method for power equipment based on deep learning according to claim 3, characterized in that, After performing the label generation step and before performing the pre-training step, the method further includes: Each target image is enhanced using SimCLR-style enhancement processing to obtain an enhanced image pair for each target image; A contrastive learning model is constructed, comprising an encoder and a projector head. The encoder is the backbone network of the YOLOv8s model, and the projector head comprises a 2-layer MLP. The contrastive learning model is iteratively trained using augmented image pairs until a preset stopping condition is met. Then, the training stops, the parameters of the backbone network are output, and the backbone network parameters are updated in the pre-constructed YOLOv8s model to perform the pre-training step.

5. The intelligent identification method for power equipment based on deep learning according to claim 4, characterized in that, The method further includes: after updating the backbone network parameters to the pre-built YOLOv8s model, freezing the parameters of the backbone network in the pre-built YOLOv8s model during the pre-training step.

6. The intelligent identification method for power equipment based on deep learning according to any one of claims 2-5, characterized in that, After determining the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, the method further includes: An initial confidence level is set for each minimum bounding rectangle. A region segmentation algorithm is used to segment the stitched image labeled with the overall bounding box. Image blocks containing device components are filtered out, and the coordinate range of each image block is obtained. Determine whether the coordinate range of the i-th image patch falls within the coordinate range of the corresponding minimum bounding rectangle. If yes, increase the confidence of the minimum bounding rectangle to which the i-th image patch belongs; otherwise, decrease the confidence of the minimum bounding rectangle to which the i-th image patch belongs and delete the minimum bounding rectangle with a confidence value less than the preset confidence threshold.

7. The intelligent identification method for power equipment based on deep learning according to claim 6, characterized in that, If there are overlapping minimum bounding rectangles in the subgraph, the method further includes: retaining the minimum bounding rectangle with the highest confidence and updating the remaining minimum bounding rectangles to the shape of the component outline.

8. The intelligent identification method for power equipment based on deep learning according to any one of claims 1-5, characterized in that, The method further includes designating the historical images contained in the stitched image as the first image, and also includes: The SIFT algorithm is used to extract key points and SIFT descriptors from the first image. Based on the key points and SIFT descriptors, initial matching point pairs are generated through the nearest neighbor matching strategy. The RANSAC algorithm is used to filter the inliers in the initial matching point pairs to obtain the homography matrix between the first images. The first images are then subjected to perspective transformation based on the homography matrix to obtain a new first image. The new first images are then stitched together to obtain a new stitched image.

9. The intelligent identification method for power equipment based on deep learning according to claim 8, characterized in that, The method further includes: The gradient field of the new first image is calculated using the Sobel operator. The Poisson equation is constructed using the gradient field as a constraint, and the pixel values ​​of the stitching boundary region are optimized by the least squares method.

10. A deep learning-based intelligent identification system for power equipment, characterized in that, include: Memory and processor The memory contains a computer-readable storage medium; When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in any one of claims 1-9.

Images