A deep learning-based power equipment intelligent identification method and system

Abstract

This application provides a method and system for intelligent identification of power equipment based on deep learning, relating to the technical field of equipment identification. The method includes: firstly, acquiring real-time and historical images of the power equipment; stitching the historical images together to form a stitched image; randomly cropping a sub-image of the same size as the historical image from the stitched image; adding training labels to the historical image and the sub-image to obtain training data; then constructing a classification model using a deep learning algorithm; training the classification model with the training data; and finally inputting the real-time image into the trained classification model to output the classification result. This application can improve the accuracy of power equipment identification results even with limited training samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of device recognition, and in particular to a power equipment intelligent recognition method and system based on deep learning. BACKGROUND

[0002] In the daily operation and management of the power system, accurate identification of power equipment is a key link to ensure the safe and stable operation of the power grid. With the continuous expansion and complication of the power network, the types and quantities of power equipment are increasing, and the distribution range is becoming more extensive. The traditional manual inspection method is not only inefficient, but also requires a lot of manpower, material resources and time. Moreover, due to the subjectivity and limitations of manual judgment, it is easy to miss or misjudge, which cannot meet the high requirements of modern power systems on the accuracy and timeliness of equipment identification. Therefore, it is an important problem to be solved in the power industry to realize efficient and accurate intelligent identification of power equipment, which is of great significance to improve the operation and management level of the power system and ensure the reliability of power supply.

[0003] To solve the problem of power equipment identification, a variety of technical means have been proposed and applied. Among them, the traditional method based on image processing is a common technology. This kind of method mainly extracts features such as color, shape and texture from power equipment images, and then uses pattern recognition algorithms to analyze and classify the extracted features, so as to realize the identification of power equipment. In addition, with the rapid development of artificial intelligence technology, methods based on machine learning have also been gradually applied in the field of power equipment identification. Some researches use traditional machine learning algorithms such as support vector machine (SVM) and decision tree to build classification models for classification and identification of power equipment images.

[0004] In recent years, the rise of deep learning technology has brought new ideas and methods to power equipment identification. Deep learning algorithms have strong feature learning and expression capabilities, and can automatically learn effective feature representations from a large number of image data. However, there are deficiencies in training data acquisition and processing, and it is difficult to obtain a large amount of high-quality and accurately labeled power equipment image data, which increases the cost and time of data collection. SUMMARY

[0005] In order to improve the accuracy of power equipment identification results under the condition of limited training samples, the present application provides a power equipment intelligent recognition method and system based on deep learning.

[0006] In the first aspect, the present application provides a power equipment intelligent recognition method based on deep learning, which adopts the following technical scheme:

[0007] A power equipment intelligent recognition method based on deep learning, comprising the following steps:

[0008] collecting real-time images and historical images of power equipment, performing stitching processing on the historical images to obtain a stitched image, randomly intercepting a sub-image of the same size as the historical image in the stitched image, adding training labels to the historical image and the sub-image to obtain training data;

[0009] constructing a classification model using a deep learning algorithm, training the classification model using the training data to obtain a trained classification model, inputting the real-time image into the trained classification model, and outputting a classification result.

[0010] The present application effectively expands the training data set, increases the diversity and richness of the data, helps the classification model to learn more comprehensive image features, improves the classification accuracy and generalization ability of the classification model, and enables accurate classification when facing different scenarios and different states of power equipment images. The classification model constructed by the present application can quickly classify the collected real-time images of power equipment, realizing real-time monitoring of the state of the power equipment. By timely outputting the classification result, the present application can help the operation and maintenance personnel to quickly understand the operating condition of the power equipment, timely discover potential problems, provide strong support for the maintenance and management of the power equipment, and help to improve the stability and reliability of the power system.

[0011] Optionally, the power equipment includes a plurality of equipment components, and after randomly intercepting a sub-image of the same size as the historical image in the stitched image, the method further includes:

[0012] using a pre-constructed YOLOv8s model to label the overall bounding box of the power equipment in the stitched image to obtain a stitched image labeled with an overall bounding box, performing semantic segmentation processing on the stitched image labeled with the overall bounding box to obtain a segmentation mask of the equipment components, extracting a component contour in the segmentation mask, and calculating a minimum bounding rectangle according to the component contour.

[0013] constructing a coordinate system, determining the coordinate range of the minimum bounding rectangle and the coordinate range of the sub-image, and deleting the sub-image containing the incomplete minimum bounding rectangle based on the coordinate range of the minimum bounding rectangle and the coordinate range of the sub-image.

[0014] The present application retains the sub-image containing the complete equipment component, so that the training data focuses more on the effective equipment component, which helps the classification model to better learn the feature pattern of the equipment component and improves the training efficiency and classification accuracy of the classification model. After deleting part of the sub-image, the amount of data to be processed in the training process of the classification model is reduced, the consumption of computing resources is reduced, and the training speed is improved.

[0015] Optionally, the YOLOv8s model includes a backbone network for feature extraction, a neck network for feature fusion, and a detection head for predicting the overall bounding box. Before annotating the overall bounding boxes of electrical equipment contained in the stitched image using the pre-built YOLOv8s model, the method further includes:

[0016] Generate labels: Obtain images of electrical equipment contained in the ImageNet-X dataset, denote them as target images, use annotation tools to annotate the overall bounding boxes of electrical equipment contained in the target images, and generate YOLO format labels;

[0017] Pre-training: The pre-built YOLOv8s model is pre-trained using target images labeled with YOLO format to obtain the pre-trained YOLOv8s model;

[0018] Fine-tuning: Freeze the parameters of the first m layers of the backbone network and the parameters of the downsampling path of the neck network in the pre-trained YOLOv8s model, obtain n stitched images labeled with the overall bounding box as the training set, and use the training set to fine-tune the parameters of the unfrozen part of the pre-trained YOLOv8s model to obtain the YOLOv8s model after transfer training.

[0019] Model update: Update the transfer-trained YOLOv8s model to a pre-built YOLOv8s model.

[0020] This application uses annotation tools to annotate the overall bounding boxes of power equipment images in the ImageNet-X dataset and generate YOLO format labels. This provides clear target information for model training. The YOLO format labels are concise and easy for YOLOv8s models to analyze, enabling them to quickly understand the location of power equipment in the images. This allows them to learn the mapping relationship between features and bounding box locations in a targeted manner, which helps the YOLOv8s model converge quickly in the early stages of training. The ImageNet-X dataset contains a large number of power equipment images under different scenes, angles, and lighting conditions. By utilizing this dataset to generate YOLO format labels, this application exposes the YOLOv8s model to diverse power equipment image samples, enabling it to learn a wider range of feature representations, enhancing its adaptability to various complex situations, and improving its generalization ability.

[0021] This application uses target images labeled with YOLO format to pre-train a pre-built YOLOv8s model, enabling the YOLOv8s model to learn some basic features and patterns of power equipment before formal training. During subsequent training, the YOLOv8s model can learn more efficiently based on this existing knowledge, reducing the number of training iterations, accelerating convergence, and saving training time and computational resources. In the fine-tuning step, this application uses stitched images labeled with overall bounding boxes as the training set. The YOLOv8s model can learn the unique features and bounding box distribution patterns of power equipment in the stitched images, improving the model's annotation accuracy on the stitched images.

[0022] Optionally, after performing the label generation step and before performing the pre-training step, the method further includes:

[0023] Each target image is enhanced using SimCLR-style enhancement processing to obtain an enhanced image pair for each target image;

[0024] A contrastive learning model is constructed, comprising an encoder and a projector head. The encoder is the backbone network of the YOLOv8s model, and the projector head comprises a 2-layer MLP. The contrastive learning model is iteratively trained using augmented image pairs until a preset stopping condition is met. Then, the training stops, the parameters of the backbone network are output, and the backbone network parameters are updated in the pre-constructed YOLOv8s model to perform the pre-training step.

[0025] This application applies SimCLR-style enhancement processing to each target image, generating enhanced image pairs. This allows for the creation of new images that are similar to but different from the original images. In practical applications, images of power equipment may be affected by various factors, such as changes in lighting, different shooting angles, and dirt on the equipment surface. SimCLR-style enhancement processing can simulate these real-world variations, exposing the YOLOv8s model to more image samples from similar real-world scenarios during training. By generating enhanced image pairs, this application increases data diversity, helping the YOLOv8s model learn more generalizable features and minimizing overfitting to specific image features. This allows the YOLOv8s model to better adapt to power equipment images under different scenarios and conditions.

[0026] The contrastive learning model constructed in this application uses the backbone network of the YOLOv8s model as the encoder. By training the contrastive learning model, the backbone network can learn more discriminative feature representations. Contrastive learning is an unsupervised learning method that does not require manual annotation of additional information. It only needs to be trained by enhancing the similarity and dissimilarity between image pairs, which reduces the workload and cost of manual annotation to a certain extent. This application uses a large amount of unlabeled power equipment image data for training, further enriching the learning samples of the contrastive learning model and improving its performance.

[0027] This application enhances the feature extraction capability of the backbone network through contrastive learning, thereby providing higher-quality feature input for the bounding box annotation task of the entire YOLOv8s model. In subsequent pre-training and fine-tuning processes, the YOLOv8s model can better utilize these features for bounding box prediction, thus improving the accuracy and robustness of the YOLOv8s model in annotating the overall bounding boxes of power equipment in stitched images.

[0028] Optionally, the method further includes: after updating the backbone network parameters to the pre-built YOLOv8s model, freezing the parameters of the backbone network in the pre-built YOLOv8s model during the pre-training step.

[0029] This application trains the backbone network using SimCLR-style contrastive learning, enabling it to learn highly discriminative feature representations. Freezing the backbone network parameters ensures that the high-quality features acquired during contrastive learning in the pre-training phase are not destroyed or altered by subsequent training processes. Freezing the backbone network parameters means that only the parameters of the neck network and detector head need to be updated during the pre-training phase, reducing the number of parameters that need to be trained, accelerating the convergence speed of pre-training, and saving training time and computational resources. Moreover, during the backpropagation process in pre-training, the gradient only needs to be propagated to the neck network and detector head, making the gradient propagation path more stable and direct. This helps the YOLOv8s model adjust the parameters of the neck network and detector head more quickly, enabling the YOLOv8s model to better adapt to the pre-training dataset and accelerating the convergence process of the entire YOLOv8s model.

[0030] Optionally, after determining the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, the method further includes:

[0031] An initial confidence level is set for each minimum bounding rectangle. A region segmentation algorithm is used to segment the stitched image labeled with the overall bounding box. Image blocks containing device components are filtered out, and the coordinate range of each image block is obtained.

[0032] Determine whether the coordinate range of the i-th image patch falls within the coordinate range of the corresponding minimum bounding rectangle. If yes, increase the confidence of the minimum bounding rectangle to which the i-th image patch belongs; otherwise, decrease the confidence of the minimum bounding rectangle to which the i-th image patch belongs and delete the minimum bounding rectangle with a confidence value less than the preset confidence threshold.

[0033] This application uses a region segmentation algorithm to segment a stitched image labeled with an overall bounding box, and then filters out image blocks containing device components. The overall bounding box region, which may originally contain multiple device components or background interference, is further divided into more precise image blocks containing the device components.

[0034] This application dynamically increases or decreases the confidence level of the minimum bounding rectangle by judging the relationship between the coordinate range of the image patch and the coordinate range of the minimum bounding rectangle. When the coordinate range of the image patch falls within the minimum bounding rectangle, it indicates that the minimum bounding rectangle may accurately contain the device component, thus increasing its confidence level; otherwise, the confidence level is decreased. The above scheme can filter and optimize the minimum bounding rectangle based on the actual image segmentation results, and delete the minimum bounding rectangle with a confidence level less than the preset confidence threshold, thereby removing bounding boxes that may contain incorrect annotations or irrelevant regions and improving the accuracy of the final annotation results.

[0035] Optionally, if there are overlapping minimum bounding rectangles in the subgraph, the method further includes: retaining the minimum bounding rectangle with the highest confidence and updating the remaining minimum bounding rectangles to the shape of the component outline.

[0036] When the minimum bounding rectangles of two components overlap, erroneous annotation information may be generated, interfering with the identification and analysis of the target components. This application effectively reduces the possibility of mislabeling by retaining rectangles with high confidence. Overlapping rectangles may blur the definition of the target range, affecting the accurate understanding of the subgraph content. Reducing overlap allows for a more precise determination of the minimum bounding rectangle of each target, thus accurately judging whether the subgraph contains these complete target ranges. This is because determining whether a subgraph contains a complete minimum bounding rectangle requires a series of calculations and comparisons, such as determining whether the vertex coordinates of the rectangles are within the subgraph's range. When there are a large number of overlapping rectangles, the calculation process becomes complex and time-consuming. Reducing overlapping rectangles reduces the number of rectangles that need to be processed, and the computational load is correspondingly reduced, thereby improving the efficiency of the judgment.

[0037] Optionally, the historical image contained in the stitched image is designated as the first image, and the method further includes:

[0038] The SIFT algorithm is used to extract key points and SIFT descriptors from the first image. Based on the key points and SIFT descriptors, initial matching point pairs are generated through the nearest neighbor matching strategy.

[0039] The RANSAC algorithm is used to filter the inliers in the initial matching point pairs to obtain the homography matrix between the first images. The first images are then subjected to perspective transformation based on the homography matrix to obtain a new first image. The new first images are then stitched together to obtain a new stitched image.

[0040] The SIFT algorithm can detect keypoints in images that are scale-invariant and rotation-invariant. SIFT descriptors effectively describe the local features surrounding keypoints. Due to the high uniqueness of SIFT descriptors, false matches are reduced, improving the accuracy of initial matching point pairs. The nearest neighbor matching strategy quickly finds the most similar matching point pairs by calculating the distance between keypoint descriptors, thus rapidly filtering potential matching points from a large number of keypoints. The RANSAC algorithm, through random sampling and model validation, accurately estimates the homography matrix between the first images, improving the accuracy of geometric transformations. This application performs perspective transformation on the first image based on the homography matrix estimated by the RANSAC algorithm, accurately aligning images taken from different perspectives. The stitched image generated by the above scheme is visually more natural, without obvious stitching artifacts, and retains the details and features of the original first image.

[0041] Optionally, the method further includes:

[0042] The gradient field of the new first image is calculated using the Sobel operator. The Poisson equation is constructed using the gradient field as a constraint, and the pixel values ​​of the stitching boundary region are optimized by the least squares method.

[0043] The Sobel operator can effectively calculate the gradient field of an image. It obtains the gradient information of the new first image in both directions by performing convolution operations on the image in the horizontal and vertical directions, and then constructs a complete gradient field. This application constructs the Poisson equation with the gradient field as a constraint and uses the least squares method to optimize the pixel values ​​in the stitching boundary region, which can make the pixel values ​​on both sides of the stitching boundary achieve a smooth transition.

[0044] Secondly, this application provides a deep learning-based intelligent identification system for power equipment, employing the following technical solution:

[0045] A deep learning-based intelligent identification system for power equipment includes: a memory and a processor.

[0046] The memory contains a computer-readable storage medium;

[0047] When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in the first aspect.

[0048] In summary, this application includes at least one of the following beneficial technical effects:

[0049] 1. This application effectively expands the training dataset by stitching historical images to obtain stitched images and extracting sub-images from the stitched images. This increases the diversity and richness of the data, helps the classification model learn more comprehensive image features, improves the classification accuracy and generalization ability of the classification model, and enables it to make accurate classifications when faced with power equipment images in different scenarios and states.

[0050] 2. The classification model constructed in this application can quickly classify the collected images of power equipment, realizing real-time monitoring of the power equipment status. By outputting classification results in a timely manner, this application can help operation and maintenance personnel quickly understand the operating status of the equipment, promptly identify potential problems, provide strong support for the maintenance and management of power equipment, and help improve the stability and reliability of the power system.

[0051] 3. This application performs SimCLR-style enhancement processing on each target image to generate enhanced image pairs, thereby creating new images that are similar to but different from the original images. In practical applications, images of power equipment may be affected by various factors, such as changes in lighting, different shooting angles, and dirt on the equipment surface. SimCLR-style enhancement processing can simulate these changes in real-world scenarios, allowing the YOLOv8s model to encounter more image samples similar to real-world situations during training. By generating enhanced image pairs, this application increases the diversity of data, helping the YOLOv8s model learn more generalizable features and minimizing overfitting to specific image features, enabling the YOLOv8s model to better adapt to images under different scenes and conditions. This application improves the feature extraction capability of the backbone network through contrastive learning, thus providing higher-quality feature input for the bounding box annotation task of the entire YOLOv8s model. In subsequent pre-training and fine-tuning processes, the YOLOv8s model can better utilize these features for bounding box prediction, thereby improving the accuracy and robustness of the YOLOv8s model in the task of annotating the overall bounding boxes of power equipment in stitched images. Attached Figure Description

[0052] Figure 1 This is a flowchart of Embodiment 1 of this application;

[0053] Figure 2 This is a flowchart of Embodiment 2 of this application;

[0054] Figure 3 This is a flowchart of Embodiment 3 of this application;

[0055] Figure 4 This is a flowchart of Embodiment 4 of this application. Detailed Implementation

[0056] The following combination Figures 1 to 4 This application will be described in further detail.

[0057] Example 1: This example discloses a deep learning-based intelligent identification method for power equipment, referring to... Figure 1 The method includes: S11 data acquisition and processing, and S12 classification. First, real-time and historical images of the power equipment are acquired. The historical images are stitched together to form a stitched image. A sub-image of the same size as the historical image is randomly cropped from the stitched image. Training labels are added to the historical image and the sub-image to obtain training data. Then, a classification model is constructed using a deep learning algorithm. After training the classification model with the training data, the real-time image is input into the trained classification model to output the classification result. The execution process of each step in this embodiment is as follows:

[0058] S11 data acquisition and processing acquires real-time images of power equipment through AR devices worn by maintenance personnel, and retrieves historical images of that power equipment / type of power equipment from a pre-existing database based on the unique identifier of that power equipment.

[0059] In this embodiment, all historical images are the same size. These historical images are arbitrarily stitched together to obtain a stitched image. The stitching process includes left-right stitching, top-bottom stitching, and stitching in a grid pattern.

[0060] The Monte Carlo sampling method is used to randomly crop a sub-image of the same size as the historical image from the stitched image. Training labels are added to the historical image and the sub-image in turn. The historical image with training labels and the sub-image with training labels are used as training data.

[0061] The training labels include: normal, rust, oil leak, loose wiring, instrument malfunction, and abnormal switch position.

[0062] In other embodiments, sub-images can be extracted from the stitched image by sliding a window with a fixed step size.

[0063] S12 classification uses deep learning algorithms to build a classification model, which can be a CNN model, a ResNet model, etc.

[0064] The training data in S11 data acquisition and processing is divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used to train the classification model, the validation set is used to verify whether the classification results of the trained classification model are correct, and the test set is used to test the training effect of the classification model.

[0065] During training, this embodiment uses the Adam optimizer with an initial learning rate of 0.001. Overfitting is avoided by decaying the learning rate. The cross-entropy loss function is used, and the number of training rounds is set to 50-100. The stopping criterion is that the accuracy of the validation set no longer improves or reaches 100 iterations, thus obtaining the trained classification model.

[0066] Real-time images are input into the trained classification model, and the classification results are displayed on the interactive interface of the AR device.

[0067] This embodiment transforms limited historical images into a massive amount of diverse samples through stitching and sampling, thereby expanding the training sample set and enabling the classification model to fully learn the common and individual differences of power equipment, thus improving the accuracy of classification results.

[0068] Example 2: Refer to Figure 2 The difference between this embodiment and Embodiment 1 is that the power equipment includes multiple equipment components. After randomly selecting a sub-image of the same size as the historical image from the stitched image, the method further includes:

[0069] S21 constructs a YOLOv8s model, including a backbone network, a neck network, and a detection head. The structure of each part is described below:

[0070] The backbone network of the YOLOv8s model adopts the CSPDarknet structure. The CSPDarknet structure achieves efficient feature extraction through cross-stage local connectivity design. It contains 5 convolutional modules and 4 CSP modules, which progressively downsample the input image to an 8×8 size to generate feature maps of different scales (P3, P4, P5), corresponding to the feature information of small, medium and large targets, respectively.

[0071] The neck network of the YOLOv8s model adopts the PAN-FPN (Path Aggregation Network - Feature Pyramid Network) structure. The PAN-FPN structure realizes multi-scale feature fusion. It transmits high-level semantic information through the top-down feature pyramid (FPN) and supplements low-level localization details by combining the bottom-up path aggregation (PAN), and finally outputs three fused feature maps of different scales.

[0072] The YOLOv8s model's detection head uses a decoupled head design, separating the classification and regression tasks into two parallel branches. Each branch contains three convolutional layers, which output the target class probability, bounding box coordinate offset, and target confidence, respectively.

[0073] S22 generates labels by acquiring images of power equipment contained in the ImageNet-X dataset, denoted as target images, and using annotation tools (LabelImg or LabelStudio) to annotate the overall bounding boxes of the power equipment contained in the target images, generating YOLO format labels. The YOLO format labels include: normalized class ID, normalized x-coordinate of the center coordinate, normalized y-coordinate of the center coordinate, normalized width, and normalized height.

[0074] For example, in a historical image, the bounding box of a transformer (category ID=0) has the coordinates of the top left corner (100, 150) and the bottom right corner (500, 400), and the image size is 640×640. Then the YOLO format labels are: 0, 0.4688, 0.4375, 0.6250, 0.3906.

[0075] S23 integrates the dataset and generates two related but not identical enhanced images, i.e., enhanced image pairs, for each target image. The enhancement operations include:

[0076] Geometric transformations: random cropping, random horizontal or vertical flipping, random rotation;

[0077] Pixel transformation: Randomly adjust brightness (±20%), contrast (±20%), and saturation (±20%), and randomly add Gaussian noise (σ≤0.1);

[0078] Professional Enhancement: Simulates common interferences in the environment where power equipment is located, such as adding dust blurring, light spot obstruction, and localized reflection effects on the equipment.

[0079] All augmented image pairs are named according to the original image ID_augmentation method 1 and the original image ID_augmentation method 2, and stored uniformly in the contrastive learning dataset to form positive sample pairs (i.e. different augmented images of the same power equipment) and implicit negative sample pairs (i.e. augmented images of different power equipment).

[0080] S24 Constructs a contrastive learning model. The contrastive learning model includes an encoder and a projector head. The encoder is the backbone network of the YOLOv8s model. The encoder is responsible for converting the input image into a 512-dimensional feature vector. The projector head includes two MLP layers. The first MLP layers map the 512-dimensional features to 256 dimensions, and the second MLP layers further map them to 128 dimensions, outputting a feature vector used to calculate the contrastive loss.

[0081] The contrastive learning model is trained iteratively using enhanced image pairs until a preset stopping condition is met. Then, the training stops, the parameters of the backbone network are output, the backbone network parameters are updated into the pre-built YOLOv8s model, and S25 pre-training is performed.

[0082] The preset stopping conditions include: the number of training iterations reaching a preset threshold or the accuracy of the comparative learning model reaching a preset threshold.

[0083] S25 pre-training: Freeze the parameters of the backbone network in the pre-built YOLOv8s model, and use target images with YOLO format labels to pre-train the YOLOv8s model with frozen backbone network parameters to obtain the pre-trained YOLOv8s model.

[0084] S26 fine-tuning: Freeze the parameters of the first m layers of the backbone network in the pre-trained YOLOv8s model (in this embodiment, the value of m is 10) and the parameters of the downsampling path of the neck network, and obtain n stitched images labeled with the overall bounding box as the training set. The training set covers a variety of combinations such as left-right stitching, top-bottom stitching, and grid stitching to simulate the scenario of multiple devices in the same frame during actual maintenance.

[0085] In this embodiment, a portion of the stitched images are labeled with overall bounding boxes, and these are integrated into a training set, i.e., n is less than the total number of stitched images.

[0086] By fine-tuning the parameters of the unfrozen parts of the pre-trained YOLOv8s model using the training set, a transfer-trained YOLOv8s model is obtained. The transfer-trained YOLOv8s model can achieve a bounding box localization accuracy (IoU, intersection-over-union ratio) of over 85% for power equipment in historical images.

[0087] The S27 model update updates the transfer-trained YOLOv8s model to a pre-built YOLOv8s model.

[0088] S28 calculates the minimum bounding rectangle, and uses the YOLOv8s model trained in the S27 model update to annotate the overall bounding box of the power equipment in the stitched image, thus obtaining a stitched image with annotated overall bounding boxes.

[0089] Semantic segmentation is performed on the regions within the overall bounding box labeled by the YOLOv8s model after transfer training to obtain the segmentation mask of the device component. The Canny edge detection algorithm is used to extract the component contour from the segmentation mask. The Rotating Calipers algorithm is used to calculate the minimum bounding rectangle of the contour. The coordinates of the four vertices of the minimum bounding rectangle (x1,y1), (x2,y2), (x3,y3), (x4,y4) and the rotation angle, i.e., the angle with the horizontal axis, are output.

[0090] S29 Delete sub-images. Establish a pixel-level coordinate system with the top left corner of the stitched image as the origin (0,0), the horizontal axis to the right as the horizontal axis, and the vertical axis downward as the vertical axis. Map the coordinates of the bounding box, the segmentation mask, and the vertex coordinates of the minimum bounding rectangle to this coordinate system, and clarify the position range of the sub-image in this coordinate system. For the sub-images generated by Monte Carlo sampling in S11 data acquisition and processing, determine whether the minimum bounding rectangles of all the device components contained in each sub-image are complete, and delete all sub-images containing incomplete minimum bounding rectangles.

[0091] The complete standard for the minimum bounding rectangle of all device components contained in the subgraph is that the coordinates of all four vertices of the minimum bounding rectangle fall within the coordinate range of the subgraph.

[0092] By adopting the above scheme, this embodiment filters the subgraphs, improves the quality of the subgraphs, reduces the computational resource consumption of subsequent steps, and improves the training efficiency and accuracy of the classification model.

[0093] Example 3: Reference Figure 3 The difference between this embodiment and Embodiment 2 is that, after determining the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, the method further includes the following steps:

[0094] S31 Image segmentation sets an initial confidence level for each minimum bounding rectangle. In this embodiment, the initial confidence level for each minimum bounding rectangle is the same, which is 0.8.

[0095] In other embodiments, the initial confidence level for the minimum bounding rectangle can also be set using the following scheme:

[0096] If the semantic segmentation IoU of a certain device component is greater than or equal to 90%, the initial confidence level is set to any value in the range of 0.8-0.9;

[0097] If the semantic segmentation IoU of a certain device component is between 80% and 90% (including 80% but excluding 90%), the initial confidence level is set to any value in the range of 0.6 to 0.7.

[0098] If the semantic segmentation IoU of a certain device component is less than 80%, the initial confidence level is set to any value in the range of 0.4-0.5.

[0099] A region segmentation algorithm is used to segment the stitched image with an overall bounding box. Image blocks containing device components are selected. The pixel-level coordinate system constructed based on the S29 deletion sub-image is used to obtain the coordinate range of each image block.

[0100] S32 updates the confidence level by determining whether the coordinate range of the i-th image patch falls within the coordinate range of the corresponding minimum bounding rectangle. If the coordinate range of the i-th image patch falls completely within the coordinate range of a minimum bounding rectangle, the confidence level of the minimum bounding rectangle to which the i-th image patch belongs is increased by 5%. Otherwise, the confidence level of the minimum bounding rectangle to which the i-th image patch belongs is decreased by 10%, and minimum bounding rectangles with confidence levels less than the preset confidence threshold are deleted.

[0101] In the subgraph, the minimum bounding rectangles of multiple device components may overlap due to the dense distribution of device components (if the semantic segmentation IoU of two minimum bounding rectangles is greater than or equal to 30%, then the two minimum bounding rectangles are considered to be in an overlapping state). If they are directly retained, it will lead to duplicate annotation or feature confusion. In this embodiment, the minimum bounding rectangle with the highest confidence will be retained, and the remaining minimum bounding rectangles will be updated to the shape of the component outline.

[0102] This embodiment improves the accuracy of device component localization by eliminating rectangles generated by semantic segmentation misjudgments through region segmentation and confidence adjustment. The high-confidence rectangles and corrected component outlines retained in this embodiment can more accurately determine whether a sub-image contains a complete component, thus improving the quality of samples during the training of the classification model.

[0103] Example 4: Reference Figure 4 The difference between this embodiment and Embodiment 1 is that the historical image contained in the stitched image is designated as the first image, and the method further includes the following steps:

[0104] S41 matching uses the SIFT algorithm to extract key points and SIFT descriptors in the first image. For the k-th SIFT descriptor in a certain first image, the Euclidean distance between it and all SIFT descriptors in another first image is calculated. The descriptor with the smallest Euclidean distance in the other first image is taken as the nearest neighbor of the k-th SIFT descriptor, and the corresponding two key points form an initial matching point pair.

[0105] Four sets of non-collinear points are randomly selected from the initial matched point pairs. The homography matrix is ​​solved using the DLT (Direct Linear Transform) algorithm. The projection error after perspective transformation of all matched points is calculated. Points with projection errors less than a preset error threshold are considered interior points.

[0106] In iteration S42, the operations contained in matching S41 are performed on all matching points to obtain all interior points. The position of a certain first image in another first image is calculated using the homography matrix. The transformed first image is generated using inverse mapping and bilinear interpolation and is denoted as the new first image. The new first image is then stitched together to obtain a new stitched image.

[0107] S43 optimizes pixel values ​​by using the Sobel operator to calculate the gradient field of the new first image. Using the gradient field as a constraint, a Poisson equation is constructed and transformed into a sparse linear system. The pixel values ​​of the stitching boundary region are solved by the least squares method, and the solved pixel values ​​are used to replace the pixel values ​​of the stitching boundary region contained in the stitched image.

[0108] This embodiment unifies the coordinate values ​​of the first image and optimizes the pixel values ​​at the stitching point, which can achieve a natural transition of the stitching boundary while maintaining computational efficiency and improving the visual effect.

[0109] Example 5: This example discloses a deep learning-based intelligent identification system for power equipment. The system includes a memory and a processor.

[0110] The memory contains a computer-readable storage medium;

[0111] When the processor processes the computer program stored on the computer-readable storage medium, it implements the deep learning-based intelligent identification method for power equipment.

[0112] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for intelligent identification of power equipment based on deep learning, characterized in that, include: Real-time and historical images of power equipment are collected. The historical images are stitched together to obtain a stitched image. A sub-image of the same size as the historical image is randomly cropped from the stitched image. Training labels are added to the historical image and the sub-image to obtain training data. A classification model is built using deep learning algorithms. The classification model is trained using training data to obtain the trained classification model. Real-time images are input into the trained classification model to output the classification results. The power equipment comprises multiple equipment components. After randomly selecting a sub-image of the same size as a historical image from the stitched image, the method further includes: The overall bounding boxes of power equipment in the stitched image are labeled using a pre-built YOLOv8s model to obtain a stitched image with labeled overall bounding boxes. Semantic segmentation processing is performed on the stitched image with labeled overall bounding boxes to obtain the segmentation mask of the equipment components. The component contours are extracted from the segmentation mask, and the minimum bounding rectangle is calculated based on the component contours. Construct a coordinate system, determine the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, and delete the subgraph containing an incomplete minimum bounding rectangle based on the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph. After determining the coordinate range of the minimum bounding rectangle and the coordinate range of the subgraph, the method further includes: An initial confidence level is set for each minimum bounding rectangle. A region segmentation algorithm is used to segment the stitched image labeled with the overall bounding box. Image blocks containing device components are filtered out, and the coordinate range of each image block is obtained. Determine whether the coordinate range of the i-th image patch falls within the coordinate range of the corresponding minimum bounding rectangle. If yes, increase the confidence of the minimum bounding rectangle to which the i-th image patch belongs; otherwise, decrease the confidence of the minimum bounding rectangle to which the i-th image patch belongs and delete the minimum bounding rectangle with a confidence of less than the preset confidence threshold. If there are overlapping minimum bounding rectangles in the subgraph, the method further includes: retaining the minimum bounding rectangle with the highest confidence and updating the remaining minimum bounding rectangles to the shape of the component outline.

2. The intelligent identification method for power equipment based on deep learning according to claim 1, characterized in that, The YOLOv8s model includes a backbone network for feature extraction, a neck network for feature fusion, and a detection head for predicting overall bounding boxes. Before annotating the overall bounding boxes of electrical equipment contained in the stitched image using a pre-built YOLOv8s model, the method further includes: Generate labels: Obtain images of electrical equipment contained in the ImageNet-X dataset, denote them as target images, use annotation tools to annotate the overall bounding boxes of electrical equipment contained in the target images, and generate YOLO format labels; Pre-training: The pre-built YOLOv8s model is pre-trained using target images labeled with YOLO format to obtain the pre-trained YOLOv8s model; Fine-tuning: Freeze the parameters of the first m layers of the backbone network and the parameters of the downsampling path of the neck network in the pre-trained YOLOv8s model, obtain n stitched images labeled with the overall bounding box as the training set, and use the training set to fine-tune the parameters of the unfrozen part of the pre-trained YOLOv8s model to obtain the YOLOv8s model after transfer training. Model update: Update the transfer-trained YOLOv8s model to a pre-built YOLOv8s model.

3. The intelligent identification method for power equipment based on deep learning according to claim 2, characterized in that, After performing the label generation step and before performing the pre-training step, the method further includes: Each target image is enhanced using SimCLR-style enhancement processing to obtain an enhanced image pair for each target image; A contrastive learning model is constructed, comprising an encoder and a projector head. The encoder is the backbone network of the YOLOv8s model, and the projector head comprises a 2-layer MLP. The contrastive learning model is iteratively trained using augmented image pairs until a preset stopping condition is met. Then, the training stops, the parameters of the backbone network are output, and the backbone network parameters are updated in the pre-constructed YOLOv8s model to perform the pre-training step.

4. The intelligent identification method for power equipment based on deep learning according to claim 3, characterized in that, The method further includes: after updating the backbone network parameters to the pre-built YOLOv8s model, freezing the parameters of the backbone network in the pre-built YOLOv8s model during the pre-training step.

5. The intelligent identification method for power equipment based on deep learning according to any one of claims 1-4, characterized in that, The method further includes designating the historical images contained in the stitched image as the first image, and also includes: The SIFT algorithm is used to extract key points and SIFT descriptors from the first image. Based on the key points and SIFT descriptors, initial matching point pairs are generated through the nearest neighbor matching strategy. The RANSAC algorithm is used to filter the inliers in the initial matching point pairs to obtain the homography matrix between the first images. The first images are then subjected to perspective transformation based on the homography matrix to obtain a new first image. The new first images are then stitched together to obtain a new stitched image.

6. The intelligent identification method for power equipment based on deep learning according to claim 5, characterized in that, The method further includes: The gradient field of the new first image is calculated using the Sobel operator. The Poisson equation is constructed using the gradient field as a constraint, and the pixel values ​​of the stitching boundary region are optimized by the least squares method.

7. A deep learning-based intelligent identification system for power equipment, characterized in that, include: Memory and processor The memory contains a computer-readable storage medium; When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in any one of claims 1-6.

Images