Electric transmission and transformation equipment semantic segmentation method based on cavity space pyramid

Abstract

The invention provides a power transmission and transformation equipment semantic segmentation method based on a cavity space pyramid, and the method comprises the following steps: carrying out the normalization processing of an equipment image, and obtaining a normalized image; inputting the normalized image into a semantic segmentation model; performing feature extraction on the normalized image based on a backbone network module, and extracting feature images corresponding to different semantic features based on the features; performing pooling operation on each feature image based on the cavity space pyramid module, and performing image splicing on each feature image based on an operation result to obtain a spliced image; performing encoding operation on the spliced image based on the encoder module to obtain an encoded image; and carrying out decoding operation on the coded image based on the decoder module to obtain a semantic segmentation image. The method at least improves the segmentation precision and efficiency.

Description

Technical Field

[0001] The invention relates to data processing technology, and in particular to a semantic segmentation method for power transmission and transformation equipment based on a hollow space pyramid. Background Art

[0002] Power transmission and transformation equipment is a vital component of the power system. The monitoring of its operating status and fault diagnosis are not only directly related to the safety and stability of the power grid, but also have important significance for improving the emergency response capability of the power system, reducing accident risks, and ensuring power supply reliability. In the context of the current energy transformation and the rapid development of smart grids, the efficient operation and maintenance of power transmission and transformation equipment is crucial to promoting the access and utilization of green energy and improving the level of smart grids. In recent years, with the development of artificial intelligence technology and computer vision, the state monitoring technology of power transmission and transformation equipment based on image analysis has gradually become a research hotspot, providing technical support for more accurate and efficient equipment management. Among them, semantic segmentation technology, as a method that can classify objects in images at the pixel level, has shown great application potential in tasks such as state recognition, defect detection, and fault location of power transmission and transformation equipment. However, since power transmission and transformation equipment usually has complex structures, diverse morphologies, and detailed edge features, traditional semantic segmentation methods face many challenges in dealing with these characteristics. Therefore, more efficient and sophisticated algorithms are urgently needed to meet the needs of the development of smart grids.

[0003] Semantic segmentation technology based on neural networks has received extensive attention and application in the field of computer vision. Fully Convolutional Network (FCN) is one of the earliest methods that has made a profound impact on semantic segmentation tasks. FCN effectively improves segmentation performance without sacrificing spatial resolution by replacing fully connected layers with convolutional layers. In order to restore the spatial resolution lost after level-by-level pooling, FCN uses bilinear interpolation to upsample low-resolution features and concatenates them with the intermediate feature layer. However, this method of FCN does not fully consider how to efficiently extract multi-scale features, and due to the simple operation of bilinear interpolation, it is usually impossible to finely reconstruct the boundaries of objects, resulting in certain limitations in capturing details. With the deepening of research, Zhou et al. proposed a skip connection strategy to combine the contextual information of feature layers at different stages, further improving the segmentation results. Although this method effectively improves the diversity of feature expression, it may lead to large computational overhead due to the direct concatenation of feature layers of different scales, especially when the network depth increases. When extracting multi-scale features, this method may cause the loss of high-level semantic information due to over-reliance on low-level features, thereby affecting the accuracy of segmentation results. Subsequently, some decoder architectures based on deconvolution were proposed to improve the upsampling effect of FCN. These methods restore spatial resolution through deconvolution layers, but still face the problems of blurred edges and loss of details to a certain extent, especially when dealing with complex shapes and textures. In addition, with the introduction of deconvolution layers, the demand for computing resources has increased dramatically, especially for large networks that require multiple deconvolution operations, the computational overhead is significant and the efficiency is low. To solve these problems, DeepLab proposed atrous convolution, which effectively expands the receptive field of the filter by skipping downsampling, thereby alleviating the problem of detail loss in the pooling process to a certain extent. However, while atrous convolution improves the receptive field, it also introduces a new problem: as the dilation rate increases, atrous convolution may cause a large computational burden when processing high-resolution images, and atrous convolution is also limited in its ability to process fine-grained information.

[0004] In addition to designing deeper and more complex network architectures, some studies have attempted to combine heterogeneous inputs with RGB images to further improve the segmentation effect. For example, the depth image is directly integrated into the CNN as an additional input channel, or other features in the depth image are extracted before inputting into the CNN. The HHA method represents depth information through three channels: horizontal disparity, height above the ground, and local surface normal. Although it can enrich the input features, this method usually ignores the internal structure of the depth image, resulting in limited performance improvement in different scenarios. In addition, integrating depth information usually increases the dimension of the input features, resulting in a significant increase in the number of network parameters and computational complexity, which is a considerable challenge for application scenarios with high real-time requirements. Recently, Wang and Neumann proposed the Depth-Aware Convolution technique, which uses local depth similarity to enhance the depth perception ability of traditional convolution operations. This method effectively improves the segmentation accuracy, especially in complex scenes. However, although this method enhances the expression ability of the convolution kernel through local depth information, it still cannot avoid the shortcomings of multi-scale feature extraction, especially when processing high-resolution images and fine-grained features, the performance improvement is still limited. The introduction of boundary information is also an important direction to improve the accuracy of semantic segmentation in recent years.

[0005] Bertasius et al. and Kokkinos improved the boundary processing effect by learning boundary features and integrating them into the post-processing CRF part of the DeepLab method. However, CRF post-processing operations usually significantly increase the calculation time and are inefficient in processing large-scale images. In summary, how to strike a balance between maintaining feature map resolution and efficient computing resource utilization remains an urgent problem to be solved. Summary of the invention

[0006] Based on the above problems, the present invention is proposed to provide a semantic segmentation method for power transmission and transformation equipment based on a hollow space pyramid, which overcomes the above problems or at least partially solves the above problems.

[0007] According to one aspect of the present invention, a method for semantic segmentation of power transmission and transformation equipment based on a hollow space pyramid is provided, comprising the following steps:

[0008] Acquire a device image of the corresponding power transmission and transformation equipment, and perform normalization processing on the device image to obtain a normalized image;

[0009] Inputting the normalized image into a pre-built semantic segmentation model, wherein the semantic segmentation model includes a backbone network module, a hole space pyramid module, an encoder module, and a decoder module;

[0010] Performing feature extraction on the normalized image based on the backbone network module, and extracting feature images corresponding to different semantic features based on the features;

[0011] Based on the hollow space pyramid module, each feature image is subjected to a pooling operation, and based on the operation result, each feature image is subjected to image stitching to obtain a stitched image;

[0012] Performing an encoding operation on the spliced ​​image based on the encoder module to obtain an encoded image;

[0013] The encoded image is decoded based on the decoder module to obtain a semantic segmentation image.

[0014] Optionally, in the method according to the present invention, normalizing the device image to obtain a normalized image includes:

[0015] Acquire each pixel constituting the device image, and convert the device image from a native format to a tensor format corresponding to a preset normalization value based on each pixel;

[0016] The device image in the corresponding tensor format is pixel-adjusted so that each pixel constituting the device image is adjusted to have a mean value and a unit variance.

[0017] Optionally, in the method according to the present invention, converting the device image from a native format to a tensor format corresponding to a preset normalization value based on each pixel point includes:

[0018] Convert the device image corresponding to the [0,255] native format to the device image corresponding to the [0,1] tensor format based on the following formula:

[0019]

[0020] Among them, I cij is the pixel value of the pixel points that make up the device image in the corresponding native format, I′ cij The pixel values ​​of the pixels that make up the device image in the corresponding tensor format.

[0021] Optionally, in the method according to the present invention, pixel adjustment is performed on the device image in the corresponding tensor format so that each pixel constituting the device image is adjusted to have a zero mean and a unit variance, including:

[0022] The device image of the corresponding tensor format is pixel-adjusted using the following formula:

[0023]

[0024] Among them, μ c=[0.485, 0.456, 0.406], which is the average value of RGB channels, σ c =[0.229, 0.224, 0.225], which is the standard deviation value of RGB channels, I″ cij is the pixel value of each pixel adjusted to zero mean and unit variance.

[0025] Optionally, in the method according to the present invention, feature extraction is performed on the normalized image based on the backbone network module, and feature images corresponding to different semantic features are extracted based on the features, including:

[0026] In response to the normalized image being input into the backbone network module, the backbone network module performs image extraction based on deep features on the normalized image to obtain visual information located in the normalized image;

[0027] The normalized image is converted based on the visual information to obtain feature images corresponding to different semantic features.

[0028] Optionally, in the method according to the present invention, pooling operations are performed on each feature image based on the hole space pyramid module, and image stitching is performed on each feature image based on the operation result to obtain a stitched image, including:

[0029] In response to each feature image being input into the atrous spatial pyramid module, the atrous spatial pyramid module performs channel dimension-based transformation on each feature image to obtain each first transformed image;

[0030] Performing convolution operations based on different sampling rates on each first conversion image to obtain each feature information corresponding to each first conversion image;

[0031] Performing a pooling operation on each first conversion image to obtain context information corresponding to each first conversion image;

[0032] Based on each characteristic information and each context information, each first conversion image is stitched to obtain a stitched image.

[0033] Optionally, in the method according to the present invention, performing convolution operations based on different sampling rates on each first conversion image to obtain each feature information corresponding to each first conversion image respectively includes:

[0034] Convolution operations based on the first sampling rate, the second sampling rate and the third sampling rate are performed on each first conversion image to obtain feature information corresponding to each first conversion image, wherein the first sampling rate is 3x3 Conv Rate3, the second sampling rate is 3x3 Conv Rate 6, and the third sampling rate is 3x3 Conv Rate 12.

[0035] Optionally, in the method according to the present invention, performing an encoding operation on the spliced ​​image based on the encoder module to obtain an encoded image includes:

[0036] In response to the stitched image being input into the encoder module, the encoder module performs an encoding operation on the stitched image to integrate and refine each feature information and each context information in the stitched image to obtain an encoded image.

[0037] Optionally, in the method according to the present invention, performing a decoding operation on the encoded image based on the decoder module to obtain a semantic segmentation image comprises:

[0038] In response to the encoded image and input to the encoder module, the encoder module performs a low-level upsampling operation on the encoded image to obtain low-level features corresponding to the low-level layer;

[0039] splicing the low-level features with the coded image, and integrating and refining the spliced ​​coded image to obtain an integrated optimized image;

[0040] The integrated optimized image is subjected to a low-level upsampling operation to obtain a semantic segmentation image having the same size as the encoded image.

[0041] According to another aspect of the present invention, a system for semantic segmentation of power transmission and transformation equipment based on a hollow space pyramid is provided, comprising:

[0042] A normalization module is configured to obtain a device image of the corresponding power transmission and transformation equipment, and perform normalization processing on the device image to obtain a normalized image;

[0043] An image input module, configured to input the normalized image into a pre-built semantic segmentation model;

[0044] A backbone network module is configured to perform feature extraction on the normalized image, and obtain feature images corresponding to different semantic features based on the feature extraction;

[0045] The atrous spatial pyramid module is configured to perform a pooling operation on each feature image respectively, and to perform image stitching on each feature image based on the operation result to obtain a stitched image;

[0046] an encoder module, configured to perform an encoding operation on the spliced ​​image based on the encoding operation to obtain an encoded image;

[0047] The decoder module is configured to perform a decoding operation on the encoded image to obtain a semantic segmentation image.

[0048] According to the solution of the present invention, in the semantic segmentation task of power transmission and transformation equipment, the present invention solves the technical problems of multi-scale feature extraction, resolution loss and inefficient utilization of computing resources by introducing a void space pyramid module, a mechanism to maintain feature map resolution and an efficient computing resource utilization strategy. The invention not only improves the segmentation accuracy and efficiency, but also provides strong technical support for real-time monitoring and fault diagnosis of power transmission and transformation equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of a method for semantic segmentation of power transmission and transformation equipment based on a hollow space pyramid according to an embodiment of the present invention is shown;

[0050] Figure 2 A schematic diagram showing the structure of the semantic segmentation model in this embodiment is shown;

[0051] Figure 3 A structural block diagram of a power transmission and transformation equipment semantic segmentation system based on a hollow spatial pyramid according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0052] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] Based on the content of the background technology, it can be seen that the existing technology for image recognition of power transmission and transformation equipment generally still has the following technical problems:

[0054] (1) Limitations of single-scale features

[0055] The scales of power transmission and transformation equipment in images vary greatly, from large transmission towers and transformers to small insulators and equipment parts, and they have significant size differences in space. Traditional convolutional neural networks (CNNs) usually use a single-scale feature extraction method, which assumes that all target objects in the image have similar scales. However, in practical applications, the diversity of power transmission and transformation equipment makes it difficult for single-scale convolution to effectively capture the features of equipment of different sizes. When processing such images, single-scale feature extraction may lead to missed detection or loss of details of small equipment, or inaccurate segmentation boundaries for large equipment. Therefore, traditional methods often have large errors and limitations when segmenting these multi-scale equipment.

[0056] (2) Feature map resolution loss

[0057] In semantic segmentation tasks, accurate location information is crucial for identifying the boundaries of equipment, distinguishing adjacent equipment, or identifying the location of equipment damage. Traditional convolutional neural networks reduce the size of feature maps through downsampling operations (such as pooling layers) to accelerate the network training and inference process. However, excessive downsampling will lead to a loss of spatial resolution, which in turn affects the capture of equipment details. In the segmentation task of power transmission and transformation equipment, due to the complex structure and high precision requirements of the equipment, the reduction in feature map resolution will lead to the loss of detail information, and it is impossible to accurately distinguish the boundaries between equipment or locate damaged areas, which is crucial for real-time monitoring and equipment fault diagnosis.

[0058] (3) Inefficient use of computing resources

[0059] The monitoring images of power transmission and transformation equipment usually have high resolution. Processing these images requires a lot of computing resources, especially when the image size is large. Pooling, upsampling and other operations in traditional networks will further increase the demand for computing resources. Although multi-scale feature extraction methods (for example, by introducing multiple convolutional layers and pooling layers) can improve the accuracy of segmentation, these methods usually require multiple processing of features of different scales, resulting in increased computational complexity. Details and texture information in high-resolution images may be ignored due to excessive pooling and upsampling operations, further aggravating the waste of computing resources. Therefore, traditional methods often lack sufficient computational efficiency when performing multi-scale feature extraction and cannot meet the needs of real-time monitoring and processing of large-scale image data.

[0060] (4) Network architecture lacks flexibility

[0061] Different power transmission and transformation scenarios and equipment types, such as outdoor transmission lines and indoor substation equipment, have very different image features. In these different environments, the size, texture, and spatial relationship of the equipment are significantly different. Traditional convolutional neural networks usually lack flexibility and cannot be optimized for these different scenarios and equipment. Therefore, networks based on fixed architectures may not be able to fully adapt to changes in various equipment types or different scenarios, resulting in unsatisfactory segmentation results. In particular, in some special environments, the accuracy and robustness of segmentation often cannot meet the requirements of practical applications.

[0062] Based on the above-mentioned technical problems, an embodiment of the present invention proposes a semantic segmentation method for power transmission and transformation equipment based on a hollow space pyramid, wherein: Figure 1 The flowchart of this embodiment is shown. The method is suitable for being executed in a computing device.

[0063] like Figure 1 As shown, the power transmission and transformation equipment semantic segmentation method based on the hollow space pyramid proposed in this embodiment starts from step S102. In step S102, the following contents are included:

[0064] A device image of the corresponding power transmission and transformation equipment is acquired, and the device image is normalized to obtain a normalized image.

[0065] For example, in this embodiment, the above-mentioned “normalizing the device image to obtain a normalized image” may specifically include the following contents:

[0066] Acquire each pixel constituting the device image, and convert the device image from a native format to a tensor format corresponding to a preset normalization value based on each pixel;

[0067] The device image in the corresponding tensor format is pixel-adjusted so that each pixel constituting the device image is adjusted to have a mean value and a unit variance.

[0068] Furthermore, the device image corresponding to the [0,255] native format is converted to the device image corresponding to the [0,1] tensor format based on the following formula:

[0069]

[0070] Among them, I cij is the pixel value of the pixel points that make up the device image in the corresponding native format, I′ cij The pixel values ​​of the pixels that make up the device image in the corresponding tensor format.

[0071] Similarly, the device image corresponding to the tensor format is pixel-adjusted using the following formula:

[0072]

[0073] Among them, μ c =[0.485, 0.456, 0.406], which is the average value of RGB channels, σ c =[0.229, 0.224, 0.225], which is the standard deviation value of RGB channels, I″ cij is the pixel value of each pixel adjusted to zero mean and unit variance.

[0074] It should be noted that the acquisition and normalization of the device image can be understood as being in the data preparation stage in this embodiment, wherein the specific process can be as follows:

[0075] Initially, we convert the image from its native representation (where pixel values ​​are [0,255]) to a tensor format where the values ​​are normalized to [0,1] and for each pixel I in channel c cij use We then apply channel-wise normalization to this tensor, adjusting each pixel value to zero mean and unit variance using the formula:

[0076]

[0077] Among them, μ = [0.485, 0.456, 0.406] is the average value of the RGB channels, and σ = [0.229, 0.224, 0.225] is the standard deviation value of the RGB channels.

[0078] The corresponding COCO (Common Objects in Context) annotations are stored in JSON format, where the annotated masks support the semantic segmentation task. These masks are grouped by object category and generated by drawing polygons around the specified objects, which are then used as the true value for model training. Indicates the presence (1) or absence (0) of object j in image i.

[0079] We split the original dataset into 80% training set and 20% validation set, resizing all images to 700 × 700 pixels. The training set is used to fine-tune the model. The validation set val is used to extract explanations for the model predictions.

[0080] In step S102, the following contents are included:

[0081] The normalized image is input into a pre-built semantic segmentation model, wherein the semantic segmentation model includes a backbone network module, a hole space pyramid module, an encoder module and a decoder module.

[0082] For example, after completing the acquisition of the normalized image, we can input it into the semantic segmentation model to perform the corresponding semantic segmentation process. It should be noted that the semantic segmentation model is pre-established. When establishing the corresponding semantic segmentation model, it can be implemented based on the following method steps:

[0083] Model preparation: The proposed algorithm architecture is as follows Figure 2 The model combines the advantages of the encoder-decoder architecture and separable convolutions to improve segmentation accuracy, especially along object boundaries.

[0084] The encoder module part uses a deep convolutional neural network (CNN) model as the backbone, such as ResNet or MobileNet, which is modified for the segmentation task by adding deep separable convolutions, reducing the number of parameters and computational cost while maintaining effective feature extraction. ImageNet pre-trained weights are used for encoder initialization. Natural convolution allows the network to grasp contextual information at multiple scales without increasing computational requirements through the atrous spatial pyramid module.

[0085] The decoder module focuses on refining the segmentation output, which is crucial for achieving high resolution and accurate boundary representation. By gradually recovering spatial information and improving feature resolution, the decoder helps produce more accurate segmentation maps. The weights and activations of the convolutional layers are in single-precision floating point format (float32).

[0086] Loss Function: Dice loss function is used to train the model, which is particularly useful for unbalanced classes in image segmentation tasks because it takes into account the overlap between the predicted mask and the true value mask. Let P and GT denote the predicted true value mask and the true value mask, respectively. The loss function can be defined as:

[0087]

[0088] The accuracy of the segmentation model is evaluated using the IoU metric, also known as the Jaccard coefficient, which is one of the most commonly used metrics in semantic segmentation. The IoU metric is a commonly used evaluation metric in semantic segmentation. It quantifies the overlap between the predicted segmentation and the true value for each class. Mathematically, for a certain class, the predicted mask P c and the true value mask GT c , IoU is calculated as follows:

[0089]

[0090] The model was optimized using the Adam optimizer with an initial learning rate of ≤0.0001. After 25 epochs, the latency was halved to 1e-5 to facilitate convergence when the model was close to optimal performance. The batch size was set to 8 to balance computational efficiency and memory constraints during training.

[0091] After completing the establishment of the corresponding model, it is necessary to conduct a result-based model evaluation, which includes the following processes:

[0092] Plausibility, i.e., the consistency of explanations with human intuition, can be evaluated using metrics such as Energy-Based Pointing Game (EBPG), Intersection over Union (IoU), and Bounding Box (Bbox). Based on human annotations, these metrics validate the model by evaluating the statistical strength of the explanations.

[0093] Energy-Based Pointing Game (EBPG): evaluates the accuracy and denoising ability of the method to identify the most influential regions in the image for a given prediction. It computes a saliency map L in pixel units c How much the energy of (i,j) falls within the true value. The EBPG formula is defined as follows:

[0094]

[0095] IoU evaluates the localization ability and importance of the attributes captured in the explanation graph. It measures the overlap between the saliency map and the ground-truth annotations. IOU can be defined as:

[0096]

[0097] Bounding Box (Bbox) is a variant of the IoU metric that adapts to the size of the object of interest. It measures the overlap between the bounding box of the saliency map and the ground-truth bounding box. Bbox is defined as:

[0098]

[0099] Confidence, i.e., the consistency of the explanation with the model’s predicted behavior, is evaluated using removal and insertion metrics. These measures quantify the extent to which the explanation is consistent with the model’s predicted behavior. Given a black-box model Θ, the input image I, the saliency map Lc, and the number of pixels removed at each step.

[0100] "Removal" measures the accuracy of salient regions by removing pixels from the input image in descending order of significance. More accurate salient regions will have steeper removal curves, and smaller removal metric values ​​indicate higher accuracy. Del Defined as:

[0101]

[0102] Among them, h i ←f(I), and I has non-zero pixels, and according to L c , set the next operation pixel in I to 0, until n iterations.

[0103] "Insertion" evaluates the globality of salient regions by removing all pixels from the input image and then restoring them in descending order of saliency. More comprehensive salient regions will require fewer pixels to restore the target and have a faster rising interpolation curve. Higher interpolation metric values ​​indicate more comprehensive salient regions. Insertion d Ins Defined as:

[0104]

[0105] Among them, I ′ ←Blur(I),h i ←f(I),I≠I ′ , and according to L c , will I ′ The next pixel in is set to the corresponding pixel at each iteration until n iterations.

[0106] That is, the model building and the corresponding evaluation process can be completed through the above content, and then the corresponding normalized image can be input into the pre-built semantic segmentation model.

[0107] In step S103, the following contents are included:

[0108] Based on the backbone network module, feature extraction is performed on the normalized image, and feature images corresponding to different semantic features are extracted based on the features.

[0109] For example, in this embodiment, the above-mentioned “extracting features from the normalized image based on the backbone network module, and extracting feature images corresponding to different semantic features based on the features” may also include the following steps:

[0110] In response to the normalized image being input into the backbone network module, the backbone network module performs image extraction based on deep features on the normalized image to obtain visual information located in the normalized image;

[0111] The normalized image is converted based on the visual information to obtain feature images corresponding to different semantic features.

[0112] It should be noted that, in this embodiment, the input image is first passed to the backbone network (Backbone). The backbone network consists of a series of complex convolutional layers, pooling layers and activation functions, and its main function is to perform deep feature extraction on the input image. Through these operations, the backbone network can convert the visual information in the original image into feature maps with rich semantic features. These feature maps will serve as the input of subsequent modules and provide a basis for higher-level feature processing.

[0113] In step S104, the following contents are included:

[0114] Based on the hollow space pyramid module, a pooling operation is performed on each feature image respectively, and based on the operation result, an image splicing is performed on each feature image to obtain a spliced ​​image.

[0115] For example, in this embodiment, the above-mentioned “performing a pooling operation on each feature image based on the hole space pyramid module, and stitching each feature image based on the operation result to obtain a stitched image” may also include the following steps:

[0116] In response to each feature image being input into the atrous spatial pyramid module, the atrous spatial pyramid module performs channel dimension-based transformation on each feature image to obtain each first transformed image;

[0117] Performing convolution operations based on different sampling rates on each first conversion image to obtain each feature information corresponding to each first conversion image;

[0118] Performing a pooling operation on each first conversion image to obtain context information corresponding to each first conversion image;

[0119] Based on each characteristic information and each context information, each first conversion image is stitched to obtain a stitched image.

[0120] Furthermore, in this embodiment, the above-mentioned “performing convolution operations based on different sampling rates on each first conversion image to obtain each feature information corresponding to each first conversion image” may also include:

[0121] Convolution operations based on the first sampling rate, the second sampling rate and the third sampling rate are performed on each first conversion image to obtain feature information corresponding to each first conversion image, wherein the first sampling rate is 3x3 Conv Rate3, the second sampling rate is 3x3 Conv Rate 6, and the third sampling rate is 3x3 Conv Rate 12.

[0122] It should be noted that, in this embodiment, based on the above content, the feature map output from the backbone network enters the hollow space pyramid module. The hollow space pyramid module is one of the core parts of the entire network, which consists of multiple different operations. The first is a 1x1 convolution layer (1x1 Conv), which transforms the features in the channel dimension without changing the spatial resolution of the feature map, which helps to fuse the subsequent features of different scales. Then there are three 3x3 convolution layers, which have different sampling rates, namely 3x3 Conv Rate 3, 3x3 Conv Rate 6 and 3x3 ConvRate12. These convolution operations with different sampling rates can capture multi-scale feature information under different receptive fields, so that the network can effectively process objects of different sizes in the image. In addition, the hollow space pyramid module also includes an image pooling layer (Image Pooling), which obtains global context information by performing a pooling operation on the entire feature map, which is crucial for understanding the semantic relationship in the image. The feature maps obtained by the above different operations will be spliced ​​together and further processed through a 1x1 convolution layer to adjust the number of feature channels so that the features can better adapt to subsequent operations.

[0123] In step S105, the following contents are included:

[0124] An encoding operation is performed on the spliced ​​image based on the encoder module to obtain an encoded image.

[0125] For example, in this embodiment, the above-mentioned “encoding the spliced ​​image based on the encoder module to obtain an encoded image” may also include the following steps:

[0126] In response to the stitched image being input into the encoder module, the encoder module performs an encoding operation on the stitched image to integrate and refine each feature information and each context information in the stitched image to obtain an encoded image.

[0127] It should be noted that, based on the above content, the feature map processed by the dilated spatial pyramid module enters the encoder. The encoder encodes the input feature map, further extracts and compresses the feature information, integrates and refines the semantic information in the feature map, and provides a more representative feature representation for subsequent decoding operations.

[0128] In step S106, the following contents are included:

[0129] The encoded image is decoded based on the decoder module to obtain a semantic segmentation image.

[0130] For example, in this embodiment, the above-mentioned “decoding the encoded image based on the decoder module to obtain a semantic segmentation image” may also include the following steps:

[0131] In response to the encoded image and input to the encoder module, the encoder module performs a low-level upsampling operation on the encoded image to obtain low-level features corresponding to the low-level layer;

[0132] splicing the low-level features with the coded image, and integrating and refining the spliced ​​coded image to obtain an integrated optimized image;

[0133] The integrated optimized image is subjected to a low-level upsampling operation to obtain a semantic segmentation image having the same size as the encoded image.

[0134] It should be noted that based on the above content, the decoder is another key part of the network. First, the decoder performs a 4-fold upsampling operation (Upsamplingx4) on the low-level features (Low Level Features). These low-level features contain detailed information in the image, and the upsampling operation helps to restore these detailed information. Then, the upsampled low-level features are concatenated (Concat) with the output of the encoder. This concatenation operation can fuse the low-level detailed information with the high-level semantic information to form a richer and more comprehensive feature representation. The concatenated feature map is processed through a 3x3 convolution layer (3x3 Conv) to further integrate and optimize the feature information. After that, a 4-fold upsampling operation (Upsampling x4) is performed again to restore the feature map to a size close to the original image in preparation for the final prediction.

[0135] Finally, the feature map processed by the decoder is used for prediction. In tasks such as semantic segmentation, the prediction result is usually the category label of each pixel, which constitutes the semantic segmentation result of the original input image, thus realizing the complete process from the original image input to the semantic segmentation output.

[0136] In summary, in this embodiment, in the semantic segmentation task of power transmission and transformation equipment, this embodiment solves the technical problems of multi-scale feature extraction, resolution loss and inefficient utilization of computing resources by introducing a void space pyramid module, a feature map resolution maintenance mechanism and an efficient computing resource utilization strategy. This invention not only improves the segmentation accuracy and efficiency, but also provides strong technical support for real-time monitoring and fault diagnosis of power transmission and transformation equipment.

[0137] In addition, the role and effect of this embodiment can be further elaborated in detail from the three perspectives of technical benefit, economic benefit and social benefit described below.

[0138] (1) Breakthrough in multi-scale feature extraction

[0139] By introducing the dilated spatial pyramid module, this embodiment can effectively capture the characteristics of power transmission and transformation equipment of different scales, thereby solving the performance bottleneck of traditional convolutional neural networks in multi-scale scenarios. Dilated convolution enables the network to expand the receptive field without sacrificing resolution, ensuring that all kinds of details of the equipment can be accurately segmented regardless of the size of the equipment.

[0140] (2) Accurate device positioning and detail retention

[0141] The feature map resolution preservation mechanism proposed in this embodiment avoids the resolution loss caused by traditional pooling operations through hole convolution. This enables the segmentation network to maintain fine position information in high-resolution images, effectively distinguish adjacent devices and their details, and improve the ability to accurately segment device status, which is especially important in equipment fault diagnosis and damage location.

[0142] (3) Efficient utilization of computing resources

[0143] By reducing unnecessary pooling and upsampling operations, this embodiment greatly reduces the computational complexity and significantly improves the processing speed of high-resolution power transmission and transformation equipment monitoring images. Compared with traditional multi-scale feature extraction methods, the present invention can optimize the use of computing resources while ensuring segmentation accuracy to meet the needs of real-time monitoring.

[0144] (4)Flexible adaptation to the needs of different scenarios and devices

[0145] The network architecture of this embodiment is highly flexible, and the parameters and structure of the dilated convolution can be adjusted according to different power transmission and transformation scenarios and equipment types. This flexibility enables the segmentation model to be optimized for different equipment and monitoring environments, ensuring that the segmentation effect in different scenarios can achieve the best performance.

[0146] Economic benefits:

[0147] (1) Improve equipment inspection and maintenance efficiency

[0148] By improving the segmentation accuracy of power transmission and transformation equipment, this embodiment can help power companies and related enterprises to perform equipment detection and maintenance more efficiently. Accurate segmentation and equipment positioning can quickly discover equipment failures and potential problems, and perform maintenance or replacement in advance, thereby reducing the incidence of sudden failures, reducing downtime losses, and ensuring the stability of the power system. This will greatly save the cost of equipment maintenance and fault handling.

[0149] (2) Reduce manual monitoring costs

[0150] Traditional power transmission and transformation equipment inspection often relies on manual inspection, which is time-consuming and labor-intensive, and has certain human errors. Through automated semantic segmentation technology, this embodiment makes the equipment monitoring process more intelligent and efficient, reduces manual intervention, and reduces labor costs. Especially in the monitoring of large-scale power transmission and transformation networks, automated segmentation technology can replace a large number of manual inspections, significantly reducing monitoring costs.

[0151] (3) Promoting the development of related industries

[0152] The technical innovation of this embodiment not only has direct economic benefits for the power industry, but also provides new application scenarios for the development of technologies in the fields of image processing, artificial intelligence, deep learning, etc. The progress and application of related technologies will promote the development of industries such as computer vision and automated monitoring, further promote the growth of related equipment manufacturing, software development, and technical service industries, and drive the improvement of economic benefits of related industrial chains.

[0153] Social benefits:

[0154] (1) Ensure the stability and security of the power system

[0155] The stable operation of power transmission and transformation equipment is the basis for ensuring the normal power supply of society. By improving the segmentation accuracy and fault diagnosis capability of the equipment, this embodiment helps to timely discover equipment hidden dangers, reduce the failure rate of the power system, and ensure the safety and stability of power supply. This is of great significance for improving the overall power guarantee capability of society and promoting the sustainable development of society.

[0156] (2) Improving emergency response capabilities

[0157] When equipment failure or natural disasters occur, traditional manual monitoring and troubleshooting usually take a long time and are difficult to respond quickly. With the efficient segmentation technology of this embodiment, the power company can obtain the status information of the equipment in real time, quickly locate the faulty equipment and carry out emergency repairs. This will greatly enhance the emergency response capability of the power system, ensure that power supply can be quickly restored in emergencies, and reduce the impact of power outages on social and economic activities.

[0158] (3) Promoting the development of green energy and smart grid

[0159] With the development of green energy and smart grids, the power industry is facing increasingly complex equipment management and monitoring tasks. This embodiment provides important support for the construction and development of smart grids by improving the intelligent monitoring capabilities of power transmission and transformation equipment. Smart grids can achieve automation, remote monitoring and precise control, thereby reducing energy waste and improving energy efficiency.

[0160] (4) Improving the overall level of social intelligence

[0161] This embodiment, as an innovative application of deep learning technology in the power industry, has promoted the popularization and application of intelligent monitoring technology. After being widely used in the field of power monitoring, related technologies are expected to be expanded to other industries, such as transportation, urban management, etc., further improving the overall level of social intelligence and promoting social scientific and technological progress and modernization.

[0162] Another embodiment of the present invention provides a semantic segmentation system for power transmission and transformation equipment based on a hollow space pyramid. Figure 3 As a corresponding device block diagram, the system includes:

[0163] A normalization module is configured to obtain a device image of the corresponding power transmission and transformation equipment, and perform normalization processing on the device image to obtain a normalized image;

[0164] An image input module, configured to input the normalized image into a pre-built semantic segmentation model;

[0165] A backbone network module is configured to perform feature extraction on the normalized image, and obtain feature images corresponding to different semantic features based on the feature extraction;

[0166] The atrous spatial pyramid module is configured to perform a pooling operation on each feature image respectively, and to perform image stitching on each feature image based on the operation result to obtain a stitched image;

[0167] an encoder module, configured to perform an encoding operation on the spliced ​​image based on the encoding operation to obtain an encoded image;

[0168] The decoder module is configured to perform a decoding operation on the encoded image to obtain a semantic segmentation image.

[0169] In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the examples of the present invention. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to implement the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the preferred embodiment of the present invention.

[0170] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0171] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0172] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.

[0173] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of submodules or subunits or subcomponents.

[0174] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present invention and to form different embodiments.

[0175] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0176] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.

[0177] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, rather than for explaining or limiting the subject matter of the present invention.

Claims

1. A semantic segmentation method for power transmission and transformation equipment based on a hollow space pyramid, characterized in that: The following steps are involved: Acquire a device image of the corresponding power transmission and transformation equipment, and perform normalization processing on the device image to obtain a normalized image; Inputting the normalized image into a pre-built semantic segmentation model, wherein the semantic segmentation model includes a backbone network module, a hole space pyramid module, an encoder module, and a decoder module; Performing feature extraction on the normalized image based on the backbone network module, and extracting feature images corresponding to different semantic features based on the features; Based on the hollow space pyramid module, each feature image is subjected to a pooling operation, and based on the operation result, each feature image is subjected to image stitching to obtain a stitched image; Performing an encoding operation on the spliced ​​image based on the encoder module to obtain an encoded image; The encoded image is decoded based on the decoder module to obtain a semantic segmentation image.

2. According to claim 1, a method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid is characterized in that: Normalizing the device image to obtain a normalized image includes: Acquire each pixel constituting the device image, and convert the device image from a native format to a tensor format corresponding to a preset normalization value based on each pixel; The device image in the corresponding tensor format is pixel-adjusted so that each pixel constituting the device image is adjusted to have a mean value and a unit variance.

3. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 2, characterized in that: The device image is converted from a native format to a tensor format corresponding to a preset normalization value based on each pixel point, including: Convert the device image corresponding to the [0,255] native format to the device image corresponding to the [0,1] tensor format based on the following formula: Among them, I cij is the pixel value of the pixel points that make up the device image in the corresponding native format, I′ cij The pixel values ​​of the pixels that make up the device image in the corresponding tensor format.

4. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 3 is characterized in that: Perform pixel adjustment on the device image in the corresponding tensor format so that each pixel constituting the device image is adjusted to have a zero mean and a unit variance, including: The device image of the corresponding tensor format is pixel-adjusted using the following formula: Among them, μ c =[0.485, 0.456, 0.406], which is the average value of RGB channels, σ c =[0.229, 0.224, 0.225], which is the standard deviation value of RGB channels, I″ cij is the pixel value of each pixel adjusted to zero mean and unit variance.

5. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 1, characterized in that: Based on the backbone network module, the normalized image is subjected to feature extraction, and feature images corresponding to different semantic features are extracted based on the features, including: In response to the normalized image being input into the backbone network module, the backbone network module performs image extraction based on deep features on the normalized image to obtain visual information located in the normalized image; The normalized image is converted based on the visual information to obtain feature images corresponding to different semantic features.

6. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 5, characterized in that: Based on the hollow space pyramid module, each feature image is subjected to a pooling operation respectively, and each feature image is subjected to image stitching based on the operation result to obtain a stitched image, including: In response to each feature image being input into the atrous spatial pyramid module, the atrous spatial pyramid module performs channel dimension-based transformation on each feature image to obtain each first transformed image; Performing convolution operations based on different sampling rates on each first conversion image to obtain each feature information corresponding to each first conversion image; Performing a pooling operation on each first conversion image to obtain context information corresponding to each first conversion image; Based on each characteristic information and each context information, each first conversion image is stitched to obtain a stitched image.

7. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 6, characterized in that: Convolution operations based on different sampling rates are performed on each first conversion image to obtain feature information corresponding to each first conversion image, including: Convolution operations based on the first sampling rate, the second sampling rate and the third sampling rate are performed on each first conversion image to obtain feature information corresponding to each first conversion image, wherein the first sampling rate is 3x3 Conv Rate 3, the second sampling rate is 3x3 Conv Rate 6, and the third sampling rate is 3x3 Conv Rate 12.

8. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 6, characterized in that: Performing an encoding operation on the spliced ​​image based on the encoder module to obtain an encoded image includes: In response to the stitched image being input into the encoder module, the encoder module performs an encoding operation on the stitched image to integrate and refine each feature information and each context information in the stitched image to obtain an encoded image.

9. The method for semantic segmentation of power transmission and transformation equipment based on hollow space pyramid according to claim 8, characterized in that: The method further comprises: performing a decoding operation on the encoded image based on the decoder module to obtain a semantic segmentation image, including: In response to the encoded image and input to the encoder module, the encoder module performs a low-level upsampling operation on the encoded image to obtain low-level features corresponding to the low-level layer; splicing the low-level features with the coded image, and integrating and refining the spliced ​​coded image to obtain an integrated optimized image; The integrated optimized image is subjected to a low-level upsampling operation to obtain a semantic segmentation image having the same size as the encoded image.

10. A semantic segmentation system for power transmission and transformation equipment based on hollow space pyramid, characterized in that: include: A normalization module is configured to obtain a device image of the corresponding power transmission and transformation equipment, and perform normalization processing on the device image to obtain a normalized image; An image input module, configured to input the normalized image into a pre-built semantic segmentation model; A backbone network module is configured to perform feature extraction on the normalized image, and obtain feature images corresponding to different semantic features based on the feature extraction; The atrous spatial pyramid module is configured to perform a pooling operation on each feature image respectively, and to perform image stitching on each feature image based on the operation result to obtain a stitched image; an encoder module, configured to perform an encoding operation on the spliced ​​image based on the encoding operation to obtain an encoded image; The decoder module is configured to perform a decoding operation on the encoded image to obtain a semantic segmentation image.

Images