A method for training a crack semantic segmentation model, a segmentation method, and related devices

Abstract

The present application discloses a method for training a crack semantic segmentation model, a segmentation method and related devices. The training method includes: inputting a crack training image into a constructed segmentation network, and extracting a crack feature map of the crack training image through an encoder; normalizing the crack feature map through a frequency feature aggregation module, uniformly dividing the normalized crack feature map into window feature maps of a preset size, and sequentially performing fast Fourier transform, convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each window feature map, then fusing and performing attention extraction with the normalized crack feature map, and weighting the crack feature map to obtain an enhanced feature map; performing semantic segmentation on the enhanced feature map through a decoder to obtain a predicted segmentation result; calculating a loss value according to the predicted segmentation result and the standard semantic segmentation result to update network parameters, and obtaining a crack semantic segmentation model. The present application improves the segmentation accuracy of the cracks on the surface of the switch cabinet.

Description

Technical Field

[0001] This application relates to the field of image segmentation technology, and in particular, to a method for training a crack semantic segmentation model, a segmentation method, and related devices. Background Art

[0002] High-voltage switchgear is an important device in the power system, and its main function is to realize the distribution, control, and protection of electric energy. However, due to long-term exposure to complex environments, such as factors like humidity, high temperature, ultraviolet radiation, and acid-base corrosion, cracks are likely to appear on the surface of the switchgear housing. These cracks not only affect the insulation performance of the equipment but may also cause internal electrical components to be damp, corroded, or even short-circuited, seriously threatening the safe operation of the power system. Therefore, the detection, evaluation, and protection of cracks on the surface of high-voltage switchgear housings are of great significance.

[0003] In the early stage, the detection of cracks on the surface of high-voltage switchgear housings was manual observation, which was time-consuming, laborious, and inefficient, and was no longer sufficient to meet the increasing maintenance requirements. In recent years, image processing technology based on computer vision has gradually replaced manual work and become the mainstream method for detecting cracks on the surface of high-voltage switchgear housings. Traditional image processing algorithms based on machine vision, such as median filtering, threshold algorithms, edge detection, etc. These algorithms are relatively sensitive to interference factors such as noise, illumination changes, and complex backgrounds in images, and it is difficult to detect small cracks from complex images. Improving the detection efficiency and accuracy and reducing the heavy workload of detection personnel, implementing automatic detection of cracks on the surface of high-voltage switchgear has become an inevitable trend.

[0004] Constrained by the defects of traditional image processing algorithms, crack semantic segmentation algorithms for detecting cracks on the surface of high-voltage switchgear housings have emerged in an endless stream, but the detection accuracy of existing segmentation networks is not high. Summary of the Invention

[0005] This application provides a method for training a crack semantic segmentation model, a segmentation method, and related devices, which are used to improve the segmentation accuracy of cracks on the surface of high-voltage switchgear housings.

[0006] In view of this, the first aspect of this application provides a method for training a crack semantic segmentation model, including:

[0007] Construct a segmentation network, where the segmentation network includes an encoder, a frequency feature aggregation module, and a decoder;

[0008] Input the crack training images on the surface of the high-voltage switchgear housing into the segmentation network, and perform feature extraction on the crack training images through the encoder to obtain crack feature maps;

[0009] The crack feature map is normalized by the frequency feature aggregation module, and the normalized crack feature map is evenly divided into window feature maps of a preset size. Fast Fourier transform is performed on each window feature map to obtain window frequency feature maps. Convolution, channel attention feature extraction, inverse fast Fourier transform, and window merging are sequentially performed on each of the window frequency feature maps. The merged feature map is fused with the normalized crack feature map to obtain a fused feature map. Attention extraction is performed on the fused feature map to obtain a feature attention map, and the crack feature map is weighted by the feature attention map to obtain an enhanced feature map;

[0010] The enhanced feature map is subjected to semantic segmentation by the decoder to obtain a predicted segmentation result of the crack training image;

[0011] A loss value is calculated according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and the network parameters of the segmentation network are updated by the loss value to obtain a trained crack semantic segmentation model.

[0012] Optionally, the encoder and the decoder are residual networks stacked by improved residual convolution blocks, and the improved residual convolution block includes a channel convolution block, a spatial convolution block, and a feature splicing module;

[0013] The channel convolution block is used to split the input feature map into channels, perform attention evaluation on each channel feature map to obtain attention weights for each channel, multiply the attention weights for each channel by the convolution kernel, and then perform convolution processing on the input feature map to obtain crack channel features;

[0014] The spatial convolution block is used to group the input feature map, perform spatial displacement operations on each group of feature maps, so that each group of feature maps is feature-aligned along the channel direction to obtain crack spatial features;

[0015] The feature splicing module is used to splice the crack channel features and the crack spatial features.

[0016] Optionally, the segmentation network further includes a multi-layer perceptron;

[0017] The multi-layer perceptron is used to perform feature extraction on the fused features after feature splicing of the feature maps extracted from each layer in the encoder, and weight the enhanced feature map by the extracted feature map to obtain a weighted enhanced feature map;

[0018] The step of subjecting the enhanced feature map to semantic segmentation by the decoder to obtain a predicted segmentation result of the crack training image includes:

[0019] The weighted enhanced feature map is semantically segmented by the decoder to obtain the predicted segmentation result of the crack training image.

[0020] Optionally, the segmentation network further includes an attention feature refinement module, and the attention includes a channel branch module, a spatial branch module, and a residual connection module;

[0021] The spatial branch module is used to perform depth convolution on the input feature map to generate a spatial feature map, perform a pixel shuffle operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the attention feature refinement module is the feature map extracted by the multi-layer perceptron multiplied by the weighted enhanced feature map; the output of the attention feature refinement module is connected to the decoder;

[0022] The channel branch module is used to perform global average pooling on the input feature map, and then perform channel compression and expansion on the pooled feature map to obtain a channel attention map;

[0023] The residual connection module is used to perform residual connection on the input feature map, the spatial attention map, and the channel attention map, and the output of the residual connection module is used as the input of the last convolutional layer of the encoder.

[0024] Optionally, the method further includes:

[0025] Performing data augmentation on the crack training image.

[0026] A second aspect of the present application provides a method for semantic segmentation of cracks on the surface of a high-voltage switchgear shell, including:

[0027] Collecting an image of the surface of the high-voltage switchgear shell;

[0028] Inputting the image of the surface of the high-voltage switchgear shell into a crack semantic segmentation model for semantic segmentation; wherein, the crack semantic segmentation model is trained by using any one of the crack semantic segmentation model training methods in the first aspect.

[0029] A third aspect of the present application provides a device for training a crack semantic segmentation model, including:

[0030] A construction unit, configured to construct a segmentation network, where the segmentation network includes an encoder, a frequency feature aggregation module, and a decoder;

[0031] A training unit for inputting a crack training image on the surface of the high-voltage switchgear cabinet into a segmentation network; extracting features from the crack training image through the encoder to obtain a crack feature map; normalizing the crack feature map through the frequency feature aggregation module, evenly dividing the normalized crack feature map into window feature maps of a preset size, performing a fast Fourier transform on each window feature map to obtain a window frequency feature map, successively performing convolution, channel attention feature extraction, inverse fast Fourier transform, and window merging on each of the window frequency feature maps, fusing the merged feature map with the normalized crack feature map to obtain a fused feature map, extracting attention from the fused feature map to obtain a feature attention map, and weighting the crack feature map through the feature attention map to obtain an enhanced feature map; performing semantic segmentation on the enhanced feature map through the decoder to obtain a predicted segmentation result of the crack training image;

[0032] A parameter update unit for calculating a loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and updating the network parameters of the segmentation network through the loss value to obtain a trained crack semantic segmentation model.

[0033] The fourth aspect of this application provides a crack semantic segmentation device for the surface of a high-voltage switchgear cabinet, including:

[0034] An image acquisition unit for acquiring an image of the surface of the high-voltage switchgear cabinet;

[0035] A segmentation unit for inputting the image of the surface of the high-voltage switchgear cabinet into the crack semantic segmentation model for semantic segmentation;

[0036] Wherein, the crack semantic segmentation model is trained by using any one of the crack semantic segmentation model training methods in the first aspect.

[0037] The fifth aspect of this application provides an electronic device, characterized in that the device includes a processor and a memory;

[0038] The memory is used for storing program codes and transmitting the program codes to the processor;

[0039] The processor is used for executing any one of the crack semantic segmentation model training methods in the first aspect according to the instructions in the program codes, or executing the crack semantic segmentation method for the surface of the high-voltage switchgear cabinet in the second aspect.

[0040] The sixth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, it implements any one of the crack semantic segmentation model training methods in the first aspect, or implements the high-voltage switchgear housing surface crack semantic segmentation method in the second aspect.

[0041] As can be seen from the above technical solutions, the present application has the following advantages:

[0042] A crack semantic segmentation model training method provided by the present application adds a frequency feature aggregation module between the encoder and the decoder to extract crack frequency features, and uses a windowing method to solve the problem of input mismatch in the frequency domain. By modeling the depth feature frequency, it improves the defect that it is difficult to distinguish similar pixel points in the time domain, enhances the crack edge features, and thus improves the crack segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a schematic flowchart of a crack semantic segmentation model training method provided by an embodiment of the present application;

[0045] Figure 2 It is a schematic structural diagram of a channel space convolution module provided by an embodiment of the present application;

[0046] Figure 3 It is a schematic structural diagram of a standard residual convolution block and an improved residual convolution block provided by an embodiment of the present application;

[0047] Figure 4 It is a schematic structural diagram of a frequency feature aggregation module provided by an embodiment of the present application;

[0048] Figure 5 It is a schematic structural diagram of a crack semantic segmentation model provided by an embodiment of the present application;

[0049] Figure 6 It is a comparison of the training results of different models on each dataset provided by an embodiment of the present application;

[0050] Figure 7 It is a visualization result of the segmentation maps output by each model provided by an embodiment of the present application;

[0051] Figure 8The comparison results of the backbone network performance provided by the embodiments of this application;

[0052] Figure 9 The visualization results of the heat maps of each model provided by the embodiments of this application;

[0053] Figure 10 The results of the model ablation experiment provided by the embodiments of this application;

[0054] Figure 11 A schematic flow diagram of a method for semantic segmentation of cracks on the surface of the high-voltage switchgear cabinet shell provided by the embodiments of this application;

[0055] Figure 12 A schematic structural diagram of a device for training a crack semantic segmentation model provided by the embodiments of this application;

[0056] Figure 13 A schematic structural diagram of a device for semantic segmentation of cracks on the surface of a pressure switchgear cabinet shell provided by the embodiments of this application. Detailed implementation manners

[0057] In order to enable those skilled in the art to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0058] For ease of understanding, please refer to Figure 1 , the embodiments of this application provide a method for training a crack semantic segmentation model, including:

[0059] Step 110, construct a segmentation network.

[0060] The segmentation network in this application includes an encoder, a frequency feature aggregation module, and a decoder. Among them, a residual network (such as ResNet-50) can be used as the encoder and decoder.

[0061] Step 120, input the crack training images on the surface of the high-voltage switchgear cabinet shell into the segmentation network, and use the encoder to extract features from the crack training images to obtain crack feature maps.

[0062] The crack training images on the surface of the high-voltage switchgear cabinet shell can be obtained from existing public datasets, or the crack images on the surface of the high-voltage switchgear cabinet shell can be used as training images, and at the same time, they are labeled to obtain standard crack segmentation images.

[0063] To strengthen the training process of the network, prevent the network from falling into local optimal solutions, and ensure the segmentation accuracy of the model, data augmentation processing can be performed on the dataset. Methods such as randomly cropping, randomly rotating, and randomly scaling the crack training images can be used to enhance the data. Image denoising processing can also be performed on the training images to reduce the deviation of the brightness distribution of the dataset and smooth the noise of the images, thereby reducing the impact of problems such as jitter, noise, and ghosting that occur during the acquisition of the high-voltage switchgear cabinet shell surface crack dataset on the model training.

[0064] In one embodiment, an existing residual network can be directly used as an encoder for feature extraction.

[0065] In another embodiment, to improve the crack segmentation accuracy of the model, the existing residual network is improved. Specifically, the residual convolution blocks in the residual network are improved to better encode the high-voltage switchgear cabinet shell surface crack features. The present application proposes a channel space convolution module (CSConv), as Figure 2 shown, the channel space convolution module includes a channel convolution block, a spatial convolution block, and a feature splicing module.

[0066] The channel convolution block is used to perform channel splitting (Split) on the input feature map, perform attention (Attention) evaluation on each channel feature map to obtain the attention weights of each channel (Weight Generation), multiply the attention weights of each channel by the convolution kernel (Kernal Size), and then perform convolution processing on the input feature map to obtain crack channel features; the channel convolution block first splits the input feature map by channel, then uses attention to evaluate the importance of each channel to generate weights, multiplies the weights by the convolution kernel, and makes the convolution kernel have a dynamic size; at this time, convolution is performed on the input feature map again, which can discard unimportant features and enable the encoder to better extract the high-voltage switchgear cabinet shell surface crack features.

[0067] The spatial convolution block is used to perform grouping (Clusters) on the input feature map, perform spatial displacement operations (Padding) on each group of feature maps, so that each group of feature maps is aligned in the channel direction to obtain crack spatial features (TargetFeature). A parameter-free spatial displacement operation is introduced in the spatial convolution block to align adjacent features of the input feature map in the channel direction, thereby realizing spatial feature aggregation. Specifically, first group the input feature map, perform spatial displacement on the feature channels of different groups to improve the aggregation of adjacent features at the corresponding positions of the convolution; finally, the feature splicing module splices the crack channel features and the crack spatial features, fusing the crack channel features and the crack spatial features, where the crack spatial features are convolved and then spliced with the crack channel features.

[0068] In this application, a channel spatial convolution module is used to construct a residual convolution block, which can better encode the crack features on the surface of the high-voltage switchgear cabinet. Please refer to Figure 3 , Figure 3 . (a) in Figure 3 is a standard residual convolution block, and (b) in

[0069] is an improved residual convolution block (DSBlock). The improved residual convolution block replaces the original two 3x3 standard convolutions with CSConv, enhancing the feature extraction ability. An encoder is constructed based on the improved residual convolution block to extract the depth features of the cracks on the surface of the high-voltage switchgear cabinet, and on the premise of ensuring performance, the parameters of the convolution are reduced.

[0070] In the time domain, the influence of complex backgrounds will obscure the boundary between cracks and backgrounds, making crack detection more difficult. Moreover, time-domain signals mainly focus on the changes of signals over time, but have limited understanding of the structural information of signals in space, making the detection of fine structures such as cracks by the model inaccurate.

[0071] To address the above problems, this application proposes a Frequency Feature Aggregation Module, as shown in Figure 4 . The Frequency Feature Aggregation Module first uses a LayerNorm to standardize the input crack feature map to average the input distribution. To overcome the feature loss caused by frequency input mismatch, the Frequency Feature Aggregation Module evenly partitions the standardized crack feature map into windows of size N (WindowsPartition), and the size of each window is SN = [B, C, N, N]. Then, these windows are transformed to the frequency domain by a two-dimensional fast Fourier transform (2D FFT):

[0072]

[0073] where F(*) represents the fast Fourier transform; b and c represent the indices of batches and channels; u and v represent the u-th horizontal and v-th vertical frequencies in the frequency spectrum.

[0074] To further extract the depth features of cracks on the surface of the high-voltage switchgear cabinet, this application simultaneously considers the real and imaginary parts of complex frequencies and uses convolution operations to aggregate frequency features. The processing of any complex frequency is as follows:

[0075]

[0076] Among them, 、 represent the real and imaginary parts of the frequency; 、 are the real and imaginary parts of the frequency after convolution processing; Kr and Ki represent the real and imaginary parts of the complex convolution kernel.

[0077] Then, the channel attention mechanism (CA) is used to model the surface crack features of the high-voltage switchgear cabinet in the spectrum to highlight the useful features in the frequency channels; then, the inverse two-dimensional fast Fourier transform (2DIFFT) is performed on the window frequency feature map and then the windows are merged (Patch Mergin), and the size of the merged feature map is restored to [B, C, H, W]. Finally, the normalization layer and the sigmoid activation function are used in sequence for processing to generate a feature attention map, and the generated attention map is multiplied by the original input feature map to enhance the crack features in the feature map and obtain an enhanced feature map.

[0078] This application designs a frequency feature aggregation module for the characteristics of cracks, uses the windowing method to solve the problem of input mismatch in the frequency domain, and simultaneously considers the complex frequencies of features; models the depth feature frequencies through channel attention, improves the defect that it is difficult to distinguish similar pixel points in the time domain, and enhances the crack edge features.

[0079] Step 140: Perform semantic segmentation on the enhanced feature map through a decoder to obtain the predicted segmentation result of the crack training image.

[0080] In one embodiment, after the enhanced feature map is extracted by the frequency feature aggregation module, it is directly input into the decoder for semantic segmentation to obtain the predicted segmentation result of the crack training image.

[0081] In another embodiment, a multi-layer perceptron (MLP) can also be introduced into the segmentation network. The multi-layer perceptron includes a fully connected layer FC, a Relu activation function, and a Softmax layer; the feature maps extracted by each layer of the encoder are feature concatenated and then input into the multi-layer perceptron. The multi-layer perceptron extracts features from the input features, generates an attention map to enhance the weights of the useful features at the bottom layer, weights the enhanced feature map with the feature map extracted by the multi-layer perceptron to obtain a weighted enhanced feature map; then the weighted enhanced feature map is input into the decoder for semantic segmentation.

[0082] In yet another embodiment, to reduce feature redundancy in multi-scale feature fusion and prevent useful features from being overwhelmed, an Attention Feature Refinement Module is added to the segmentation network, as Figure 5 shown. The Attention Feature Refinement Module is used to refine crack features and improve segmentation accuracy.

[0083] Attention includes a channel branch module, a spatial branch module, and a residual connection module; the spatial branch module is used to perform depth convolution on the input feature map to generate a spatial feature map, perform a pixel shuffle operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the Attention Feature Refinement Module is the feature map obtained by multiplying the feature map extracted by the multi-layer perceptron and the weighted enhanced feature map; the output of the Attention Feature Refinement Module is connected to the decoder; the channel branch module is used to perform global average pooling on the input feature map, then perform channel compression and expansion on the pooled feature map to obtain a channel attention map; the residual connection module is used to perform residual connection on the input feature map, the spatial attention map, and the channel attention map, and the output of the residual connection module is used as the input to the last convolutional layer of the encoder.

[0084] High-voltage switchgear cracks usually exhibit different shapes, including linear, reticular, and cracked. Linear cracks are usually slender cracks extending along the expansion direction, while reticular cracks present an intersecting crack network, and cracked cracks present fine cracks similar to cracked earth. As the network deepens, the backbone network can extract rich crack semantic features, which are crucial for distinguishing cracks from the background. However, with the extraction of local crack features, the spatial dimension of the feature map continuously shrinks, resulting in the loss of spatial correlation information. Therefore, this application introduces an Attention Feature Refinement Module, aiming to alleviate the semantic differences between the feature maps extracted by the backbone network, enhance their spatial correlation, and reduce redundant features. The Attention Feature Refinement Module consists of a spatial branch and a channel branch. The spatial branch first uses depth convolution to generate a spatial feature map of the crack, then extracts the correlation between pixels through a pixel shuffle operation, and generates an attention map through the Sigmoid function. The channel branch first performs global average pooling on the input feature map to convert it into a tensor of size 1×1×C (where C is the number of channels). Subsequently, through a reduction and expansion layer, the number of channels is reduced to one-third of the original number of channels, and then it is restored to enhance the semantic details of the channels, thereby improving the segmentation ability of the cracks. Finally, the input feature map is residually connected to the attention maps generated by the two branches to prevent network degradation, and the feature map after residual connection is input into the last convolutional layer in the encoder. Through the convolutional layer and upsampling, the final semantic segmentation map of the surface cracks of the high-voltage switchgear housing is generated to obtain the predicted segmentation result.

[0085] In this application, due to the long-term exposure of high-voltage switchgear to electric fields, mechanical stresses, and environmental factors (such as temperature changes, humidity, contamination, ultraviolet radiation, etc.), its surface is prone to aging and cracking, seriously affecting power supply safety. Especially in the case of substandard quality or long usage time, it is crucial to promptly identify and fill the cracks on the surface of the high-voltage switchgear housing. Aiming at the problem that the segmentation performance of the existing semantic segmentation model for the cracks on the surface of the high-voltage switchgear housing deteriorates, and the standard convolution is difficult to capture the spatial and channel coupling relationships between pixels, making it difficult to distinguish crack pixels from background pixels in complex backgrounds, this application proposes a semantic segmentation model for cracks on the surface of the high-voltage switchgear housing that combines channel-spatial convolution and frequency feature aggregation. By simultaneously modeling the pixel spatial relationship and channel feature connection through grouped spatial displacement and dynamic convolution kernel weights, crack pixels can be accurately identified. To enhance the contrast of the crack edges, a frequency domain feature aggregation module is proposed, which uses a simple windowing strategy to solve the problem of frequency domain input mismatch. At the same time, considering the influence of the imaginary part of the frequency on the features, it effectively models the deep frequency features and aggregates the crack main body and edge information through the frequency domain feature aggregation module. Finally, an attention feature refinement module is designed to refine the semantic features and further improve the segmentation accuracy.

[0086] Step 150: Calculate the loss value based on the predicted segmentation result and the corresponding standard semantic segmentation result of the crack training image, and update the network parameters of the segmentation network through the loss value to obtain a trained crack semantic segmentation model.

[0087] To calculate the loss value based on the predicted segmentation result and the corresponding standard semantic segmentation result of the crack training image, the loss function of the existing segmentation network can be used, and no specific limitation is made here. Update the network parameters of the segmentation network through the loss value until the segmentation network converges (such as reaching the maximum number of training times or the training error is lower than the preset error threshold, etc.) to obtain a trained crack semantic segmentation model.

[0088] To verify the effectiveness of the segmentation model proposed in this application, a crack dataset is introduced. This dataset is collected by an industrial area array camera of a German company with a resolution of 512×512 pixels. To improve the robustness of the segmentation model, different lighting conditions, shadows, sundries and other scenarios are selected for taking crack pictures, and a total of 2026 crack pictures are collected. To improve the generalization ability of the model, the method of data augmentation is used to expand the dataset. The quality of the dataset is enhanced and expanded by using the data augmentation method, and finally a total of 6400 crack images are obtained. The crack dataset is divided into a training set and a test set in a ratio of 8:2, and fine annotation is carried out using the Labelme software.

[0089] The programming environment for the experiment is Python 3.6, and the torch version is 1.10.8. The hardware environment is an Intel I7-7800X CPU with an NVIDIA GeForce RTX 3090 graphics card with 24GB of video memory and 16GB of RAM. To prevent the model from falling into a local optimum during training, the Adam optimizer is used to optimize the training.

[0090] To evaluate segmentation models of different depths, this application uses classic evaluation metrics in the field of semantic segmentation, including the mean Intersection over Union (mIoU), the mean Pixel Accuracy (mPA), and Precision. As shown in the following formula:

[0091]

[0092]

[0093]

[0094] In the formula, TP represents the number of true positives, FP represents the number of false positives, and FN represents the number of false negatives. A represents the sum of the pixel accuracies of each image averaged to obtain the mean pixel accuracy over the entire dataset. Pi represents the accuracy after comparing the crack pixel points identified in picture i with the pixels in the label, and n is the number of samples in the test set. This matter also introduces FLOPs (floating-point operation counts) and Params to measure the computational complexity of the network.

[0095] This application verifies the proposed semantic segmentation model for cracks on the surface of high-voltage switchgear enclosures. The proposed segmentation model is compared with current popular semantic segmentation models, Unet, ResUnet, DeepLabV3, and AttentionUnet. The experimental results are as Figure 6 shown Figure 7 which shows the performance of each model on the self-built dataset (Self-built), Crack Tree, and CrackForest Dataset.

[0096] From Figure 6It can be seen that the segmentation model proposed in this application outperforms other models on each dataset, achieving a balance between performance and parameters. In terms of the number of parameters and FLOPs, the segmentation model proposed in this application has only 12.836M parameters and 6.846G FLOPs, indicating that this segmentation model can represent more crack features with relatively fewer parameters. Although DeepLabV3+ has fewer parameters, its segmentation performance is inferior to the segmentation model proposed in this application. In terms of segmentation accuracy, the models proposed in this invention have accuracies of 85.99%, 77.15%, and 83.07% on the three datasets respectively. Compared with Unet and ResUnet, the segmentation model proposed in this application has made great improvements, leading by 11.57% and 10.4% respectively on the self-built dataset. Compared with DeepLabV3+, the dilated convolution greatly reduces the number of parameters but also reduces the segmentation performance. AttentionUnet adds multiple attention mechanisms on the basis of Unet, making the accuracy reach the highest, but it also increases the computational complexity of the model and is difficult to train. In contrast, the segmentation model proposed in this application greatly reduces the training difficulty without sacrificing too much accuracy.

[0097] Figure 7 The visualization results of the segmentation of each model are shown. The first column is the original image, the second column is the image annotation, and the third to fifth columns are ours, DeepLabV3+, Unet, ResUnet, and AttentionUnet respectively. Six relatively typical images are selected from each dataset to compare the image segmentation results. The segmentation model proposed in this application has stronger generalization performance and is more accurate in segmenting the surface cracks of high-voltage switchgear cabinets under complex backgrounds. DeepLabV3+ is severely under-segmented when facing complex cracks, especially at the joints of cracks, because DeepLabV3+ does not consider the correlation between spatial information and the perception field and cannot make full use of the correlation between shallow semantics and deep semantics. Unet and ResUnet only obtain local context information through standard convolution and lack rich spatial details. Therefore, they are severely under-segmented and over-segmented when the contrast between cracks and the background is low. AttentionUnet uses multiple attention mechanisms to model spatial and channel context and can achieve good segmentation results, but it still shows limited segmentation ability for the details of cracks.

[0098] Through comprehensive comparison, the model proposed in this application leads in performance among current popular models. This is because the model proposed in this application is constructed by lightweight CSConv, which greatly reduces the parameters and computational complexity of the model while maintaining good feature extraction ability; and the added frequency feature aggregation module and attention feature refinement module further improve the segmentation accuracy of the model.

[0099] Furthermore, this application also evaluated the feature extraction capabilities of the proposed segmentation model compared to current popular backbone networks. These include the backbone networks based on the CNN architecture, namely ShuffleNetV2, MobileNetV3, GhostNet, EfficientNetV2, and the backbone network that combines CNN and ViT, MobileViT. Each model was trained on the self-built crack dataset, and the experimental results are as Figure 8 shown.

[0100] As Figure 8 can be seen, the segmentation model proposed in this application is more superior in terms of crack feature extraction capabilities compared to other popular backbone networks. This is due to the fact that CSConv models the local features of the cracks on the surface of the high-voltage switchgear enclosure, which not only enhances the connection between feature channels but also the spatial connection between crack pixels. The convolutional block proposed in this application is based on the CNN architecture. Compared with other CNN-based backbone networks, it makes up for the deficiency of standard convolution in local spatial feature extraction through spatial displacement operations, making the backbone network more sensitive to the spatial features of the cracks on the surface of the high-voltage switchgear enclosure and enabling it to extract the detailed features of the cracks more precisely. Compared with MobileViT, the FLOPs of the segmentation model in this application only increase by 0.696G, but the mIoU is increased by 2.13%. In contrast, fixed-size convolutional kernels cannot effectively capture features of different scales and sizes and have a limited receptive field, while the dynamic convolutional kernel of CSConv can specifically focus on crack features, thereby improving the segmentation accuracy.

[0101] To analyze the regional weights of the crack pictures on the surface of the high-voltage switchgear enclosure and compare the changes in the crack picture weights of different models, this application introduced gradient-weighted class activation mapping to generate saliency maps for each convolutional layer, thereby highlighting the important regions that affect the crack picture prediction. As Figure 9 shown, ResNet50, ShuffleNet, MobileNet, and Ours were selected for heatmap visualization comparison, respectively showing the crack heatmaps after feature extraction for each layer. From Figure 9It can be seen that there are differences among these four models in terms of their attention to cracks and the handling of the influence of grooving. Specifically, ResNet50 has a relatively high attention to cracks, but it performs poorly in handling the influence of grooving, misidentifying grooving as cracks and resulting in errors. Although ShuffleNet overcomes the influence of grooving, it has insufficient attention to cracks and has defects in identifying crack features in the global area. The MobileNet model focuses on the global area but has difficulty distinguishing between grooving and cracks, and as the network depth increases, the crack feature part will be lost, leading to poor segmentation results. In contrast, the segmentation model proposed in this application pays more attention to the crack area and ignores the invalid area, reflecting the powerful feature extraction ability of CSConv. CSConv can effectively fuse the spatial and channel features of cracks, enhance the features of the effective area, and thus improve the segmentation ability.

[0102] Furthermore, in order to verify the influence of different modules on the segmentation ability, taking ResNet50 as the baseline, CSConv, the frequency feature aggregation module (FFAM), and the attention feature refinement module (AFRM) were gradually added to it, and training was carried out on the self-built dataset. The experimental results are as Figure 10 shown.

[0103] CSConv improves the deficiency of the standard convolution spatial modeling, mainly contributing to the average pixel accuracy of the model. The spatial displacement operation realizes the aggregation of neighboring features at the corresponding positions, improving the average pixel accuracy of the model by 9.3%. And because the displacement operation is parameter-free, not many parameters are added. The frequency feature aggregation module enhances the edge contrast of cracks in the frequency domain, and the windowed patches enhance the local details of the features, contributing 4.95% of the mean intersection over union to the model. The attention feature refinement module reduces the semantic gap between the underlying feature maps, making the model pay more attention to the crack details on the surface of the high-voltage switchgear enclosure, contributing 3.37% of the accuracy to the model.

[0104] Generally speaking, the backbone network of the segmentation model in this application only has 29.2% of the parameter quantity and 29.7% of the FLOPs of ResNet50, but each module improves the segmentation ability of the model.

[0105] In view of the current situation that the performance of existing semantic segmentation models is limited in the task of crack detection on the surface of high-voltage switchgear enclosures, this application proposes a semantic segmentation model for cracks on the surface of high-voltage switchgear enclosures that combines channel-spatial convolution and frequency feature aggregation. By introducing a new convolutional block, it can more effectively model the pixel spatial relationship and channel feature connection, thereby accurately identifying crack pixels; in addition, to enhance the contrast of crack edges, a frequency domain feature aggregation module is proposed, which effectively solves the problem of mismatched frequency domain inputs. At the same time, considering the influence of the imaginary part of the frequency on the features, it effectively models the deep frequency features; finally, an attention feature refinement module is designed to refine the semantic features to further improve the segmentation accuracy.

[0106] Please refer to Figure 11 , this application embodiment also provides a method for semantic segmentation of cracks on the surface of high-voltage switchgear enclosures, including:

[0107] Step 210: Collect images of the surface of the high-voltage switchgear enclosure.

[0108] An industrial camera can be used to collect images of the surface of the high-voltage switchgear enclosure. After the images are collected, preprocessing such as image denoising can also be performed on them.

[0109] Step 220: Input the images of the surface of the high-voltage switchgear enclosure into the crack semantic segmentation model for semantic segmentation.

[0110] The crack semantic segmentation model is trained by using the crack semantic segmentation model training method in the foregoing method embodiment.

[0111] This application uses a semantic segmentation model for cracks on the surface of high-voltage switchgear enclosures that combines channel-spatial convolution, frequency feature aggregation, and attention feature refinement to segment cracks on the surface of high-voltage switchgear enclosures. Through the channel spatial convolution module, it can more effectively model the pixel spatial relationship and channel feature connection, thereby accurately identifying crack pixels; through the frequency domain feature aggregation module, the problem of mismatched frequency domain inputs can be effectively solved. At the same time, considering the influence of the imaginary part of the frequency on the features, it effectively models the deep frequency features and enhances the contrast of crack edges; through the attention feature refinement module, the semantic features are refined, further improving the segmentation accuracy.

[0112] Please refer to Figure 12 , this application embodiment also provides a crack semantic segmentation model training device, including:

[0113] A construction unit for constructing a segmentation network, where the segmentation network includes an encoder, a frequency feature aggregation module, and a decoder;

[0114] A training unit for inputting a crack training image on the surface of the high-voltage switchgear cabinet shell into a segmentation network; extracting features from the crack training image through an encoder to obtain a crack feature map; normalizing the crack feature map through a frequency feature aggregation module, uniformly dividing the normalized crack feature map into window feature maps of a preset size, performing a fast Fourier transform on each window feature map to obtain a window frequency feature map, successively performing convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each window frequency feature map, fusing the merged feature map with the normalized crack feature map to obtain a fused feature map, extracting attention from the fused feature map to obtain a feature attention map, and weighting the crack feature map through the feature attention map to obtain an enhanced feature map; performing semantic segmentation on the enhanced feature map through a decoder to obtain a predicted segmentation result of the crack training image;

[0115] A parameter update unit for calculating a loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and updating the network parameters of the segmentation network through the loss value to obtain a trained crack semantic segmentation model.

[0116] Please refer to Figure 13 In addition, an embodiment of the present application further provides a device for semantic segmentation of cracks on the surface of a high-voltage switchgear cabinet shell, including:

[0117] An image acquisition unit for acquiring an image on the surface of the high-voltage switchgear cabinet shell;

[0118] A segmentation unit for inputting the image on the surface of the high-voltage switchgear cabinet shell into the crack semantic segmentation model for semantic segmentation;

[0119] Wherein, the crack semantic segmentation model is trained by using the crack semantic segmentation model training method in the foregoing method embodiment.

[0120] An embodiment of the present application further provides an electronic device, which includes a processor and a memory;

[0121] The memory is used for storing program codes and transmitting the program codes to the processor;

[0122] The processor is used for executing the crack semantic segmentation model training method in the foregoing method embodiment according to the instructions in the program codes, or executing the method for semantic segmentation of cracks on the surface of the high-voltage switchgear cabinet shell in the foregoing method embodiment.

[0123] An embodiment of the present application further provides a computer-readable storage medium, which is used for storing program codes, and when the program codes are executed by a processor, the crack semantic segmentation model training method in the foregoing method embodiment is implemented, or the method for semantic segmentation of cracks on the surface of the high-voltage switchgear cabinet shell in the foregoing method embodiment is implemented.

[0124] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the devices and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0125] In the specification of this application and the above-mentioned drawings, terms such as "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that comprises a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0126] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist simultaneously. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (one) of the following" or similar expressions refer to any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, in each embodiment of the present application, each functional unit may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0130] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks, or optical discs.

[0131] As described above, the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for training a crack semantic segmentation model, characterized in that Including: Construct a segmentation network, the segmentation network includes an encoder, a frequency feature aggregation module and a decoder; Input the crack training image on the surface of the high-voltage switchgear cabinet into the segmentation network, and extract features from the crack training image through the encoder to obtain a crack feature map; Normalize the crack feature map through the frequency feature aggregation module, evenly divide the normalized crack feature map into window feature maps of a preset size, perform fast Fourier transform on each window feature map to obtain window frequency feature maps, perform convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each of the window frequency feature maps in sequence, fuse the merged feature map with the normalized crack feature map to obtain a fused feature map, extract attention from the fused feature map to obtain a feature attention map, and weight the crack feature map through the feature attention map to obtain an enhanced feature map; Perform semantic segmentation on the enhanced feature map through the decoder to obtain the predicted segmentation result of the crack training image; Calculate the loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, update the network parameters of the segmentation network through the loss value, and obtain a trained crack semantic segmentation model.

2. The method for training a crack semantic segmentation model according to claim 1, wherein The encoder and the decoder are residual networks stacked by improved residual convolution blocks, and the improved residual convolution block includes a channel convolution block, a spatial convolution block and a feature splicing module; The channel convolution block is used to split the input feature map by channels, evaluate the attention of each channel feature map to obtain the attention weights of each channel, multiply each channel attention weight by the convolution kernel and then perform convolution processing on the input feature map to obtain crack channel features; The spatial convolution block is used to group the input feature map, perform spatial displacement operations on each group of feature maps, so that each group of feature maps is feature-aligned along the channel direction to obtain crack spatial features; The feature splicing module is used to splice the crack channel features and the crack spatial features.

3. The method for training a crack semantic segmentation model according to claim 1, wherein The segmentation network further includes a multi-layer perceptron; The multi-layer perceptron is used to extract features from the fused features after feature splicing of the feature maps extracted by each layer in the encoder, and weight the enhanced feature map through the extracted feature map to obtain a weighted enhanced feature map; The performing semantic segmentation on the enhanced feature map through the decoder to obtain the predicted segmentation result of the crack training image includes: Performing semantic segmentation on the weighted enhanced feature map through the decoder to obtain the predicted segmentation result of the crack training image.

4. The method for training a crack semantic segmentation model according to claim 3, wherein The segmentation network further includes an attention feature refinement module, and the attention includes a channel branch module, a spatial branch module and a residual connection module; The spatial branch module is used to perform depth convolution on the input feature map to generate a spatial feature map, perform a pixel shuffle operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the attention feature refinement module is the feature map extracted by the multi-layer perceptron multiplied by the weighted enhanced feature map; the output of the attention feature refinement module and the decoder; The channel branch module is used to perform global average pooling on the input feature map, and then perform channel compression and expansion on the pooled feature map to obtain a channel attention map; The residual connection module is used to perform residual connection on the input feature map, the spatial attention map, and the channel attention map, and the output of the residual connection module is used as the input of the last convolutional layer of the encoder.

5. The method for training a crack semantic segmentation model according to claim 1, wherein The method further includes: Performing data augmentation on the crack training images.

6. A semantic segmentation method for cracks on the surface of the high-voltage switchgear shell, characterized in that, Including: Collecting the surface image of the high-voltage switchgear cabinet; Inputting the surface image of the high-voltage switchgear cabinet into the crack semantic segmentation model for semantic segmentation; Wherein, the crack semantic segmentation model is trained by using the crack semantic segmentation model training method described in any one of claims 1-5.

7. A training device for a crack semantic segmentation model, characterized in that Including: A construction unit for constructing a segmentation network, the segmentation network including an encoder, a frequency feature aggregation module, and a decoder; A training unit for inputting the crack training images on the surface of the high-voltage switchgear cabinet into the segmentation network; extracting features of the crack training images through the encoder to obtain crack feature maps; normalizing the crack feature maps through the frequency feature aggregation module, uniformly dividing the normalized crack feature maps into window feature maps of a preset size, performing fast Fourier transform on each window feature map to obtain window frequency feature maps, performing convolution, channel attention feature extraction, inverse fast Fourier transform, and window merging on each of the window frequency feature maps in sequence, fusing the merged feature maps with the normalized crack feature maps to obtain a fused feature map, performing attention extraction on the fused feature map to obtain a feature attention map, and weighting the crack feature maps through the feature attention map to obtain enhanced feature maps; performing semantic segmentation on the enhanced feature maps through the decoder to obtain the predicted segmentation results of the crack training images; A parameter update unit for calculating a loss value according to the predicted segmentation results of the crack training images and the corresponding standard semantic segmentation results, and updating the network parameters of the segmentation network through the loss value to obtain a trained crack semantic segmentation model.

8. A semantic segmentation device for cracks on the surface of the shell of a high-voltage switchgear, characterized in that, Including: An image acquisition unit for collecting the surface image of the high-voltage switchgear cabinet; A segmentation unit for inputting the surface image of the high-voltage switchgear cabinet into the crack semantic segmentation model for semantic segmentation; wherein, the crack semantic segmentation model is trained by using the crack semantic segmentation model training method described in any one of claims 1-5.

9. An electronic device, characterized in that, The device includes a processor and a memory; The memory is used to store program codes and transmit the program codes to the processor; The processor is configured to execute the crack semantic segmentation model training method according to any one of claims 1-5 based on the instructions in the program code, or execute the crack semantic segmentation method for the surface of the high-voltage switchgear cabinet according to claim 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store program code, which, when executed by a processor, implements the crack semantic segmentation model training method according to any one of claims 1-5, or implements the crack semantic segmentation method for the surface of the high-voltage switchgear cabinet according to claim 6.

Images