Crack semantic segmentation model training method, segmentation method and related device

Abstract

The invention discloses a crack semantic segmentation model training method, a segmentation method and a related device, and the training method comprises the steps: inputting a crack training image into a constructed segmentation network, and extracting a crack feature map of the crack training image through an encoder; the method comprises the following steps: standardizing a crack feature map through a frequency feature aggregation module, uniformly segmenting the standardized crack feature map into window feature maps with preset sizes, and sequentially performing fast Fourier transform, convolution, channel attention feature extraction, inverse fast Fourier transform and window combination on each window feature map to obtain a channel attention feature extraction result; carrying out fusion and attention extraction on the fractured feature map and a standardized fractured feature map, and weighting the fractured feature map to obtain an enhanced feature map; performing semantic segmentation on the enhanced feature map through a decoder to obtain a prediction segmentation result; and calculating a loss value according to a prediction segmentation result and a standard semantic segmentation result to update network parameters and obtain a crack semantic segmentation model. According to the invention, the segmentation precision of the surface crack of the switch cabinet is improved.

Description

Technical Field

[0001] The present application relates to the technical field of image segmentation, and in particular to a crack semantic segmentation model training method, a segmentation method and related devices. Background Art

[0002] High-voltage switchgear is an important equipment in the power system, and its main function is to realize the distribution, control and protection of electric energy. However, due to long-term exposure to complex environments, such as humidity, high temperature, ultraviolet radiation, acid and alkali corrosion, cracks are prone to appear on the surface of the switchgear shell. These cracks not only affect the insulation performance of the equipment, but may also cause internal electrical components to become damp, corroded, or even short-circuited, seriously threatening the safe operation of the power system. Therefore, the detection, evaluation and protection of cracks in the high-voltage switchgear shell are of great significance.

[0003] In the early days, the surface crack detection of high-voltage switchgear casing was done manually, which was time-consuming, labor-intensive and inefficient, and was no longer sufficient to meet the increasing maintenance needs. In recent years, image processing technology based on computer vision has gradually replaced manual work and become the mainstream method for surface crack detection of high-voltage switchgear casing. Traditional image processing algorithms based on machine vision, such as median filtering, threshold algorithm, edge detection, etc. These algorithms are sensitive to interference factors such as noise, illumination changes, and complex backgrounds in the image, and it is difficult to detect small cracks from complex images. Improving detection efficiency and accuracy, reducing the heavy workload of detection personnel, and implementing automatic detection of surface cracks in high-voltage switchgear have become an inevitable trend.

[0004] Due to the defects of traditional image processing algorithms, surface crack detection algorithms for high-voltage switchgear casings based on semantic segmentation have emerged in an endless stream, but the detection accuracy of existing segmentation networks is not high. Summary of the invention

[0005] The present application provides a crack semantic segmentation model training method, a segmentation method and related devices, which are used to improve the segmentation accuracy of cracks on the surface of a high-voltage switch cabinet casing.

[0006] In view of this, the first aspect of the present application provides a crack semantic segmentation model training method, comprising:

[0007] Constructing a segmentation network, wherein the segmentation network includes an encoder, a frequency feature aggregation module, and a decoder;

[0008] Inputting a crack training image on the surface of a high-voltage switch cabinet housing into a segmentation network, performing feature extraction on the crack training image through the encoder, and obtaining a crack feature map;

[0009] The crack feature map is standardized by the frequency feature aggregation module, the standardized crack feature map is evenly divided into window feature maps of preset size, each window feature map is fast Fourier transformed to obtain a window frequency feature map, each window frequency feature map is sequentially convolved, channel attention feature extracted, inverse fast Fourier transformed and window merged, the merged feature map is fused with the standardized crack feature map to obtain a fused feature map, attention is extracted on the fused feature map to obtain a feature attention map, and the crack feature map is weighted by the feature attention map to obtain an enhanced feature map;

[0010] Performing semantic segmentation on the enhanced feature map by the decoder to obtain a predicted segmentation result of the crack training image;

[0011] A loss value is calculated according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and a network parameter of the segmentation network is updated according to the loss value to obtain a trained crack semantic segmentation model.

[0012] Optionally, the encoder and the decoder are residual networks formed by stacking improved residual convolution blocks, and the improved residual convolution blocks include a channel convolution block, a spatial convolution block and a feature splicing module;

[0013] The channel convolution block is used to perform channel splitting on the input feature map, perform attention evaluation on each channel feature map, obtain the attention weight of each channel, multiply the attention weight of each channel by the convolution kernel, and then perform convolution processing on the input feature map to obtain the crack channel feature;

[0014] The spatial convolution block is used to group the input feature maps, and perform spatial displacement operations on each group of feature maps, so that each group of feature maps is feature-aligned along the channel direction to obtain crack spatial features;

[0015] The feature splicing module is used to perform feature splicing on the fracture channel feature and the fracture space feature.

[0016] Optionally, the segmentation network further includes a multi-layer perceptron;

[0017] The multi-layer perceptron is used to extract features from the fused features after feature splicing of the feature maps extracted from each layer in the encoder, and weight the enhanced feature map by the extracted feature map to obtain a weighted enhanced feature map;

[0018] The performing semantic segmentation on the enhanced feature map by the decoder to obtain a predicted segmentation result of the crack training image includes:

[0019] The decoder performs semantic segmentation on the weighted enhanced feature map to obtain a predicted segmentation result of the crack training image.

[0020] Optionally, the segmentation network further includes an attention feature refinement module, wherein the attention includes a channel branch module, a spatial branch module and a residual connection module;

[0021] The spatial branch module is used to perform deep convolution on the input feature map to generate a spatial feature map, perform pixel shuffling operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the attention feature refinement module is the feature map obtained by multiplying the feature map extracted by the multilayer perceptron and the weighted enhancement feature map; the output of the attention feature refinement module is connected to the decoder;

[0022] The channel branch module is used to perform global average pooling on the input feature map, and then perform channel compression and expansion on the pooled feature map to obtain a channel attention map;

[0023] The residual connection module is used to perform residual connection on the input feature map, the spatial attention map and the channel attention map, and the output of the residual connection module is used as the input of the last convolutional layer of the encoder.

[0024] Optionally, the method further includes:

[0025] Data enhancement is performed on the crack training image.

[0026] A second aspect of the present application provides a method for semantic segmentation of surface cracks of a high-voltage switch cabinet casing, comprising:

[0027] Collect surface images of high-voltage switchgear casing;

[0028] The surface image of the high-voltage switch cabinet casing is input into a crack semantic segmentation model for semantic segmentation; wherein the crack semantic segmentation model is trained using any crack semantic segmentation model training method described in the first aspect.

[0029] The third aspect of the present application provides a crack semantic segmentation model training device, comprising:

[0030] A construction unit, used to construct a segmentation network, wherein the segmentation network includes an encoder, a frequency feature aggregation module and a decoder;

[0031] A training unit is used to input a crack training image on the surface of a high-voltage switch cabinet shell into a segmentation network; perform feature extraction on the crack training image through the encoder to obtain a crack feature map; standardize the crack feature map through the frequency feature aggregation module, evenly divide the standardized crack feature map into window feature maps of a preset size, perform fast Fourier transform on each window feature map to obtain a window frequency feature map, perform convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each of the window frequency feature maps in sequence, fuse the merged feature map with the standardized crack feature map to obtain a fused feature map, perform attention extraction on the fused feature map to obtain a feature attention map, and weight the crack feature map through the feature attention map to obtain an enhanced feature map; perform semantic segmentation on the enhanced feature map through the decoder to obtain a predicted segmentation result of the crack training image;

[0032] The parameter updating unit is used to calculate the loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and update the network parameters of the segmentation network according to the loss value to obtain a trained crack semantic segmentation model.

[0033] A fourth aspect of the present application provides a device for semantic segmentation of surface cracks in a high-voltage switch cabinet housing, comprising:

[0034] An image acquisition unit, used for acquiring an image of the surface of a high-voltage switch cabinet housing;

[0035] A segmentation unit, used for inputting the surface image of the high-voltage switch cabinet shell into a crack semantic segmentation model for semantic segmentation;

[0036] The crack semantic segmentation model is trained by using any crack semantic segmentation model training method described in the first aspect.

[0037] A fifth aspect of the present application provides an electronic device, characterized in that the device includes a processor and a memory;

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is used to execute any one of the crack semantic segmentation model training methods described in the first aspect, or execute the high-voltage switch cabinet casing surface crack semantic segmentation method described in the second aspect according to the instructions in the program code.

[0040] The sixth aspect of the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, it implements any one of the crack semantic segmentation model training methods described in the first aspect, or implements the high-voltage switch cabinet casing surface crack semantic segmentation method described in the second aspect.

[0041] It can be seen from the above technical solutions that this application has the following advantages:

[0042] The present application provides a crack semantic segmentation model training method, which adds a frequency feature aggregation module between the encoder and the decoder to extract crack frequency features, and uses a windowing method to solve the input mismatch problem in the frequency domain. By modeling deep feature frequencies, the defect of difficulty in distinguishing similar pixels in the time domain is improved, and the crack edge features are enhanced, thereby improving the crack segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a crack semantic segmentation model training method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a channel space convolution module provided in an embodiment of the present application;

[0046] Figure 3 A structural intention of the standard residual convolution block and the improved residual convolution block provided in the embodiment of the present application;

[0047] Figure 4 A structural intention of the frequency feature aggregation module provided in the embodiment of the present application;

[0048] Figure 5 A structural intent of the crack semantic segmentation model provided in the embodiment of the present application;

[0049] Figure 6 Comparison of training results of different models provided in the embodiments of the present application on various data sets;

[0050] Figure 7 The visualization results of the segmentation maps output by each model provided in the embodiments of the present application;

[0051] Figure 8The backbone network performance comparison results provided by the embodiments of the present application;

[0052] Fig. 9 The heat map visualization results of each model provided in the embodiments of the present application;

[0053] Fig.10 The model ablation experiment results provided in the embodiments of the present application;

[0054] Fig.11 A schematic flow chart of a method for semantic segmentation of surface cracks in a high-voltage switch cabinet casing provided in an embodiment of the present application;

[0055] Fig.12 A structural schematic diagram of a crack semantic segmentation model training device provided in an embodiment of the present application;

[0056] Fig.13 A structural schematic diagram of a device for semantic segmentation of cracks on the surface of a switchgear casing provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] For easier understanding, please refer to Figure 1 , the embodiment of the present application provides a crack semantic segmentation model training method, including:

[0059] Step 110: construct a segmentation network.

[0060] The segmentation network in the present application includes an encoder, a frequency feature aggregation module and a decoder, wherein a residual network (such as ResNet-50) can be used as an encoder and a decoder.

[0061] Step 120: input the crack training image on the surface of the high-voltage switch cabinet casing into the segmentation network, perform feature extraction on the crack training image through an encoder, and obtain a crack feature map.

[0062] Crack training images on the surface of the high-voltage switchgear casing can be obtained from existing public data sets, or crack images on the surface of the high-voltage switchgear casing can be used as training images and annotated to obtain standard crack segmentation images.

[0063] In order to strengthen the network training process, prevent the network from falling into the local optimal solution, and ensure the segmentation accuracy of the model, the data set can be enhanced. The crack training images can be randomly cropped, randomly rotated, randomly scaled, and other methods can be used to enhance the data. The training images can also be subjected to image denoising to reduce the brightness distribution deviation of the data set and smooth the image noise, thereby reducing the impact of jitter, noise, and ghosting problems that occur during the collection of the high-voltage switchgear shell surface crack data set on model training.

[0064] In one embodiment, the existing residual network can be directly used as an encoder to perform feature extraction.

[0065] In another embodiment, in order to improve the crack segmentation accuracy of the model, the existing residual network is improved, and specifically the residual convolution block in the residual network is improved to better encode the surface crack characteristics of the high-voltage switch cabinet casing. This application proposes a channel space convolution module (CSConv), such as Figure 2 As shown, the channel spatial convolution module includes a channel convolution block, a spatial convolution block and a feature splicing module.

[0066] The channel convolution block is used to split the input feature map by channel, perform attention evaluation on each channel feature map, obtain the attention weight of each channel, multiply the attention weight of each channel by the convolution kernel (Kernal Size), and then convolve the input feature map to obtain the crack channel feature; the channel convolution block first splits the input feature map by channel, and then uses attention to evaluate the importance of each channel to generate weights, and multiplies the weights by the convolution kernel to make the convolution kernel have a dynamic size; at this time, convolving the input feature map again can discard unimportant features, so that the encoder can better extract the surface crack features of the high-voltage switchgear casing.

[0067] The spatial convolution block is used to group the input feature maps (Clusters) and perform spatial displacement operations (Padding) on ​​each group of feature maps, so that each group of feature maps is feature-aligned along the channel direction to obtain the crack space feature (TargetFeature). A parameter-free spatial displacement operation is introduced in the spatial convolution block to align the input feature map with adjacent features along the channel direction, thereby realizing spatial feature aggregation. Specifically, the input feature maps are first grouped, and spatial displacement is performed on feature channels of different groups to improve the aggregation of adjacent features at the corresponding positions of the convolution; finally, the feature splicing module splices the crack channel features and the crack space features, and fuses the crack channel features and the crack space features, wherein the crack space features are convolved and then spliced ​​with the crack channel features.

[0068] In this application, the channel space convolution module is used to construct the residual convolution block, which can better encode the surface crack characteristics of the high-voltage switch cabinet shell. Please refer to Figure 3 , Figure 3 (a) in is the standard residual convolution block. Figure 3 (b) in the figure is the improved residual convolution block (DSBlock). The improved residual convolution block replaces the original two 3x3 standard convolutions with CSConv to enhance the feature extraction capability. An encoder is constructed based on the improved residual convolution block to extract the deep features of the surface cracks of the high-voltage switchgear casing, and the convolution parameters are reduced while ensuring performance.

[0069] Step 130, standardize the crack feature map through the frequency feature aggregation module, evenly divide the standardized crack feature map into window feature maps of preset size, perform fast Fourier transform on each window feature map to obtain a window frequency feature map, perform convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each window frequency feature map in sequence, fuse the merged feature map with the standardized crack feature map to obtain a fused feature map, perform attention extraction on the fused feature map to obtain a feature attention map, and weight the crack feature map through the feature attention map to obtain an enhanced feature map.

[0070] In the time domain, the influence of complex background will confuse the boundary between cracks and background, making crack detection more difficult. In addition, time domain signals mainly focus on the changes in time of signals, but have limited understanding of the structural information of signals in space, making the model inaccurate for detecting fine structures such as cracks.

[0071] To address the above issues, this application proposes a frequency feature aggregation module (Frequency Feature Aggregation Module), such as Figure 4 As shown. The frequency feature aggregation module first uses the normalization layer (LayerNorm) to normalize the input crack feature map to average the input distribution. In order to overcome the feature loss caused by frequency input mismatch, the frequency feature aggregation module evenly divides the standardized crack feature map into windows of size N (WindowsPartition), and the size of each window is SN=[B,C,N,N]. These windows are then subjected to two-dimensional fast Fourier transform (2D FFT) to the frequency domain:

[0072]

[0073] Where F(*) represents fast Fourier transform; b and c represent the batch and channel indices; u and v represent the u-th horizontal and v-th vertical frequency in the spectrum.

[0074] In order to further extract the depth characteristics of the surface cracks of the high-voltage switchgear casing, this application considers both the real and imaginary parts of the complex frequency and uses convolution operation to aggregate the frequency characteristics. The processing of any complex frequency is as follows:

[0075]

[0076] in, , Represents the real and imaginary parts of frequency; , are the real and imaginary parts of the frequency after convolution processing; Kr, Ki represent the real and imaginary parts of the complex convolution kernel.

[0077] Then, the channel attention mechanism (CA) is used to model the surface crack features of the high-voltage switchgear casing in the spectrum, highlighting the useful features in the frequency channel; the window frequency feature map is then inversely transformed into a two-dimensional fast Fourier transform (2DIFFT) and then the window is merged (Patch Mergin), and the size of the merged feature map is restored to [B, C, H, W]. Finally, the normalization layer and the sigmoid activation function are used in sequence to generate a feature attention map, and the generated attention map is multiplied with the original input feature map to enhance the crack features in the feature map and obtain an enhanced feature map.

[0078] This application designs a frequency feature aggregation module based on the characteristics of cracks, uses a windowing method to solve the problem of input mismatch in the frequency domain, and considers the complex frequency of the features; through channel attention modeling of deep feature frequencies, it improves the defect of difficulty in distinguishing similar pixels in the time domain and enhances the edge characteristics of cracks.

[0079] Step 140: Perform semantic segmentation on the enhanced feature map through a decoder to obtain a predicted segmentation result of the crack training image.

[0080] In one embodiment, after the frequency feature aggregation module extracts the enhanced feature map, it is directly input into the decoder for semantic segmentation to obtain a predicted segmentation result of the crack training image.

[0081] In another embodiment, a multi-layer perceptron (MLP) can be introduced into the segmentation network, and the multi-layer perceptron includes a fully connected layer FC, a Relu activation function and a Softmax layer; the feature maps extracted by each layer of the encoder are concatenated and input into the multi-layer perceptron, the multi-layer perceptron extracts the input features, generates an attention map to enhance the weights of the underlying useful features, and the feature map extracted by the multi-layer perceptron is weighted to the enhanced feature map to obtain a weighted enhanced feature map; and then the weighted enhanced feature map is input into the decoder for semantic segmentation.

[0082] In yet another embodiment, in order to reduce feature redundancy in multi-scale feature fusion and reduce the drowning of useful features, an attention feature refinement module is added to the segmentation network, such as Figure 5 As shown, the attention feature refinement module is used to refine the crack features and improve the segmentation accuracy.

[0083] Attention includes channel branch module, spatial branch module and residual connection module; the spatial branch module is used to perform deep convolution on the input feature map to generate a spatial feature map, perform pixel shuffling operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the attention feature refinement module is the feature map obtained by multiplying the feature map extracted by the multilayer perceptron and the weighted enhancement feature map; the output of the attention feature refinement module is connected to the decoder; the channel branch module is used to perform global average pooling on the input feature map, and then perform channel compression and expansion on the pooled feature map to obtain a channel attention map; the residual connection module is used to perform residual connection on the input feature map, spatial attention map and channel attention map, and the output of the residual connection module is used as the input of the last convolutional layer of the encoder.

[0084] Cracks in high-voltage switchgear usually present different shapes, including linear, mesh, and crack-like. Linear cracks are usually long and thin cracks extending along the extension direction, while mesh cracks present a cross crack network, and crack-like cracks present small cracks similar to cracks. As the network deepens, the backbone network can extract rich crack semantic features, which are crucial for distinguishing cracks from backgrounds. However, as the local features of the cracks are extracted, the spatial dimension of the feature map continues to shrink, resulting in the loss of spatial correlation information. Therefore, this application introduces an attention feature refinement module to alleviate the semantic differences between the feature maps extracted by the backbone network, enhance their spatial correlation, and reduce redundant features. The attention feature refinement module consists of a spatial branch and a channel branch. The spatial branch first generates a spatial feature map of the crack using deep convolution, then extracts the correlation between pixels through a pixel shuffle operation, and generates an attention map through a Sigmoid function. The channel branch first performs global average pooling on the input feature map and converts it into a tensor of size 1×1×C (where C is the number of channels). Subsequently, the number of channels is reduced to one third of the original number of channels by reducing and expanding the layers, and then restored to enhance the semantic details of the channels, thereby improving the crack segmentation ability. Finally, the input feature map is residually connected with the attention maps generated by the two branches to prevent network degradation, and the feature map after residual connection is input to the last convolutional layer in the encoder. The final semantic segmentation map of the surface cracks of the high-voltage switchgear casing is generated through the convolutional layer and upsampling to obtain the predicted segmentation result.

[0085] This application aims at the problem that the surface of high-voltage switchgear is prone to aging and cracking due to long-term exposure to electric fields, mechanical stress and environmental factors (such as temperature changes, humidity, dirt, ultraviolet radiation, etc.), which seriously affects the safety of power supply; especially in the case of poor quality or long-term use, it is crucial to timely identify and fill the surface cracks of the high-voltage switchgear shell. In view of the fact that the existing semantic segmentation model has a poor performance in segmenting images of surface cracks in the high-voltage switchgear shell, standard convolution is difficult to capture the spatial and channel coupling relationship between pixels, and it is difficult to distinguish crack pixels from background pixels under complex backgrounds. This application proposes a semantic segmentation model for surface cracks in the high-voltage switchgear shell that combines channel-space convolution and frequency feature aggregation; by grouping spatial displacement and dynamic convolution kernel weights, pixel spatial relationships and channel feature connections are modeled simultaneously to accurately identify crack pixels; in order to enhance the contrast of crack edges, a frequency domain feature aggregation module is proposed, which uses a simple windowing strategy to solve the problem of mismatched frequency domain inputs, and takes into account the influence of the imaginary part of the frequency on the features, effectively models deep frequency features, and aggregates the crack body and edge information through the frequency domain feature aggregation module; finally, an attention feature refinement module is designed to refine semantic features to further improve segmentation accuracy.

[0086] Step 150: Calculate a loss value based on the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and update the network parameters of the segmentation network according to the loss value to obtain a trained crack semantic segmentation model.

[0087] The loss value is calculated based on the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result. The loss function of the existing segmentation network can be used, and no specific limitation is made here. The network parameters of the segmentation network are updated by the loss value until the segmentation network converges (such as reaching the maximum number of training times or the training error is lower than the preset error threshold, etc.), and a trained crack semantic segmentation model is obtained.

[0088] In order to verify the effectiveness of the segmentation model proposed in this application, a crack dataset was introduced. The dataset was collected by an industrial array camera of a German company with a resolution of 512×512 pixels. In order to improve the robustness of the segmentation model, different lighting conditions, shadows, debris and other scenes were selected for shooting when taking crack pictures, and a total of 2026 crack pictures were collected. In order to improve the generalization ability of the model, the data set was expanded using the data enhancement method. The data enhancement method was used to enhance the quality of the data set and expand it, and finally a total of 6400 crack images were obtained. The crack dataset was divided into a training set and a test set in a ratio of 8:2, and Labelme software was used for fine annotation.

[0089] The programming environment of the experiment is Python 3.6, and the torch version is 1.10.8. The hardware environment is Intel I7-7800X CPU with NVIDIA GeForce RTX3090 graphics card with 24GB video memory and 16GB RAM. In order to prevent the model from falling into the local optimal point during training, the Adam optimizer is used to optimize the training.

[0090] In order to evaluate segmentation models of different depths, this application uses classic evaluation indicators in the field of semantic segmentation, including mean intersection over Union (mIoU), mean pixel accuracy (mPA), and precision. As shown in the following formula:

[0091]

[0092]

[0093]

[0094] Where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. A means that the pixel accuracy of each image is summed and averaged to obtain the average pixel accuracy of the entire dataset. Pi is the accuracy of the crack pixel identified in image i compared with the pixel in the label, and n is the number of samples in the test set. This work also introduces FLOPs (floating point operations) and Params to measure the computational complexity of the network.

[0095] This application verifies the proposed semantic segmentation model for surface cracks in high-voltage switchgear enclosures. The proposed segmentation model is compared with the currently popular semantic segmentation models Unet, ResUnet, DeepLabV3, and AttentionUnet. The experimental results are as follows: Figure 6 As shown, Figure 7 The performance of each model on the Self-built, Crack Tree and CrackForest Datasets is demonstrated.

[0096] from Figure 6It can be seen that the segmentation model proposed in this application performs better than other models on various data sets, achieving a balance between performance and parameters. In terms of parameter quantity and FLOPs, the segmentation model proposed in this application has only 12.836M parameters and 6.846G FLOPs, which shows that the segmentation model can characterize more crack features with relatively few parameters. Although DeepLabV3+ has fewer parameters, it is not as good as the segmentation model proposed in this application in terms of segmentation performance. In terms of segmentation accuracy, the accuracy of the model proposed in this application on three data sets is 85.99%, 77.15%, and 83.07%, respectively. Compared with Unet and ResUnet, the segmentation model proposed in this application has been greatly improved, leading by 11.57% and 10.4% on the self-built data set, respectively. Compared with DeepLabV3+, the dilated convolution greatly reduces the parameter quantity, but also reduces the segmentation performance. AttentionUnet adds multiple attention mechanisms on the basis of Unet, so that the accuracy is the highest, but it also increases the computational complexity of the model, making it difficult to train. In contrast, the segmentation model proposed in this application is greatly reduced in training difficulty without sacrificing too much accuracy.

[0097] Figure 7 The visualization results of the segmentation of each model are shown. The first column is the original image, the second column is the image annotation, and the third to fifth columns are ours, DeepLabV3+, Unet, ResUnet and AttentionUnet respectively. Six typical images are taken from each data set. Comparing the image segmentation results, the segmentation model proposed in this application has stronger generalization performance and is more accurate for the surface crack segmentation of the high-voltage switch cabinet shell under complex backgrounds. DeepLabV3+ is seriously under-segmented when facing complex cracks, especially at the junction of cracks. This is because DeeplabV3+ does not consider the correlation between spatial information and perception field, and cannot fully utilize the correlation between shallow semantics and deep semantics. Unet and ResUnet only obtain local context information through standard convolution and lack rich spatial details. Therefore, they are seriously under-segmented and over-segmented when the contrast between cracks and background is low. AttentionUnet uses multiple attention mechanisms to model spatial and channel contexts, which can achieve good segmentation results, but still shows limited segmentation ability for crack details.

[0098] In comprehensive comparison, the model proposed in this application is ahead of the current popular models in terms of performance. This is because the model proposed in this application is constructed by lightweight CSConv, which greatly reduces the parameters and calculation amount of the model while maintaining good feature extraction capabilities; and the added frequency feature aggregation module and attention feature refinement module further improve the model segmentation accuracy.

[0099] Furthermore, this application also evaluates the feature extraction capabilities of the proposed segmentation model compared to the currently popular backbone networks. These include the backbone networks based on the CNN architecture: ShuffleNetV2, MobileNetV3, GhostNet, EfficientNetV2, and the hybrid CNN and ViT backbone network MobileViT. Each model is trained on a self-built crack dataset, and the experimental results are as follows: Figure 8 shown.

[0100] according to Figure 8 It can be seen that the segmentation model proposed in this application is superior to other popular backbone networks in the ability to extract crack features. This is due to the modeling of local features of cracks on the surface of the high-voltage switchgear casing by CSConv, which not only enhances the connection between feature channels, but also enhances the spatial connection between crack pixels. The convolution block proposed in this application is based on the CNN architecture. Compared with other CNN-based backbone networks, the spatial displacement operation makes up for the shortcomings of standard convolution in local spatial feature extraction, making the backbone network more sensitive to the spatial features of cracks on the surface of the high-voltage switchgear casing, and can extract the detailed features of the cracks more finely. Compared with MobileViT, the FLOPs of the segmentation model of this application only increased by 0.696G, but increased the mIoU by 2.13%. In contrast, fixed-size convolution kernels cannot effectively capture features of different scales and sizes, and have limited receptive fields, while the dynamic convolution kernel of CSConv can focus on crack features in a targeted manner, thereby improving segmentation accuracy.

[0101] In order to analyze the regional weights of crack images on the surface of high-voltage switchgear casings and compare the changes in crack image weights of different models, this application introduces gradient weighted class activation mapping to generate a saliency map for each convolutional layer, thereby highlighting the important areas that affect crack image prediction. Fig. 9 As shown in the figure, ResNet50, ShuffleNet, MobileNet and Ours are selected for heat map visualization comparison, and the crack heat map after feature extraction of each layer is displayed respectively. Fig. 9It can be seen that these four models differ in the degree of attention paid to cracks and the impact of grooves. Specifically, ResNet50 pays more attention to cracks, but does not handle the impact of grooves well, mistaking grooves for cracks, resulting in errors. Although ShuffleNet has overcome the impact of grooves, it does not pay enough attention to cracks and has defects in the recognition of crack features in the global area. The MobileNet model focuses on the global area, but it is difficult to distinguish between grooves and cracks, and as the network depth increases, part of the crack features will be lost, resulting in poor segmentation results. In contrast, the segmentation model proposed in this application pays more attention to the crack area and ignores the invalid area, reflecting the powerful feature extraction ability of CSConv. CSConv can effectively fuse the spatial and channel features of cracks, enhance the features of the effective area, and thus improve the segmentation ability.

[0102] Furthermore, in order to verify the impact of different modules on segmentation ability, ResNet50 was used as the baseline, and CSConv, frequency feature aggregation module (FFAM) and attention feature refinement module (AFRM) were gradually added on its basis, and trained on a self-built dataset. The experimental results are shown in the figure. Fig.10 shown.

[0103] CSConv improves the shortcomings of standard convolutional spatial modeling, mainly contributing to the average pixel accuracy of the model. The spatial displacement operation realizes the aggregation of neighboring features at corresponding positions, which improves the average pixel accuracy of the model by 9.3%. And because the shift operation is parameter-free, it does not increase many parameters. The frequency feature aggregation module enhances the edge contrast of cracks in the frequency domain, and the windowed patch enhances the local details of the features, contributing an average intersection-over-union ratio of 4.95% to the model. The attention feature refinement module reduces the semantic gap between the underlying feature maps, making the model pay more attention to the details of the surface cracks of the high-voltage switchgear casing, contributing 3.37% accuracy to the model.

[0104] In general, the backbone network of the segmentation model of this application has only 29.2% of the parameters and 29.7% of the FLOPs of ResNet50, but each module improves the segmentation ability of the model.

[0105] In view of the current situation that the existing semantic segmentation model has limited performance in the task of detecting surface cracks in high-voltage switchgear casings, this application proposes a semantic segmentation model for surface cracks in high-voltage switchgear casings that combines channel-space convolution with frequency feature aggregation. By introducing a new convolution block, it is possible to more effectively model the spatial relationship of pixels and the connection between channel features, thereby accurately identifying crack pixels; in addition, in order to enhance the contrast of crack edges, a frequency domain feature aggregation module is proposed, which effectively solves the problem of mismatched frequency domain inputs, and at the same time, considering the influence of the imaginary part of the frequency on the features, effectively models deep frequency features; finally, an attention feature refinement module is designed to refine the semantic features to further improve segmentation accuracy.

[0106] Please refer to Fig.11 The embodiment of the present application also provides a method for semantic segmentation of surface cracks of a high-voltage switch cabinet housing, comprising:

[0107] Step 210: Acquire a surface image of a high-voltage switch cabinet housing.

[0108] Industrial cameras can be used to capture images of the surface of the high-voltage switch cabinet housing. After the images are captured, they can also be pre-processed, such as image denoising.

[0109] Step 220: Input the surface image of the high-voltage switch cabinet casing into the crack semantic segmentation model for semantic segmentation.

[0110] The crack semantic segmentation model is trained using the crack semantic segmentation model training method in the aforementioned method embodiment.

[0111] The present application adopts a semantic segmentation model of surface cracks in a high-voltage switchgear casing that combines channel-space convolution, frequency feature aggregation, and attention feature refinement to perform surface crack segmentation of a high-voltage switchgear casing. The channel-space convolution module can more effectively model the pixel spatial relationship and the channel feature connection, thereby accurately identifying crack pixels; the frequency domain feature aggregation module can effectively solve the problem of frequency domain input mismatch, while taking into account the influence of the imaginary part of the frequency on the features, effectively modeling the deep frequency features, and enhancing the contrast of the crack edges; the attention feature refinement module refines the semantic features, thereby further improving the segmentation accuracy.

[0112] Please refer to Fig.12 , the embodiment of the present application also provides a crack semantic segmentation model training device, including:

[0113] A construction unit, used to construct a segmentation network, the segmentation network includes an encoder, a frequency feature aggregation module and a decoder;

[0114] A training unit is used to input a crack training image on the surface of a high-voltage switch cabinet shell into a segmentation network; extract features of the crack training image through an encoder to obtain a crack feature map; standardize the crack feature map through a frequency feature aggregation module, evenly divide the standardized crack feature map into window feature maps of a preset size, perform fast Fourier transform on each window feature map to obtain a window frequency feature map, perform convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each window frequency feature map in sequence, fuse the merged feature map with the standardized crack feature map to obtain a fused feature map, perform attention extraction on the fused feature map to obtain a feature attention map, and weight the crack feature map through the feature attention map to obtain an enhanced feature map; perform semantic segmentation on the enhanced feature map through a decoder to obtain a predicted segmentation result of the crack training image;

[0115] The parameter updating unit is used to calculate the loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and update the network parameters of the segmentation network according to the loss value to obtain a trained crack semantic segmentation model.

[0116] Please refer to Fig.13 The embodiment of the present application also provides a semantic segmentation device for surface cracks of a high-voltage switch cabinet housing, comprising:

[0117] An image acquisition unit, used for acquiring an image of the surface of a high-voltage switch cabinet housing;

[0118] A segmentation unit, used for inputting the surface image of the high-voltage switch cabinet shell into the crack semantic segmentation model for semantic segmentation;

[0119] The crack semantic segmentation model is trained using the crack semantic segmentation model training method in the aforementioned method embodiment.

[0120] The embodiment of the present application also provides an electronic device, the device comprising a processor and a memory;

[0121] The memory is used to store the program code and transmit the program code to the processor;

[0122] The processor is used to execute the crack semantic segmentation model training method in the aforementioned method embodiment according to the instructions in the program code, or execute the high-voltage switch cabinet casing surface crack semantic segmentation method in the aforementioned method embodiment.

[0123] An embodiment of the present application also provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, it implements the crack semantic segmentation model training method in the aforementioned method embodiment, or implements the high-voltage switch cabinet casing surface crack semantic segmentation method in the aforementioned method embodiment.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0125] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0126] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0127] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0130] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0131] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A crack semantic segmentation model training method, characterized in that: include: Constructing a segmentation network, wherein the segmentation network includes an encoder, a frequency feature aggregation module, and a decoder; Inputting a crack training image on the surface of a high-voltage switch cabinet housing into a segmentation network, performing feature extraction on the crack training image through the encoder, and obtaining a crack feature map; The crack feature map is standardized by the frequency feature aggregation module, the standardized crack feature map is evenly divided into window feature maps of preset size, each window feature map is fast Fourier transformed to obtain a window frequency feature map, each window frequency feature map is sequentially convolved, channel attention feature extracted, inverse fast Fourier transformed and window merged, the merged feature map is fused with the standardized crack feature map to obtain a fused feature map, attention is extracted on the fused feature map to obtain a feature attention map, and the crack feature map is weighted by the feature attention map to obtain an enhanced feature map; Performing semantic segmentation on the enhanced feature map by the decoder to obtain a predicted segmentation result of the crack training image; A loss value is calculated according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and a network parameter of the segmentation network is updated according to the loss value to obtain a trained crack semantic segmentation model.

2. The crack semantic segmentation model training method according to claim 1, characterized in that: The encoder and the decoder are residual networks composed of improved residual convolution blocks stacked together, wherein the improved residual convolution blocks include a channel convolution block, a spatial convolution block and a feature splicing module; The channel convolution block is used to perform channel splitting on the input feature map, perform attention evaluation on each channel feature map, obtain the attention weight of each channel, multiply the attention weight of each channel by the convolution kernel, and then perform convolution processing on the input feature map to obtain the crack channel feature; The spatial convolution block is used to group the input feature maps, and perform spatial displacement operations on each group of feature maps, so that each group of feature maps is feature-aligned along the channel direction to obtain crack spatial features; The feature splicing module is used to perform feature splicing on the fracture channel feature and the fracture space feature.

3. The crack semantic segmentation model training method according to claim 1, characterized in that: The segmentation network also includes a multi-layer perceptron; The multi-layer perceptron is used to extract features from the fused features after feature splicing of the feature maps extracted from each layer in the encoder, and weight the enhanced feature map by the extracted feature map to obtain a weighted enhanced feature map; The performing semantic segmentation on the enhanced feature map by the decoder to obtain a predicted segmentation result of the crack training image includes: The decoder performs semantic segmentation on the weighted enhanced feature map to obtain a predicted segmentation result of the crack training image.

4. The crack semantic segmentation model training method according to claim 3, characterized in that: The segmentation network also includes an attention feature refinement module, wherein the attention includes a channel branch module, a spatial branch module, and a residual connection module; The spatial branch module is used to perform deep convolution on the input feature map to generate a spatial feature map, perform pixel shuffling operation on the spatial feature map to extract the correlation between pixels, and generate a spatial attention map through an activation function; the input feature map of the attention feature refinement module is the feature map obtained by multiplying the feature map extracted by the multilayer perceptron and the weighted enhancement feature map; the output of the attention feature refinement module is connected to the decoder; The channel branch module is used to perform global average pooling on the input feature map, and then perform channel compression and expansion on the pooled feature map to obtain a channel attention map; The residual connection module is used to perform residual connection on the input feature map, the spatial attention map and the channel attention map, and the output of the residual connection module is used as the input of the last convolutional layer of the encoder.

5. The crack semantic segmentation model training method according to claim 1, characterized in that: The method further comprises: Data enhancement is performed on the crack training image.

6. A semantic segmentation method for surface cracks of a high-voltage switch cabinet casing, characterized in that: include: Collect surface images of high-voltage switchgear casing; Inputting the surface image of the high-voltage switch cabinet shell into a crack semantic segmentation model for semantic segmentation; The crack semantic segmentation model is trained by the crack semantic segmentation model training method according to any one of claims 1 to 5.

7. A crack semantic segmentation model training device, characterized in that: include: A construction unit, used to construct a segmentation network, wherein the segmentation network includes an encoder, a frequency feature aggregation module and a decoder; A training unit is used to input a crack training image on the surface of a high-voltage switch cabinet shell into a segmentation network; perform feature extraction on the crack training image through the encoder to obtain a crack feature map; standardize the crack feature map through the frequency feature aggregation module, evenly divide the standardized crack feature map into window feature maps of a preset size, perform fast Fourier transform on each window feature map to obtain a window frequency feature map, perform convolution, channel attention feature extraction, inverse fast Fourier transform and window merging on each of the window frequency feature maps in sequence, fuse the merged feature map with the standardized crack feature map to obtain a fused feature map, perform attention extraction on the fused feature map to obtain a feature attention map, and weight the crack feature map through the feature attention map to obtain an enhanced feature map; perform semantic segmentation on the enhanced feature map through the decoder to obtain a predicted segmentation result of the crack training image; The parameter updating unit is used to calculate the loss value according to the predicted segmentation result of the crack training image and the corresponding standard semantic segmentation result, and update the network parameters of the segmentation network according to the loss value to obtain a trained crack semantic segmentation model.

8. A semantic segmentation device for surface cracks of a high-voltage switch cabinet housing, characterized in that: include: An image acquisition unit, used for acquiring an image of the surface of a high-voltage switch cabinet housing; A segmentation unit is used to input the surface image of the high-voltage switch cabinet shell into a crack semantic segmentation model for semantic segmentation; wherein the crack semantic segmentation model is trained using the crack semantic segmentation model training method described in any one of claims 1 to 5.

9. An electronic device, characterized in that: The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the crack semantic segmentation model training method described in any one of claims 1 to 5, or execute the high-voltage switch cabinet casing surface crack semantic segmentation method described in claim 6 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and when the program code is executed by the processor, it implements the crack semantic segmentation model training method described in any one of claims 1 to 5, or implements the high-voltage switch cabinet casing surface crack semantic segmentation method described in claim 6.

Images