Icing image segmentation method and device, electronic equipment and storage medium
The method enhances ice image segmentation on power transmission lines by using dilated convolution and attention units to optimize the model, addressing inefficiencies in existing deep learning methods, thereby improving training efficiency and accuracy.
Patent Information
- Application Number
- CN202411106476.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-13
AI Technical Summary
Current ice image segmentation methods based on deep learning are inefficient due to a large number of parameters, leading to time-consuming model training and reduced operational efficiency in power transmission line ice detection.
A method utilizing dilated convolution combined with an attention unit and an upsampling module to optimize the initial segmentation model, incorporating a feature extraction module to enhance ice feature integration across multiple scales while reducing parameter calculation.
Improves the efficiency and accuracy of ice-covered image segmentation on power transmission lines by optimizing the model to meet preset conditions, enabling effective discrimination of ice coverage.
Smart Images

Figure CN119068190A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of power transmission line safety control, and in particular to an ice-covered image segmentation method, device, electronic device and storage medium. Background Art
[0002] Transmission lines are easily affected by ice and snow in winter. Ice not only increases the weight of the transmission lines, resulting in increased mechanical loads, but may also cause safety accidents such as wire jumping and wire breaking, seriously affecting the safe operation of the power system. Therefore, real-time monitoring and timely clearing of ice on transmission lines is an important task to ensure the safety of the power system.
[0003] In recent years, deep learning technology has made significant progress in the field of computer vision, especially in image classification, target detection and image segmentation. The automated ice detection and segmentation method based on deep learning can automatically extract ice features through large-scale data training models, adapt to complex and changing environments, and significantly improve the accuracy and robustness of ice image segmentation of power transmission lines. However, in the current ice image segmentation method based on deep learning, there are too many parameters in the deep learning network, which makes the model training time-consuming and the operation efficiency low, reducing the efficiency of ice image segmentation of power transmission lines. Summary of the invention
[0004] The embodiments of the present invention provide a method, device, electronic device and storage medium for ice-covered image segmentation, which can quickly and accurately obtain a model that can segment ice-covered images of transmission lines, improve the efficiency of model training, further improve the efficiency of ice-covered image segmentation of transmission lines, and facilitate subsequent discrimination operations according to the ice coverage situation.
[0005] In a first aspect, an embodiment of the present invention provides an ice-covered image segmentation method, comprising:
[0006] When the pre-established initial segmentation model does not meet the preset conditions, obtaining an ice-covered image set of the transmission line; the initial segmentation model includes a feature extraction module and an upsampling module;
[0007] Performing feature extraction on the ice-covered image in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature;
[0008] Obtaining a target feature of the ice-covered image according to the first feature and the upsampling module;
[0009] The initial segmentation model is optimized based on the target features and a preset loss function until the initial segmentation model meets the preset conditions, thereby obtaining a target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line.
[0010] In a second aspect, an embodiment of the present invention provides an ice-covered image segmentation device, the device comprising:
[0011] A data acquisition module, used to acquire an ice-covered image set of the transmission line when the pre-established initial segmentation model does not meet the preset conditions; the initial segmentation model includes a feature extraction module and an upsampling module;
[0012] A first determination module, configured to extract features of the ice-covered images in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature;
[0013] A second determination module, configured to obtain a target feature of the ice-covered image according to the first feature and the upsampling module;
[0014] A model optimization module is used to optimize the initial segmentation model based on the target features and a preset loss function until the initial segmentation model meets the preset conditions, thereby obtaining a target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line.
[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an ice-covered image segmentation method as described in any one of the embodiments of the present invention is implemented.
[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an ice-covered image segmentation method as described in any one of the embodiments of the present invention.
[0017] In the embodiment of the present invention, a dilated convolution is used instead of a traditional convolution to expand the receptive field without reducing the resolution. The dilated convolution unit is combined with an attention unit to enable the target segmentation model to integrate ice feature information of multiple scales, improve the segmentation quality of the ice-covered image of the power transmission line, and suppress interference factors. The upsampling module saves the amount of parameter calculation of the target segmentation model, improves the efficiency of model training, and enables the target segmentation model to effectively segment the ice-covered image of the power transmission line. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0019] Figure 1 A first flow chart of an ice-covered image segmentation method provided by an embodiment of the present invention;
[0020] Figure 2 A schematic diagram of a feature extraction module provided by an embodiment of the present invention;
[0021] Figure 3 A model structure diagram of an initial segmentation model provided by an embodiment of the present invention;
[0022] Figure 4 A second flow chart of an ice-covered image segmentation method provided by an embodiment of the present invention;
[0023] Figure 5 A schematic diagram of a dilated convolution unit processing input features provided by an embodiment of the present invention;
[0024] Figure 6 A schematic diagram of a pooling unit processing input features provided by an embodiment of the present invention;
[0025] Figure 7 A schematic diagram of an attention unit processing input features provided by an embodiment of the present invention;
[0026] Figure 8 A third flow chart of an ice-covered image segmentation method provided by an embodiment of the present invention;
[0027] Fig. 9 A schematic diagram of the structure of an ice-covered image segmentation device provided by an embodiment of the present invention;
[0028] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0030] Figure 1This is a first flow chart of an ice-covered image segmentation method provided in an embodiment of the present invention. The method in the embodiment of the present invention can quickly and accurately obtain a model that can segment ice-covered images of power transmission lines, thereby improving the efficiency of model training, further improving the efficiency of ice-covered image segmentation of power transmission lines, and facilitating subsequent discrimination operations according to the ice coverage situation. The method can be executed by an ice-covered image segmentation device provided in an embodiment of the present invention, and the device can be implemented in software and / or hardware. The following embodiments will be described by taking the device integrated in an electronic device as an example, and the electronic device can be a computer device or a server, etc. Reference Figure 1 , the method may specifically include the following steps:
[0031] Step 101: When the pre-established initial segmentation model does not meet the preset conditions, an ice-covered image set of the transmission line is obtained.
[0032] The initial segmentation model includes a feature extraction module and an upsampling module. The initial segmentation model is a pre-established model that has not been trained. The feature extraction module is used to extract and refine features of ice-covered images of transmission lines. The upsampling module is used to upsample feature vectors. The preset condition may be that a preset loss function value reaches convergence or the number of iterations reaches a preset number.
[0033] Specifically, when the initial segmentation model does not meet the preset conditions, it means that the initial segmentation model cannot accurately segment the ice-covered image, and the initial segmentation model needs to be trained and optimized according to the ice-covered image set of the transmission line. The server can obtain the ice-covered image set of the transmission line in the historical period from the power system through a pre-set data interface, or directly receive the ice-covered image set of the transmission line uploaded by the user.
[0034] Step 102: extract features of the ice-covered images in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature.
[0035] The feature extraction module includes a dilated convolution unit, an attention unit, and a pooling unit. The dilated convolution unit consists of a dilated convolution layer. The dilated convolution expands its receptive field by adding spaces (zeros) to the convolution kernel. Figure 2 A schematic diagram of a feature extraction module provided by an embodiment of the present invention, such as Figure 2As shown, the feature extraction module includes a first dilated convolution unit, a second dilated convolution unit, a pooling unit, and an attention unit. The dilated ratio of the first dilated convolution unit is 2, and the dilated ratio of the second dilated convolution unit is 3. The pooling unit is used to increase the network depth and width of the initial segmentation model, thereby obtaining ice coverage features of multiple perspectives and scales. The attention unit is used to implement the attention mechanism, so that the initial segmentation model increases the weights of important feature maps or feature channels, reduces the weights of unimportant features, and improves the performance of the initial segmentation model.
[0036] In an optional embodiment, after obtaining the ice-covered image set, the ice-covered images in the ice-covered image set are input into the initial segmentation model. After the initial segmentation model receives the ice-covered images, the ice-covered images can be convolved and batch normalized to reduce internal covariate shift. The batch-normalized ice-covered images are activated according to a pre-set activation function (such as a ReLU activation function) to obtain a first output feature. The first output feature is input into a feature extraction module, and the hole convolution unit of the feature extraction module performs a hole convolution with a hole rate of 2 and a hole convolution with a hole rate of 3 on the first output feature in sequence to obtain a second output feature. The second output feature is input into a pooling unit, and a third output feature of multiple perspectives and multiple scales is obtained through the pooling unit. The third output feature is input into an attention unit, and the weight of the third output feature is adjusted through the attention unit to obtain the first feature.
[0037] Step 103: Obtain target features of ice-covered images for ice-covered image segmentation according to the first feature and the ice-covered image segmentation up-sampling module.
[0038] Among them, the upsampling module is used to gradually restore the resolution of the ice feature map to the size of the original input transmission line ice image. The target feature is the feature obtained by the initial segmentation model after image segmentation of the ice image. Specifically, according to the position of each pixel in the ice image, the first feature is dynamically weighted using transposed convolution and bilinear interpolation through the upsampling module. During the upsampling process, the closer the pixel position is to the edge of the ice image, the larger the weight of the bilinear interpolation, and the closer the pixel position is to the center of the ice image, the larger the weight of the transposed convolution.
[0039] In this solution, the initial segmentation model also includes a feature extraction submodule, which is composed of a hole convolution unit and an attention unit in series. The hole ratio of the hole convolution in the feature extraction submodule can be 2. In an optional embodiment, the initial segmentation model can be composed of multiple convolution layers, multiple feature extraction modules, multiple upsampling modules, multiple hole convolution units, multiple attention units and multiple convolution units. Figure 3 The model structure diagram of the initial segmentation model provided by the embodiment of the present invention. Figure 3 As shown, Figure 3 The "+" in the figure indicates channel dimension concatenation. The ice-covered image is input into the initial segmentation model. The convolution unit 1 of the initial segmentation model uses a convolution kernel of size 3×3 and a step size of 1 to perform a convolution operation on the input ice-covered image. The convolutional ice-covered image is batch normalized and then processed by nonlinear activation to obtain F1. The convolution unit 2 uses a convolution kernel of size 7×7 to convolve F1, and the convolutional F1 is input into the feature extraction module 1 to obtain F2. The convolution unit 3 uses a convolution kernel of size 3×3 to convolve the input ice-covered image, and the convolutional F2 is input into the feature extraction module 2 to obtain F3. The convolution unit 4 uses a convolution kernel of size 3×3 to convolve the input features, and the convolutional F3 is input into the hole convolution unit to obtain F4. F4 is input to the upsampling module, the features output by the upsampling module are spliced with F3 in the channel dimension, and the spliced vector is input to the feature extraction submodule 1 to obtain F5. F5 is input to the upsampling module 2 for upsampling, F2 and the upsampled F5 are spliced in the channel dimension, and the spliced vector is input to the feature extraction submodule 2 to obtain F6. F6 is input to the upsampling module 3 for upsampling, the upsampled F6 and F1 are spliced in the channel dimension, the spliced features are input to the attention unit for weight adjustment, and the weight-adjusted features are input to the convolution unit 5. The convolution unit 5 uses a convolution kernel of size 3×3 to perform a convolution operation on the input features to obtain F7. F7, F5, F6 and F4 are spliced in the channel dimension, and the spliced features are input to the convolution unit 6. The convolution unit 6 uses a convolution kernel of size 3×3 to perform a convolution operation on the input features to obtain the target features (the image segmentation result corresponding to the ice-covered image).
[0040] Step 104: Optimize the initial segmentation model based on the target features and a preset loss function until the initial segmentation model meets preset conditions to obtain a target segmentation model.
[0041] Among them, the target segmentation model is used to perform image segmentation on the ice-covered image of the transmission line. The preset condition may be that the value of the loss function reaches convergence. When the initial segmentation model does not meet the preset condition, an ice-covered image is selected from the ice-covered image set as the current image, and the current image is input into the initial segmentation model. After receiving the current image, the initial segmentation model processes the current image and outputs the current segmentation result corresponding to the current image (the target feature corresponding to the current ice-covered image). The target feature corresponding to the current ice-covered image and the predetermined label feature corresponding to the current ice-covered image (the standard segmentation result corresponding to the current ice-covered image) are input into the loss function to obtain the loss function value. After obtaining the loss function value, it is determined whether the loss function value meets the preset condition. If the preset condition is met, the initial segmentation model is determined to be the target segmentation model. If the loss function value does not meet the preset condition, the model parameters of the initial segmentation model are adjusted according to the loss function value, and the next ice-covered image is selected from the ice-covered image set as the current image, and the step of inputting the current image into the initial segmentation model is repeated, and the model parameters of the initial segmentation model are continuously adjusted until the initial segmentation model meets the preset condition to obtain the target segmentation model.
[0042] The technical solution of this embodiment is to obtain an ice-covered image set of the transmission line when the pre-established initial segmentation model does not meet the preset conditions; the initial segmentation model includes a feature extraction module and an upsampling module; based on the hole convolution unit and the attention unit of the feature extraction module, feature extraction is performed on the ice-covered image in the ice-covered image set to obtain a first feature; the target feature of the ice-covered image is obtained according to the first feature and the upsampling module; the initial segmentation model is optimized based on the target feature and a pre-set loss function until the initial segmentation model meets the preset conditions to obtain a target segmentation model, which is used to perform image segmentation on the ice-covered image of the transmission line. The technical solution of this embodiment uses hole convolution instead of traditional convolution to expand the receptive field without reducing the resolution. On the basis of the hole convolution unit, the attention unit is combined so that the target segmentation model can fuse ice feature information of multiple scales, improve the segmentation quality of the ice-covered image of the transmission line, and suppress interference factors. The upsampling module can save the parameter calculation amount of the target segmentation model, so that the target segmentation model can effectively segment the ice-covered image of the transmission line.
[0043] Figure 4 This is a second flow chart of an ice-covered image segmentation method provided by an embodiment of the present invention. This embodiment is a refinement based on the above embodiment. The specific method can be as follows Figure 4 As shown, the method may include the following steps:
[0044] Step 401: When the pre-established initial segmentation model does not meet the preset conditions, a set of ice-covered images of the transmission line is obtained.
[0045] The initial segmentation model includes a feature extraction module and an upsampling module.
[0046] Step 402: for each ice-covered image in the ice-covered image set, input the current ice-covered image into the initial segmentation model to obtain a first output feature of the current ice-covered image output by the convolution unit of the initial segmentation model.
[0047] Among them, the convolution unit includes a convolution layer, a normalization layer and an activation layer, and the convolution kernel of the convolution layer can be 3×3. In an optional embodiment, after obtaining the current ice-covered image set, the current ice-covered image is input into the initial segmentation model. After the initial segmentation model receives the current ice-covered image, it uses a convolution kernel of size 3×3 and sets the step size to 1 to perform a convolution operation on the input current ice-covered image through the convolution unit. Through the normalization layer, the current ice-covered image after convolution is batch normalized. Through the activation layer, the normalized current ice-covered image is nonlinearly activated using a nonlinear activation function to obtain the first output feature of the current ice-covered image.
[0048] Step 403: Perform a dilated convolution process on the first output feature through a dilated convolution unit to obtain a second output feature.
[0049] Among them, the dilated convolution unit includes a first convolution layer and a second convolution layer. The dilated convolution expands its receptive field by adding spaces (zeros) to the convolution kernel. In this scheme, optionally, the first output feature is subjected to dilated convolution processing by the dilated convolution unit to obtain the second output feature, including: performing dimensionality reduction convolution and dilated convolution on the first output feature by the first convolution layer of the dilated convolution unit to obtain the first dilated feature; performing dimensionality reduction convolution on the first output feature by the second convolution layer of the dilated convolution unit to obtain the second dilated feature; performing element-wise addition of the first dilated feature and the second dilated feature to obtain a candidate second output feature, and obtaining the second output feature based on the candidate second output feature.
[0050] Figure 5 Schematic diagram of a dilated convolution unit processing input features provided by an embodiment of the present invention. Figure 5 As shown, Figure 5Add in it means element-by-element addition. After obtaining the input feature (the first output feature), the dilated convolution unit uses a convolution kernel of size 1×1 to convolve the input feature through the convolution layer in the first convolution layer. The convolved input features are batch normalized through the normalization layer. The normalized input features are nonlinearly activated through the activation layer using a nonlinear activation function. The activated input features are subjected to dilated convolution with a dilation rate of 2 or 3 through the dilated convolution layer to obtain the input features after dilated convolution. The input features after dilated convolution are batch normalized and nonlinearly activated. The input features after nonlinear activation are convolved through the convolution layer using a convolution kernel of size 1×1 to obtain the first dilated feature. At the same time, the input features are convolved through the convolution layer of the second convolution layer of the dilated convolution unit using a convolution kernel of size 1×1, and the convolved input features are batch normalized to obtain the second dilated feature. After obtaining the first hole feature and the second hole feature, the first hole feature and the second hole feature are added element by element to obtain a candidate output feature. After obtaining the candidate output feature, a nonlinear activation function is used to nonlinearly activate the candidate output feature to obtain an output feature (second output feature).
[0051] For example, BN represents batch normalization, ReLU represents nonlinear activation, Input represents the input feature of the dilated convolution unit, Output represents the output feature of the dilated convolution unit, do1 represents the first dilated feature, do2 represents the second dilated feature, Add represents element-by-element addition, and conv d represents dilated convolution, conv represents convolution, n represents the dilation rate, and n is equal to 2 or 3. Furthermore, the calculation process of the output feature of the dilated convolution unit can be expressed by the following formula:
[0052]
[0053] In the above steps, dilated convolution is used instead of traditional convolution to expand the receptive field without reducing the resolution, so that the network can integrate ice feature information at multiple scales.
[0054] Step 404: extract the second output feature through a pooling unit to obtain a third output feature.
[0055] Among them, the pooling unit is used to increase the network depth and width of the initial segmentation model, so as to obtain ice cover features of multiple perspectives and multiple scales. The pooling unit includes a first convolutional layer, a second convolutional layer, a third convolutional layer and a maximum pooling layer. In this scheme, optionally, the second output feature is extracted by the pooling unit to obtain a third output feature, including: extracting the second output feature by the first convolutional layer, the second convolutional layer and the third convolutional layer respectively to obtain the first convolutional feature, the second convolutional feature and the third convolutional feature; performing feature fusion on the second output feature by the maximum pooling layer to obtain the fourth pooling feature; performing feature splicing on the first convolutional feature, the second convolutional feature, the third convolutional feature and the fourth pooling feature to obtain the third output feature.
[0056] Figure 6 Schematic diagram of the pooling unit processing input features provided by the embodiment of the present invention. Figure 6 As shown, Figure 6 The "+" in represents channel dimension concatenation. After receiving the input feature (the second output feature), the pooling unit uses a convolution kernel of size 1×1 through the first convolution layer to convolve the input feature to obtain the first convolution feature. The second convolution layer uses a convolution kernel of size 1×1 and a convolution kernel of size 3×3 to convolve the input feature to obtain the second convolution feature. The third convolution layer uses a convolution kernel of size 1×1 and two convolution kernels of size 3×3 to convolve the input feature to obtain the third convolution feature. The input feature is subjected to maximum pooling with a pooling window size of 3×3 through the maximum pooling layer, and the input feature after maximum pooling is convolved with a convolution kernel of size 1×1 to obtain the fourth pooling feature. Further, the first convolution feature, the second convolution feature, the third convolution feature and the fourth pooling feature are concatenated in the channel dimension to obtain the output feature (the third output feature).
[0057] Exemplarily, Input represents the input feature of the pooling unit, and po1 represents the first convolution feature. The number of Input channels is reduced through the first convolution layer to reduce the computational complexity, and the output po1 of the first convolution layer is obtained. Po2 represents the second convolution feature. In the second convolution layer, Input first reduces the number of channels through a 1×1 convolution kernel, and then extracts ice features through a 3×3 convolution kernel to obtain the output po2 of the second convolution layer. Po3 represents the third convolution feature. In the third convolution layer, Input first reduces the number of channels through a 1×1 convolution kernel, and then uses two consecutive 3×3 convolution kernels to extract deeper ice features to obtain the output po3 of the third convolution layer. In this solution, the convolution layer corresponding to the two 3×3 convolution kernels has the same receptive field as the convolution layer corresponding to the 5×5 convolution kernel, but has fewer computational parameters, which can reduce the number of network parameters. Let po4 represent the fourth pooling feature. In the maximum pooling layer, the input is first pooled (pooling window is 3×3), and then the ice cover feature is fused through a 1×1 convolution kernel to obtain po4 output by the maximum pooling layer. After splicing po1, po2, po3 and po4 in the channel dimension, the Output is obtained. Therefore, the calculation formula of the Output is as follows:
[0058]
[0059] conv means convolution, Concat means channel dimension concatenation, and maxpooling means maximum pooling.
[0060] In the above steps, the pooling unit can improve the performance of the target segmentation model, reduce the number of parameters, increase the depth and width of the network, and obtain ice cover features from multiple perspectives and scales.
[0061] Step 405: adjust the weight of the third output feature according to the attention unit to obtain the first feature.
[0062] Among them, the attention unit is used to implement the attention mechanism, so that the initial segmentation model increases the weights of important feature maps or feature channels and reduces the weights of unimportant features, thereby improving the performance of ice-covered image segmentation. The attention unit includes a convolution layer and a global pooling layer. In this scheme, optionally, the weight of the third output feature is adjusted according to the attention unit to obtain the first feature, including: performing dimensionality reduction convolution on the third output feature through the convolution layer of the attention unit, and spatially weighting the third output feature and the third output feature after the dimensionality reduction convolution to obtain the first attention feature; performing global pooling processing on the third output unit through the global pooling layer of the attention unit, and channel weighting the third output feature and the third output feature after the global pooling processing to obtain the second attention feature; element-wise addition of the first attention feature and the second attention feature to obtain the first feature.
[0063] Figure 7 Schematic diagram of an attention unit processing input features provided by an embodiment of the present invention. Figure 7 As shown, Figure 7 The "X" in represents weighting. After receiving the input feature (the third output feature), the attention unit uses a convolution layer to convolve the input feature using a convolution kernel of size 1×1, and uses an activation function to activate the convolved input feature to obtain the activated input feature (the third output feature after dimensionality reduction convolution). The input feature and the activated input feature are spatially weighted to obtain the first attention feature. At the same time, the input feature is globally averaged pooled through the global pooling layer, and the input feature after global average pooling is convolved using two convolution kernels of size 1×1, and the convolved input feature is activated using an activation function to obtain the activated input feature (the third output feature after global pooling). Furthermore, the input feature and the activated input feature are channel-weighted to obtain the second attention feature. The first attention feature and the second attention feature are element-wise added to obtain the output feature (the first feature).
[0064] Exemplarily, Input represents the input feature of the attention unit, and conv represents convolution using a 1x1 convolution kernel. The dimension of Input is C×H×W. First, conv(1x1) is performed on Input to obtain an attention feature S1 with a dimension of 1×H×W. Then S1 is activated by a Sigmoid function to obtain an activated attention feature S2 with a dimension of 1×H×W. Finally, the spatial weighting of S2 and Input is calculated by Spatial-wise (spatial weighted algorithm) to obtain the first attention feature S. On the other hand, global average pooling is performed on the Input with a dimension of C×H×W to obtain an attention feature C1 with a dimension of C×1×1. Then conv(1x1) is performed on C1 to obtain an attention feature S2 with a dimension of The feature C2 is taken, conv(1x1) is performed on C2, and the Sigmoid function is used to activate it, and the channel weight vector C3 with a dimension of C×1×1 is obtained. C3 and Input are channel-weighted by Channel-wise (channel weighting algorithm) to obtain the second attention feature C. Output is obtained by adding S and C. Gap is the global average pooling. Furthermore, the calculation process of the Output of the attention unit can be described by the following formula:
[0065]
[0066] In an optional implementation, for S2 of dimension 1×H×W and Input of dimension C×H×W, the 1×H×W spatial weight map S2 is first broadcasted to each channel through Spatial-wise to generate an S3 of dimension C×H×W. Each channel uses the same H×W attention map, that is, H×W is copied C times along the channel. Then S3 is element-wise multiplied with Input to obtain S:
[0067] S c,i,j =s c,i,j ×I c,i,j
[0068] Among them, S c,i,j is the value of the i-th row and j-th column of the c-th channel of S, s c,i,j is the value of the i-th row and j-th column on the c-th channel of S3, I c,i,j is the value of the i-th row and j-th column on the c-th channel of Input. For C3 of dimension C×1×1 and Input of dimension C×H×W, Channel-wise regards C3 of C×1×1 as the weight of each channel c, and then treats the 1×1 weight W of each channel c as c Broadcast to every position of H×W and get C:
[0069] C c,i,j =W c ×I c,i,j
[0070] Among them, C c,i,j is the value of the i-th row and j-th column on the c-th channel of C, I c,i,j is the value of the i-th row and j-th column of the c-th channel of Input. The attention unit enables the network to increase the weights of important feature maps or feature channels and reduce the weights of unimportant features, thereby improving the performance of the target segmentation model.
[0071] Step 406: for each first feature of the ice-covered image, determine the input feature of the upsampling module based on the first feature of the current ice-covered image, the feature extraction module and the dilated convolution unit of the feature extraction module.
[0072] Specifically, after obtaining the first feature, the first feature is input into the feature extraction module, and then the first feature output by the feature extraction module is convolved using a convolution kernel of size 1×1. The first feature after convolution is input into the feature extraction module again, and then the first feature output by the feature extraction module is convolved using a convolution kernel of size 1×1, and the first feature after convolution is input into the hole convolution unit to obtain the input feature of the upsampling module output by the hole convolution unit.
[0073] like Figure 3 As shown, Figure 3F2 in the figure is the first feature. After the Conv(3x3) operation with a step size of 2, F2 is input into the feature extraction module to extract and refine the ice feature of F2 to obtain F3. The resolution of F3 is half that of F2. After the Conv(3x3) operation with a step size of 2, F3 is input into the dilated convolution unit with a dilated rate of 2 to extract and refine the ice feature of F3 to obtain the input feature F4 of the upsampling module. The resolution of F4 is half that of F3.
[0074] Step 407: Through the upsampling module, based on the pixel position of the current ice-covered image, the input features of the upsampling module are dynamically weighted by transposed convolution and bilinear interpolation to obtain a second feature.
[0075] The pixel position of the current ice-covered image is the pixel position during the upsampling process of the current ice-covered image. Bilinear interpolation uses the four real pixel values around the target point in the original image to jointly determine a pixel value in the target image. The core idea is to perform linear interpolation in two directions. Transposed convolution is to fill the low-resolution feature map by calculating the transposed matrix, thereby achieving the amplification of the feature map resolution.
[0076] In an optional implementation, after obtaining the input features of the upsampling module, if the pixel position of the current ice-covered image is closer to the edge of the image, the weight of the bilinear interpolation is made larger. If the pixel position of the current ice-covered image is closer to the center of the image, the weight of the transposed convolution is made larger, and the detail loss and computational overhead are reduced by balancing the two. The specific weight adjustment equation is shown as follows:
[0077]
[0078] Where W and H represent the height and width of the input features of the upsampling module, respectively, and UPres(i,j) represents the pixel value corresponding to the coordinate (i,j) after upsampling. tc(i,j) represents the pixel value corresponding to the coordinate (i,j) after transposed convolution. The calculation formula of tc(i,j) is as follows:
[0079]
[0080] Among them, T is the deconvolution kernel, the size of T is K×K, S is the step size, Indicates rounding down. bl(i,j) represents the pixel value corresponding to the coordinate (i,j) after bilinear interpolation. The calculation formula of bl(i,j) is as follows:
[0081]
[0082] in, S h , Sw is the scale factor, H in , W in are the height and width of the input feature map, H out and W out are the height and width of the output feature map, respectively. Indicates rounding down. Indicates rounding up. The upsampling module can dynamically and adaptively adjust the weights of the two upsampling methods according to the position of the ice feature map, which can not only alleviate the problem of blurred edge details caused by the traditional deconvolution upsampling method, but also save parameter calculations to the maximum extent and improve the upsampling efficiency of the target segmentation model.
[0083] Step 408: determine the target feature based on the second feature, the first feature, the feature extraction module, the upsampling module, and the convolution unit of the initial segmentation model.
[0084] Specifically, after obtaining the second feature output by the upsampling module, the second feature and the first feature are further processed through the feature extraction submodule, the upsampling module, the convolution unit, the attention unit and the feature extraction submodule to obtain the target feature.
[0085] For example, Figure 3 As shown in the figure, F4 is the input feature of the upsampling module. After being upsampled by the upsampling module, F4 is spliced with F3 in the channel dimension. The features after the splicing in the channel dimension are input into the feature extraction submodule to obtain F5. The resolution of F5 is twice that of F4. After being upsampled by the upsampling module, F5 is spliced with F2 in the channel dimension. The features after the splicing in the channel dimension are input into the feature extraction submodule to obtain F6. The resolution of F6 is twice that of F5. After being upsampled by the upsampling module, F6 is spliced with F1 in the channel dimension. The features after the splicing in the channel dimension are input into the attention unit. The result of the attention unit processing is subjected to the convolution operation of Conv(3x3) to obtain F7. The resolution of F7 is twice that of F6. F7, F5, F6 and F4 are spliced in the channel dimension. After the spliced features are subjected to the convolution operation of Conv(3x3), the target features are obtained. The target features are the image segmentation results corresponding to the current ice-covered image.
[0086] Step 409: Optimize the initial segmentation model based on the target features and a preset loss function until the initial segmentation model meets preset conditions to obtain a target segmentation model.
[0087] The target segmentation model is used to perform image segmentation on ice-covered transmission line images.
[0088] In the technical solution of this embodiment, when the pre-established initial segmentation model does not meet the preset conditions, an ice-covered image set of the transmission line is obtained; the initial segmentation model includes a feature extraction module and an upsampling module; for each ice-covered image in the ice-covered image set, the current ice-covered image is input into the initial segmentation model to obtain the first output feature of the current ice-covered image output by the convolution unit of the initial segmentation model. The first output feature is subjected to a hole convolution process by the hole convolution unit to obtain a second output feature. The second output feature is subjected to feature extraction by the pooling unit to obtain a third output feature. The third output feature is weighted according to the attention unit to obtain a first feature. For the first feature of each ice-covered image, the input feature of the upsampling module is determined based on the first feature of the current ice-covered image, the feature extraction module and the hole convolution unit of the feature extraction module. Through the upsampling module, based on the pixel position of the current ice-covered image, the input feature of the upsampling module is dynamically weighted by transposed convolution and bilinear interpolation to obtain a second feature. The target feature is determined based on the second feature, the first feature, the feature extraction module, the upsampling module and the convolution unit of the initial segmentation model. The initial segmentation model is optimized based on the target features and a pre-set loss function until the initial segmentation model meets the preset conditions to obtain the target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line. In the technical solution of this embodiment, the receptive field can be expanded without reducing the resolution through the hole convolution unit, so that the network can fuse ice feature information of multiple scales. The amount of parameters is reduced by the pooling unit, while the depth and width of the network are increased to obtain ice features of multiple perspectives and scales. The attention mechanism is implemented by the attention unit, so that the network can increase the weights of important feature maps or feature channels and reduce the weights of unimportant features, thereby improving the performance of ice-covered image segmentation. The upsampling module can not only alleviate the problem of blurred edge details caused by the upsampling method of traditional deconvolution, but also save parameter calculation to the maximum extent, and improve the upsampling efficiency of the target segmentation model.
[0089] Figure 8 The third flow chart of an ice-covered image segmentation method provided by an embodiment of the present invention, the specific method can be as follows Figure 8 As shown, the method may include the following steps:
[0090] Step 801: Acquire an ice-covered image of a power transmission line to be processed.
[0091] The unprocessed ice-covered image is an image that needs to be segmented. In practical applications, when it is necessary to analyze the ice-covered condition of a power transmission line, a user can upload the unprocessed ice-covered image of the power transmission line to a server, so that the server obtains the unprocessed ice-covered image of the power transmission line.
[0092] Step 802: input the ice-covered image to be processed into the target segmentation model, perform image segmentation on the ice-covered image to be processed through the target segmentation model, and obtain a segmentation result map of the ice-covered image to be processed.
[0093] The target segmentation model is used to perform image segmentation on the ice-covered image of the power transmission line. After obtaining the ice-covered image to be processed, the ice-covered image to be processed is input into the target segmentation model. The target segmentation model can perform image segmentation on the ice-covered image to be processed and obtain a segmentation result map of the ice-covered image to be processed.
[0094] The model structure of the target segmentation model is as follows Figure 3 As shown, for example, it is assumed that the dimension of the ice-covered image to be processed is 1×1024×1024 (channel×width×height). The ice-covered image to be processed is input into the target segmentation model, and is first processed by the convolution unit (Conv(3x3)_BN_ReLU) to obtain F1 with a dimension of 16×1024×1024. After F1 is downsampled by Conv(7x7) with a step size of 2, it is input into the feature extraction module to obtain F2 with a dimension of 32×512×512. After F2 is downsampled by Conv(3x3) with a step size of 2, it is input into the feature extraction module to extract and refine the ice-covered features of F2 to obtain F3 with a dimension of 48×256×256. After downsampling by Conv(3x3) with a step size of 2, F3 is input into the dilated convolution unit with a dilation rate of 2 to extract and refine the ice cover features of F3, and obtain F4 with a dimension of 64×128×128. After upsampling by the upsampling module, F4 is concatenated with F3 in the channel dimension, and the feature map after the channel dimension concatenation is input into the feature extraction submodule to obtain a dimension of 56×256×256. After upsampling by the upsampling module, F5 is concatenated with F2 in the channel dimension, and the features after the channel dimension concatenation are input into the feature extraction submodule to obtain F6 with a dimension of 44×512×512. After upsampling by the upsampling module, F6 is concatenated with F1 in the channel dimension, and the features after the channel dimension concatenation are input into the attention unit. The result of the attention unit processing is convolved by Conv(3x3) with a step size of 1 to obtain F7 with a dimension of 44×1024×1024. The obtained F4, F5, F6 and F7 of different scales are restored to the resolution of 1024×1024 through upsampling operation, and then spliced in the channel dimension. The features with the dimension of (44+44+56+64)×1024×1024 after splicing are converted into a single channel through Conv(3x3) and convolution operation with step size = 1, and then the Sigmoid activation function is used to obtain the final transmission line ice coverage image segmentation result image with the dimension of 1×1024×1024.
[0095] Step 803: Display the segmentation result map to the user, so that the user can determine the icing condition of the transmission line based on the segmentation result map.
[0096] Specifically, after obtaining the segmentation result map, the server can display the segmentation result map to the user through a display screen, or send the segmentation result map to the user's mobile phone, tablet or other terminal device, so that the user can view the segmentation result map through the terminal device and determine the ice coverage of the transmission line based on the segmentation result map.
[0097] In the technical solution of this embodiment, an image of ice-covered power lines to be processed is obtained; the image of ice-covered power lines to be processed is input into a target segmentation model, and the image of ice-covered power lines to be processed is segmented by the target segmentation model to obtain a segmentation result map of the image of ice-covered power lines to be processed; the segmentation result map is displayed to the user so that the user can determine the ice-covered power lines based on the segmentation result map. The technical solution of this embodiment can use the target segmentation model to provide the user with an accurate segmentation result map, so that the user can effectively understand the ice-covered power lines according to the segmentation result map, thereby improving the user experience.
[0098] Fig. 9 The structure diagram of an ice-covered image segmentation device provided in an embodiment of the present invention is suitable for executing an ice-covered image segmentation method provided in an embodiment of the present invention. Fig. 9 As shown, the device may specifically include:
[0099] The data acquisition module 901 is used to acquire an ice-covered image set of the transmission line when the pre-established initial segmentation model does not meet the preset conditions; the initial segmentation model includes a feature extraction module and an upsampling module;
[0100] A first determination module 902 is used to extract features of the ice-covered images in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature;
[0101] A second determination module 903, configured to obtain a target feature of the ice-covered image according to the first feature and the upsampling module;
[0102] The model optimization module 904 is used to optimize the initial segmentation model based on the target features and a preset loss function until the initial segmentation model meets the preset conditions, thereby obtaining a target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line.
[0103] Optionally, the first determination module 902 is specifically configured to: for each ice-covered image in the ice-covered image set, input the current ice-covered image into the initial segmentation model to obtain a first output feature of the current ice-covered image output by a convolution unit of the initial segmentation model;
[0104] The first output feature is processed by the dilated convolution unit to obtain a second output feature; the second output feature is extracted by the pooling unit to obtain a third output feature; and the third output feature is weighted according to the attention unit to obtain the first feature.
[0105] Optionally, the first determining module 902 is further configured to: perform dimension reduction convolution and dilated convolution on the first output feature through the first convolution layer of the dilated convolution unit to obtain a first dilated feature;
[0106] Performing dimension reduction convolution on the first output feature through the second convolution layer of the hole convolution unit to obtain a second hole feature;
[0107] The first hole feature and the second hole feature are element-wise added to obtain a candidate second output feature, and the second output feature is obtained based on the candidate second output feature.
[0108] Optionally, the first determining module 902 is further configured to: perform feature extraction on the second output feature through the first convolution layer, the second convolution layer, and the third convolution layer, respectively, to obtain a first convolution feature, a second convolution feature, and a third convolution feature;
[0109] The second output feature is subjected to feature fusion through the maximum pooling layer to obtain a fourth pooling feature; the first convolution feature, the second convolution feature, the third convolution feature and the fourth pooling feature are subjected to feature splicing to obtain the third output feature.
[0110] Optionally, the first determination module 902 is further used to: perform dimension reduction convolution on the third output feature through the convolution layer of the attention unit, and perform spatial weighting on the third output feature and the third output feature after the dimension reduction convolution to obtain a first attention feature;
[0111] Performing global pooling processing on the third output unit through the global pooling layer of the attention unit, and performing channel weighting on the third output feature and the third output feature after the global pooling processing to obtain a second attention feature;
[0112] The first attention feature and the second attention feature are element-wise added to obtain the first feature.
[0113] Optionally, the second determination module 902 is specifically configured to: for each first feature of the ice-covered image, determine the input feature of the upsampling module based on the first feature of the current ice-covered image, the feature extraction module and the hole convolution unit of the feature extraction module;
[0114] By means of the upsampling module, based on the pixel position of the current ice-covered image, the input features of the upsampling module are dynamically weighted by transposed convolution and bilinear interpolation to obtain a second feature;
[0115] The target feature is determined based on the second feature, the first feature, the feature extraction module, the upsampling module, and a convolution unit of the initial segmentation model.
[0116] Optionally, the model optimization module 904 is specifically used to: obtain an ice-covered image to be processed of the transmission line;
[0117] Inputting the ice-covered image to be processed into the target segmentation model, performing image segmentation on the ice-covered image to be processed by the target segmentation model, and obtaining a segmentation result map of the ice-covered image to be processed;
[0118] The segmentation result map is displayed to a user, so that the user can determine the ice coverage condition of the transmission line based on the segmentation result map.
[0119] The ice-covered image segmentation device provided in the embodiment of the present invention can execute the ice-covered image segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. The contents not described in detail in this embodiment can refer to the description in any method embodiment of the present invention.
[0120] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, referring to Fig.10 , Fig.10 The electronic device 12 shown is only an example and should not limit the functions and scope of use of the embodiments of the present application. Fig.10As shown, the electronic device 12 is in the form of a general computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16). The bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any bus structure in a variety of bus structures. For example, these architectures include, but are not limited to, an industrial standard architecture (ISA) bus, a microchannel architecture (MAC) bus, an enhanced ISA bus, a video electronics standard association (VESA) local bus, and a peripheral component interconnect (PCI) bus. The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media. The system memory 28 may include computer system readable media in the form of volatile memory, such as a random access memory (RAM) 30 and / or a cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.10 not shown, usually called a "hard drive"). Although Fig.10Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) can be provided. In these cases, each driver can be connected to the bus 18 through one or more data medium interfaces. The memory 28 may include at least one program product, which has a group (such as at least one) of program modules, which are configured to perform the functions of each embodiment of the present application. A program / utility 40 with a group (at least one) of program modules 42 can be stored in, for example, the memory 28, such program modules 42 include but are not limited to an operating system, one or more application programs, other program modules and program data, and each of these examples or some combination may include the realization of a network environment. The program module 42 generally performs the functions and / or methods in the embodiments described in the present application. The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that although Fig.10 Not shown, other hardware and / or software modules may be used in combination with the electronic device 12, including but not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and data backup storage system, etc. The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, to implement an ice-covered image segmentation method provided in an embodiment of the present invention: when the pre-established initial segmentation model does not meet the preset conditions, an ice-covered image set of the transmission line is obtained; the initial segmentation model includes a feature extraction module and an upsampling module; based on the hole convolution unit and the attention unit of the feature extraction module, the ice-covered image in the ice-covered image set is subjected to feature extraction to obtain a first feature; according to the first feature and the upsampling module, the target feature of the ice-covered image is obtained; based on the target feature and the pre-set loss function, the initial segmentation model is optimized until the initial segmentation model meets the preset conditions, and a target segmentation model is obtained, and the target segmentation model is used to perform image segmentation on the ice-covered image of the transmission line.
[0121] The embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, an ice-covered image segmentation method provided by all the embodiments of the present invention is implemented. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic device, device or device of electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction-executing electronic device, device or device. A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program used by or in combination with an instruction-executing electronic device, device, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wires, optical cables, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0122] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for segmenting an ice-covered image, characterized in that: The method comprises: When the pre-established initial segmentation model does not meet the preset conditions, obtaining an ice-covered image set of the transmission line; the initial segmentation model includes a feature extraction module and an upsampling module; Performing feature extraction on the ice-covered image in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature; Obtaining a target feature of the ice-covered image according to the first feature and the upsampling module; The initial segmentation model is optimized based on the target features and a preset loss function until the initial segmentation model meets the preset conditions, thereby obtaining a target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line.
2. The method according to claim 1, characterized in that The feature extraction module further includes a pooling unit, which performs feature extraction on the ice-covered image in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature, including: For each ice-covered image in the ice-covered image set, input the current ice-covered image into the initial segmentation model to obtain a first output feature of the current ice-covered image output by a convolution unit of the initial segmentation model; Performing a dilated convolution process on the first output feature through the dilated convolution unit to obtain a second output feature; Performing feature extraction on the second output feature by the pooling unit to obtain a third output feature; The third output feature is weighted according to the attention unit to obtain the first feature.
3. The method according to claim 2, characterized in that Performing a dilated convolution on the first output feature by the dilated convolution unit to obtain a second output feature includes: Performing dimensionality reduction convolution and dilated convolution on the first output feature through the first convolution layer of the dilated convolution unit to obtain a first dilated feature; Performing dimension reduction convolution on the first output feature through the second convolution layer of the hole convolution unit to obtain a second hole feature; The first hole feature and the second hole feature are element-wise added to obtain a candidate second output feature, and the second output feature is obtained based on the candidate second output feature.
4. The method according to claim 2, characterized in that: The pooling unit includes a first convolutional layer, a second convolutional layer, a third convolutional layer and a maximum pooling layer, and the second output feature is extracted by the pooling unit to obtain a third output feature, including: Extracting features of the second output feature through the first convolution layer, the second convolution layer, and the third convolution layer, respectively, to obtain a first convolution feature, a second convolution feature, and a third convolution feature; Performing feature fusion on the second output feature through the maximum pooling layer to obtain a fourth pooling feature; The first convolution feature, the second convolution feature, the third convolution feature and the fourth pooling feature are concatenated to obtain the third output feature.
5. The method according to claim 2, characterized in that: The weight adjustment of the third output feature according to the attention unit to obtain the first feature includes: Performing dimension reduction convolution on the third output feature through the convolution layer of the attention unit, and performing spatial weighting on the third output feature and the third output feature after the dimension reduction convolution to obtain a first attention feature; Performing global pooling processing on the third output unit through the global pooling layer of the attention unit, and performing channel weighting on the third output feature and the third output feature after the global pooling processing to obtain a second attention feature; The first attention feature and the second attention feature are element-wise added to obtain the first feature.
6. The method according to claim 1, characterized in that Obtaining a target feature of the ice-covered image according to the first feature and the upsampling module includes: For each first feature of the ice-covered image, determine the input feature of the upsampling module based on the first feature of the current ice-covered image, the feature extraction module and the dilated convolution unit of the feature extraction module; By means of the up-sampling module, based on the pixel position of the current ice-covered image, the input feature of the up-sampling module is dynamically weighted by transposed convolution and bilinear interpolation to obtain a second feature; The target feature is determined based on the second feature, the first feature, the feature extraction module, the upsampling module and a convolution unit of the initial segmentation model.
7. The method according to claim 1, characterized in that The method further comprises: Acquire the ice-covered image to be processed of the transmission line; Inputting the ice-covered image to be processed into the target segmentation model, performing image segmentation on the ice-covered image to be processed by the target segmentation model, and obtaining a segmentation result image of the ice-covered image to be processed; The segmentation result map is displayed to a user, so that the user can determine the ice coverage condition of the transmission line based on the segmentation result map.
8. An ice-covered image segmentation device, characterized in that: include: A data acquisition module, used for acquiring an ice-covered image set of the transmission line when the pre-established initial segmentation model does not meet the preset conditions; The initial segmentation model includes a feature extraction module and an upsampling module; A first determination module, configured to extract features of the ice-covered images in the ice-covered image set based on the hole convolution unit and the attention unit of the feature extraction module to obtain a first feature; A second determination module, configured to obtain a target feature of the ice-covered image according to the first feature and the upsampling module; A model optimization module is used to optimize the initial segmentation model based on the target features and a preset loss function until the initial segmentation model meets the preset conditions, thereby obtaining a target segmentation model, which is used to perform image segmentation on the ice-covered image of the power transmission line.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, an ice-covered image segmentation method as described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an ice-covered image segmentation method as claimed in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Cartilage image segmentation method and device, readable storage medium and terminal equipment
CN110598714A
Pedestrian re-recognition method based on attention mechanism part shielding
CN111898431A
Image processing method and model, computer equipment and storage medium
CN115841582A
Power transmission line icing identification method and device, electronic equipment and medium
CN117830259A
Overhead transmission line icing form segmentation method
CN118411724A
