A wind turbine image edge enhancement and multi-scale reconstruction method and system

Abstract

The application discloses a kind of wind turbine image edge enhancement and multi-scale reconstruction method and system, belong to digital image processing and generation technical field.The method is carried out size transformation and normalization processing to wind turbine image collected by unmanned aerial vehicle or satellite;Fixed Sobel convolution kernel is used to extract horizontal edge and vertical edge, and is fused with hollow convolution feature to generate multi-scale edge enhancement feature;Scale recovery feature is fused by way of double-path gating of deconvolution and interpolation, and input residual error is stacked to recover image scale gradually;According to global edge map and channel average feature, spatial enhancement and feature reconstruction are implemented, and reconstructed image data composed of continuous response values of each pixel target area is output.The application can reduce small target edge loss, scale recovery blur and chessboard-like artifacts, and is used for wind farm image edge enhancement, scale recovery and contour map generation.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing, image enhancement and image generation technology, and specifically relates to a method and system for edge enhancement and multi-scale reconstruction of wind turbine images. Background Technology

[0002] Location and contour information in wind turbine images can be used for wind farm planning, turbine positioning, equipment inspection, and regional wind power resource surveys. UAVs can acquire near-to-medium range images of wind turbines, while satellites can acquire remote sensing images of large-scale wind farms. By performing edge enhancement, scale transformation, and spatial reconstruction on the pixel arrays of these digital images, target area image data that preserves the contour responses of the wind turbine region, blades, and tower can be generated.

[0003] Existing multi-scale image processing networks typically employ an encoder-decoder structure. The encoder extracts image features at different scales through continuous downsampling, while the decoder reconstructs spatial resolution through scale recovery and fuses shallow texture features and deep contextual features using skip connections. For long-range UAV and satellite images, wind turbines occupy a relatively low percentage of pixels in the overall image, and their blades and tower edges are narrow and easily affected by road, building, and ground textures. Continuous downsampling weakens local edge responses, making it difficult to reconstruct the complete contour in subsequent scale recovery.

[0004] In image scale restoration, deconvolution can learn high-frequency details, but it is prone to forming checkerboard artifacts with uneven pixel distribution; interpolation scale transformation is simple to compute and produces smooth output, but it is prone to losing edge and texture details. Using only one of these scale restoration methods makes it difficult to simultaneously preserve local contour details, spatial continuity, and output smoothness.

[0005] Furthermore, there are expressive differences between shallow image features and deep reconstructed features. Direct skip connections may introduce redundant background textures, and constraining only the final output is insufficient to stabilize the edge response at intermediate scales. Therefore, a digital image processing scheme for wind turbines is needed, which focuses on pixel array transformation, edge enhancement, scale restoration, and spatial reconstruction. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for edge enhancement and multi-scale reconstruction of wind turbine images, which solves the problems of easy loss of local edges of small targets, blurred scale recovery or the generation of checkerboard artifacts and insufficient connection of multi-scale features in digital images with complex backgrounds, and generates target area image data that maintains spatial position and contour response.

[0007] To achieve the above objectives, this invention acquires and preprocesses wind turbine images collected by UAVs or satellites, and inputs them into a multi-scale image processing network consisting of an encoding path and a reconstruction path. The encoding path uses a fixed Sobel convolution kernel to extract horizontal and vertical edges, and combines dilated convolution features to generate edge attention weights, resulting in multi-scale edge enhancement encoding features. The reconstruction path adopts a dual-path gating method of deconvolution and interpolation, fusing the two-path scale recovery features according to the gating map and its complementary weights, and superimposing the input residuals restored by channel matching and interpolation to complete the step-by-step scale recovery. The spatial edge enhancement module generates spatial attention based on the global edge map and channel average features, enhances the encoding features at the corresponding scale, and then concatenates them with the gating fusion features for spatial reconstruction. During the training phase, intermediate scale constraints are implemented through an auxiliary target region map. The image reconstruction output module maps the reconstruction features to target region representation channels, generating a wind turbine target region reconstruction map consisting of continuous response values ​​of each pixel and corresponding to the input image.

[0008] This invention also provides a system for edge enhancement and multi-scale reconstruction of wind turbine images, an electronic device, and a computer-readable storage medium. The system includes an image acquisition module, a size adjustment and normalization module, a local edge enhancement encoding module, a bottleneck feature extraction module, a multi-scale gated reconstruction module, a spatial edge enhancement module, a feature fusion and reconstruction module, an auxiliary reconstruction constraint module, an image reconstruction output module, and a joint loss calculation module, used to execute the aforementioned image processing method.

[0009] Compared with existing technologies, this invention uses a fixed Sobel convolution kernel to enhance the local image response of the narrow edges of the wind turbine. It employs a dual-path gated scale recovery method consisting of deconvolution, interpolation, and input residuals to balance high-frequency details and spatial smoothness. Furthermore, it utilizes spatial edge enhancement, feature fusion, and assisted reconstruction constraints to improve the spatial reconstruction effect of multi-scale image features. This reduces the loss of small target contours, scale recovery blurring, and checkerboard artifacts, resulting in reconstructed image data of the wind turbine target area with continuous spatial location and complete contour response. Attached Figure Description

[0010] Figure 1 This is a system structure block diagram according to an embodiment of the present invention.

[0011] Figure 2 This is a flowchart of the method described in an embodiment of the present invention.

[0012] Figure 3 This is a schematic diagram of the multi-scale local edge enhancement structure described in an embodiment of the present invention.

[0013] Figure 4 This is a schematic diagram of the dual-path gating scale recovery and spatial reconstruction structure described in an embodiment of the present invention. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions or modifications can be made by those skilled in the art without departing from the concept of the present invention, and all such substitutions or modifications should fall within the scope of protection of the present invention.

[0015] Example 1: Image Acquisition, Sample Construction, and Preprocessing. A color wind turbine image is acquired. A single original image is represented as H×W×3, where H represents the image height, W represents the image width, and 3 represents the three color channels: red, green, and blue. The training label is a single-channel target region map corresponding to the pixels of the original image, with each pixel value being a region number. In two-region processing, the numbers represent the background and the wind turbine, respectively; in three-region processing, the numbers represent the background, blades, and tower, respectively. The training label does not use three color channels to avoid mistaking color values ​​for target region numbers.

[0016] The original image and the label target region map are simultaneously adjusted to 256×512 pixels. The original image is resized using bilinear interpolation, while the label target region map is resized using nearest neighbor interpolation to prevent the generation of non-existent region numbers. Image pixels are converted to floating-point numbers and scaled to 0 to 1, then normalized according to the mean and standard deviation of the three color channels. When batching inputs into the network, the image tensor shape is B×3×256×512, and the label tensor shape is B×256×512, where B represents the batch size; B is 2 for satellite image training and 8 for UAV image training.

[0017] Let the pixel value of the c-th color channel of the original image be... The corresponding channel mean and standard deviation are respectively and Then the normalized pixels input to the network Represented as:

[0018]

[0019] Here, c takes the values ​​1, 2, or 3, corresponding to the red, green, and blue channels, respectively. Normalization is used to unify the numerical range of each channel in the color image; the label target region map retains integer region numbers and does not participate in channel normalization.

[0020] One dataset uses 224 UAV images with a resolution of 1920×1080, of which 187 are used for training and 37 for testing. Another dataset uses 981 satellite images, of which 871 are used for training and 110 for testing. In the satellite image labels, background pixels account for approximately 99.8%, and wind turbine pixels account for approximately 0.2%, representing a small target image reconstruction scenario with a low foreground area. The training set is used to update the image processing network parameters, while the test set is used to evaluate the contour integrity and region overlap of the reconstructed target region.

[0021] Samples are constructed according to the correspondence between image files and label files. Each sample includes a color image and a pixel-level label target region map. During the training phase, the order of training samples is shuffled and they are read in batches to ensure that the image tensors and label tensors in the same batch maintain a one-to-one correspondence in sample dimension and spatial location. During the testing phase, the samples are read one by one in a fixed order so that the generated target region reconstruction map can be compared with the original image and the real label target region map.

[0022] Example 2: Multi-scale local edge enhancement coding. For example... Figure 3 As shown, the preprocessed image is input into the local edge enhancement coding module. The network includes a four-level downsampling coding stage and a bottleneck feature extraction stage, with the number of output channels for the five scales being 16, 32, 64, 128, and 256, respectively. When the input size is B×3×256×512, the spatial dimensions of the features at the five scales are 256×512, 128×256, 64×128, 32×64, and 16×32, respectively. Shallow features preserve local edges and textures, while deep features expand the receptive field and form a contour-related contextual image representation.

[0023] For any encoding scale, let the input features be... First, basic convolution features are obtained through 3×3 basic convolution, and then dilated convolution features are obtained through 3×3 dilated convolution; the Sobel convolution kernel is fixed. and Both methods extract horizontal and vertical edges from the basic convolutional features. Figure 3 For simplicity, both edge extraction branches take basic convolutional features as input, and the basic feature extraction includes the dilated convolution operation. The two fixed convolutional kernels are represented as follows:

[0024]

[0025] in, This represents the horizontal Sobel convolution kernel. This represents the vertical Sobel convolution kernel. Using... and Perform grouped convolutions on the basic convolutional features to obtain the horizontal edge responses separately. and vertical edge response Synthetic edge intensity map Represented as:

[0026]

[0027] in, To maintain stable positive values ​​in image numerical calculations, this embodiment takes... The synthesized edge intensity map is concatenated with dilated convolutional features, and edge attention weights are generated through convolution and sigmoid function activation. The local edge enhancement encoding features are obtained through feature refinement convolution and residual connection. :

[0028]

[0029] in, This indicates a dilated convolution feature; square brackets indicate channel concatenation. Represents a convolution mapping. Represents an sigmoid function. This indicates element-wise multiplication. The fixed Sobel kernel does not participate in backpropagation updates and is used to stably extract local edges; the channel compression ratio of the local edge enhancement coding module is 2, and the number of intermediate channels is half the number of input channels.

[0030] Example 3: Dual-path gated scale recovery. (For example...) Figure 4 As shown, the current low-resolution image features are input into the deconvolutional scale recovery branch and the interpolation scale recovery branch, respectively. The deconvolutional branch recovers edge and texture details using learnable convolutional kernels, while the interpolation branch forms a smooth spatial distribution through parameterless interpolation. The gating generation convolutional block generates a gating map based on the current image features, and this gating map and its complementary weights are used to adjust the contributions of the two scale recovery methods at various spatial locations.

[0031] Let the input features be First, intermediate features are obtained through feature convolution. , The deconvolutional scale recovery features are obtained by performing deconvolution with a stride of 2 and bilinear interpolation followed by a 1×1 convolution. and interpolation scale recovery features Gating diagram Depend on After interpolation to the target size, it is obtained by gated convolution and sigmoid function, and its dual-path scaling relationship is as follows:

[0032]

[0033] in, For deconvolution scale recovery features, To recover features for interpolation scale, Indicates bilinear interpolation. This represents a channel matching mapping. (Gated graph) Its complementary quantity The contributions of the two image features are adjusted separately, and the input residuals are superimposed to form gated fusion features; the number of intermediate channels is the larger of half the number of input channels and 16.

[0034] Example 4: Edge Map Generation and Spatial Enhancement. A global edge map is generated by performing continuous convolutions on the encoded features at the corresponding scale that contain local edge information. Simultaneously, the encoded features are globally averaged along the channel direction to obtain channel-averaged features. The global edge map and channel-averaged features are concatenated, and spatial attention features are generated through convolution with a large receptive field. These spatial attention features are then used to enhance the encoded features. The number of intermediate channels used to generate the global edge map is twice the number of input channels, and the number of input channels and output channels of the spatial edge enhancement module are the same.

[0035] Let the corresponding scale encoding feature be Global edge map Channel average characteristics Spatial attention Image features after spatial enhancement The following are represented as follows:

[0036]

[0037]

[0038]

[0039]

[0040] in, This represents the edge map extraction unit. Represents a convolution mapping. Indicates batch normalization. Represents the linear rectification activation function. Spatial attention generated for 7×7 convolutions. Edge map. Spatial attention is generated together with channel average features. , With coding features After multiplication, convolution thinning and residual connection are performed to obtain spatially enhanced features. Do not repeat multiplication during the enhancement phase.

[0041] Example 5: Multi-scale image feature fusion, spatial reconstruction, and auxiliary constraints. Spatial enhancement features... Features of gating integration The current-level spatial reconstruction features are obtained by concatenating the channels, performing two 3×3 convolutions, batch normalization, and linear rectified activation. The four-level reconstruction recovers the input using gated scales with 256, 128, 64, and 32 channels respectively, and then fuses them with the corresponding scale encoded features with 128, 64, 32, and 16 channels respectively. Figure 2 The corresponding image features described in S6 are first spatially enhanced in S7, and then spliced ​​with the gated fusion features.

[0042] Spatial enhancement features at various scales as Figure 2 The intermediate reconstruction features described in S8 form auxiliary reconstruction constraint paths: after bilinear interpolation to restore the input size, auxiliary target region maps are generated through 3×3 convolution, batch normalization, linear rectified activation, and 1×1 convolution. Auxiliary loss is calculated during the training phase; during image processing, auxiliary loss is disabled, and only the final-level reconstruction features generate continuous response maps of the main target region.

[0043] Example 6: Overall network construction, training, and reconstruction output. (Following...) Figure 1 The network is constructed as follows: the image acquisition and preprocessing module outputs a standard-size tensor; the local edge enhancement coding module and the bottleneck feature extraction module output five-scale features; the multi-scale gated reconstruction module restores resolution through dual paths and input residuals; the spatial edge enhancement module processes the corresponding encoded features; the feature fusion and reconstruction module completes spatial reconstruction; the auxiliary reconstruction constraint module outputs an intermediate target region map; the image reconstruction output module generates a continuous target region response map; and the joint loss calculation module summarizes the loss during the training phase.

[0044] The final spatial reconstruction features are mapped to C target region representation channels via 1×1 convolution and restored to 256×512. The response of each pixel channel is normalized to obtain a continuous target region response map of B×C×256×512, which is used as the image reconstruction output. When displaying or calculating the intersection-over-union ratio (IoU), the region number with the largest response is used to generate a single-channel mask; in the two-region mask, 0 and 1 represent the background and wind turbine, while in the three-region mask, 0, 1, and 2 represent the background, blades, and tower. The continuous response map or mask is restored to the original image H×W; the mask is only used for display and evaluation.

[0045] Set pixel position In the The response on the target region characterization channel is ,in and Both represent channel indices, so the normalized channel response is... This represents the response value of the corresponding pixel in the reconstructed image of a continuous target region; when discrete display or calculation of the region intersection-over-union ratio is required, the pixel... Discrete output values Represented as:

[0046]

[0047] The system can be deployed on an electronic device that includes a processor, memory, and an image input interface. The memory stores the network structure, training parameters, and image processing program. The processor calls various modules to complete edge enhancement, scale restoration, feature fusion, spatial reconstruction, and generation of response maps for continuous target regions. The program is stored on a computer-readable storage medium, and when executed by the processor, it implements the above methods.

[0048] Cross-entropy loss and focusing loss are used during training. Similarity loss and Generation of the target region response map constrained by intersection-union ratio loss. Let... This represents the total number of channels in the region. This is a one-hot encoding of the real label target region map. For a continuous target region, the pixel in the response map belongs to the first... Normalized response of each target region For regional balancing weights, To focus parameters, This is the set of target pixels generated after discretizing the continuous response map. Given the set of real target pixels, each loss is represented as follows:

[0049]

[0050]

[0051] in, In this embodiment, the value is 2. For UAV images, a cross-entropy loss with a weight of 1.0 is used when no auxiliary reconstruction constraint is applied; when auxiliary reconstruction constraint is applied, a cross-entropy loss of 1.0 is used. Similarity loss and Cross-entropy (COP) loss is used, with weights of 1.0, 0.2, and 0.2. For satellite imagery, without auxiliary reconstruction constraints, cross-entropy loss and focus loss are used, with weights of 1.0 and 0.4 respectively; when auxiliary reconstruction constraints are applied, an additional loss loss with a weight of 0.2 is added. Similarity loss and Intersection over Union (IoU) loss. The joint loss of the reconstructed main target region map and the auxiliary target region map is expressed as:

[0052]

[0053] in, The loss is used to reconstruct the map of the main target region. The The item is the first Auxiliary target area map loss, To assist in reconstructing the number of constraint heads, The weights are non-negative auxiliary loss weights. When training with the Adam optimizer, the learning rate for UAV image tasks is 0.001 and the batch size is 8; the learning rate for satellite image tasks is 0.0001 and the batch size is 2; the maximum number of training epochs is 350. Training is stopped and the optimal model parameters for the target region reconstruction metric are saved when the validation metric fails to improve for 25 consecutive epochs.

[0054] Example 7: Verification of Target Region Reconstruction Effect. Table 1 shows the intersection-union ratio (IU) of the regions after discretization of the continuous response map. In the satellite data ablation experiment, the IU of the basic network and the wind turbine regions after sequentially adding local edge enhancement, dual-path gated scale recovery, spatial edge enhancement, and auxiliary constraints were 0.5897, 0.6093, 0.6172, and 0.6321, respectively. The complete network has 4.86M parameters and a computational load of 22.00 GFLOPs, where M represents millions of parameters and GFLOPs represents billions of floating-point operations.

[0055] Table 1 Comparison of multi-scale reconstruction results of wind turbine images

[0056] Dataset Model background blade Tower / Wind Turbine average value drones DeepLabV3Plus 0.9937 0.9129 0.8695 0.9254 drones This invention network 0.9957 0.9402 0.9155 0.9505 satellite SegNet 0.9989 — 0.5921 0.7955 satellite This invention network 0.9991 — 0.6321 0.8156

Claims

1. A method for edge enhancement and multi-scale reconstruction of wind turbine images, characterized in that, Includes the following steps: (1) Obtain the image of the wind turbine to be processed, and perform size transformation and normalization on the image of the wind turbine to be processed to obtain a standard size image; (2) Input the standard-size image into the encoder and use the local edge enhancement coding module to generate edge enhancement coding features of different scales. The local edge enhancement coding module uses a fixed Sobel convolution kernel to extract horizontal edge features and vertical edge features, and generates edge attention weights based on the horizontal edge features, the vertical edge features and the dilated convolution features. (3) Input the edge enhancement coding features of different scales into the multi-scale image reconstruction path, perform step-by-step scale recovery on the low-resolution image features, and fuse the deconvolution scale recovery features and interpolation scale recovery features through the multi-scale gated reconstruction module. Use the input features recovered by channel matching and interpolation as residuals to obtain the gated fusion features. (4) The spatial edge enhancement module generates spatial attention features based on the global edge map and channel average features, enhances the coding features at the corresponding scale based on the spatial attention features, and performs feature splicing and spatial reconstruction with the enhanced coding features and the gated fusion features; during the training phase, the auxiliary reconstruction constraint module generates auxiliary target region maps based on the enhanced coding features at each scale and implements auxiliary constraints. (5) Input the image features after hierarchical feature splicing and spatial reconstruction into the image reconstruction output module, and map them into multiple target region representation channels through convolution. Normalize the channel response of each pixel and restore it to the size of the wind turbine image to be processed, and generate a pixel-level wind turbine target region reconstruction map composed of continuous response values ​​of the target region.

2. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 1, characterized in that, In step (2), the local edge enhancement coding module includes a basic feature extraction unit, a horizontal edge extraction unit, a vertical edge extraction unit, and an attention weight generation unit.

3. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 2, characterized in that, The basic feature extraction unit extracts basic convolution features and dilated convolution features using basic convolution operations and dilated convolution operations, respectively. The horizontal edge extraction unit and the vertical edge extraction unit extract horizontal edge features and vertical edge features from the basic convolution features using a fixed Sobel convolution kernel, respectively.

4. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 3, characterized in that, The attention weight generation unit fuses the horizontal edge features and the vertical edge features into edge intensity features, and concatenates the edge intensity features with the dilated convolution features, and obtains the edge attention weights through convolution and activation operations.

5. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 1, characterized in that, In step (3), the multi-scale gated reconstruction module generates a gated graph by generating convolutional blocks through gated generation. The gated graph and its complementary weights are used to weight the deconvolutional scale recovery feature and the interpolation scale recovery feature, respectively. The weighted features are added together and superimposed with the input features that have been channel matched and interpolated to obtain the gated fusion feature.

6. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 1, characterized in that, In step (4), the spatial edge enhancement module includes an edge map extraction unit and a spatial attention generation unit. The edge map extraction unit generates a global edge map based on the coding features of the corresponding scale, and the spatial attention generation unit generates spatial attention features based on the global edge map and the channel average features. The auxiliary reconstruction constraint module outputs an auxiliary target region map based on the enhanced coding features of each scale during the training phase.

7. The wind turbine image edge enhancement and multi-scale reconstruction method according to claim 6, characterized in that, The training phase employs the main loss corresponding to the main target region reconstruction map and the auxiliary loss corresponding to the auxiliary target region map to jointly train the multi-scale image reconstruction network.

8. A system for edge enhancement and multi-scale reconstruction of wind turbine images, characterized in that, It includes an image acquisition module, a size adjustment and normalization module, a local edge enhancement coding module, a bottleneck feature extraction module, a multi-scale gated reconstruction module, a spatial edge enhancement module, a feature fusion and reconstruction module, an auxiliary reconstruction constraint module, an image reconstruction output module, and a joint loss calculation module. Each module is used sequentially for image acquisition and preprocessing, multi-scale edge enhancement coding, bottleneck feature extraction, gated scale recovery, spatial edge enhancement, feature fusion and reconstruction, auxiliary constraints, continuous target region response map output, and joint loss calculation.

9. An electronic device comprising a processor and a memory, characterized in that, The memory stores a computer program, which, when executed by the processor, implements the method of claim 1.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of claim 1.

Images