Image enhancement processing method and system based on deep learning

Abstract

The invention discloses an image enhancement processing method and system based on deep learning, and relates to the technical field of image enhancement, and the method comprises the following steps: obtaining a power equipment image, carrying out the coding and decoding of the power equipment image, and obtaining a plurality of edge domain and one core domain decoding image; performing feature extraction on the edge domain decoded image and the core domain decoded image through a preset feature extraction network; according to the image enhancement processing method based on deep learning, the problems of poor image quality and low resolution of the power equipment are solved, and the image quality of the power equipment is effectively enhanced.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and more specifically, to an image enhancement processing method and system based on deep learning. Background Art

[0002] With the continuous development of the power system, the coverage of the distribution network is expanding, the environment is becoming more and more complex, and the equipment is diverse and widely distributed. The operating status of power equipment is directly related to the stability and safety of the power system. Any fault may cause large-scale power outages and even cause serious social and economic consequences. Traditional manual inspection and maintenance methods have been difficult to meet the needs of efficient, safe and accurate operation and maintenance. Especially in complex distribution network environments, such as mountainous areas, forest areas or densely populated urban areas, power equipment is easily blocked by trees, buildings, etc., and the quality and clarity of image acquisition are greatly reduced. Although traditional image enhancement technology has achieved remarkable results in image enhancement, it also has some obvious shortcomings and limitations. For example, contrast enhancement methods such as histogram equalization are prone to excessive or insufficient enhancement of local details of the image when processing global contrast, affecting the naturalness of the image; although sharpening filters and high-pass filters can enhance the edges and details of the image, they are also very sensitive to noise and will amplify the noise in the image, resulting in a decrease in image quality.

[0003] In view of the above problems, the present invention proposes a solution. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an image enhancement processing method and system based on deep learning. Through the image enhancement processing method and system based on deep learning, the shortcomings of traditional methods in image enhancement of power equipment are solved.

[0005] To achieve the above object, the present invention provides the following technical solutions: The image enhancement processing method based on deep learning includes the following steps: acquiring an image of electric power equipment, encoding and decoding the image of electric power equipment to obtain a plurality of edge domains and a core domain decoded image; performing feature extraction on the edge domain and core domain decoded images respectively through a preset feature extraction network; and constructing an edge domain and core domain image enhancement reconstruction network for image enhancement.

[0006] In a preferred embodiment, the encoding and decoding of the power equipment image to obtain edge domain and core domain decoded images are specifically: dividing the power equipment image into an edge domain image and a core domain image; encoding the edge domain image and the core domain image; decoding the encoded edge domain and core domain images to obtain edge domain and core domain decoded images.

[0007] In a preferred embodiment, the power equipment image is segmented into an edge domain image and a core domain image, specifically: the power equipment image is converted into grayscale to obtain a grayscale image of the power equipment; based on an edge detection algorithm, edge detection is performed on the grayscale image of the power equipment to obtain an edge detection result; based on the edge detection result, the edge part and the core part of the grayscale image of the power equipment are extracted to form an edge domain image and a core domain image.

[0008] In a preferred embodiment, the edge domain and core domain decoded images are subjected to feature extraction through a preset feature extraction network respectively, specifically: a plurality of edge domain decoded images are subjected to feature extraction through a preset feature extraction network to obtain a plurality of edge domain low-resolution convolution feature maps; the core domain decoded images are subjected to feature extraction through a preset feature extraction network to obtain a core domain low-resolution convolution feature map; In a preferred embodiment, the edge domain and core domain decoded images are respectively subjected to feature extraction through a preset feature extraction network, specifically: the edge domain and core domain decoded images are subjected to convolution block operations to convert the decoded images into convolution feature values, wherein the convolution block operations include convolution operations, batch normalization, and activation function calculations; a residual convolution block is constructed to sequentially extract features from the convolution feature values, and the feature extracted feature values ​​are subjected to channel weighted average fusion for several times to obtain fused feature values, wherein the residual convolution includes convolution operations, batch normalization, activation function calculations, and jump connections; the fused feature values ​​are subjected to channel weighted average fusion with the convolution feature values ​​to obtain a low-resolution convolution feature map.

[0009] In a preferred embodiment, the residual convolution block is constructed to extract features from the convolution feature values ​​in sequence, specifically: the preset feature extraction network includes several layers of convolution blocks, residual convolution blocks and convolution layers; the convolution feature values ​​output by the convolution block of the current layer are used as inputs of the residual convolution block and the convolution layer of the same layer; the outputs of the residual convolution block and the convolution layer of the same layer are linearly summed as inputs of the convolution block of the next layer; the outputs of the residual convolution block and the convolution layer of the last layer are linearly summed as low-resolution convolution feature maps; In a preferred embodiment, the edge domain and core domain image enhancement and reconstruction network are constructed to perform image enhancement, specifically as follows: a mean absolute error loss function is constructed as a loss function in the image enhancement and reconstruction network; the obtained edge domain low-resolution convolution feature map is input into several convolution blocks with jump connections introduced in series to obtain an edge domain reconstructed feature map; the edge domain reconstructed feature is input into an upsampling convolution layer to obtain an edge domain decoded enhanced image; the obtained several edge domain low-resolution convolution feature maps are fused to obtain an edge domain fused feature map; the obtained edge domain fused feature map is concatenated with the core domain low-resolution feature map along the channel dimension to obtain a core domain reconstructed feature map.

[0010] The image enhancement processing system based on deep learning includes a data acquisition module, a feature extraction module and an image enhancement module: the data acquisition module acquires the image of the power equipment, encodes and decodes the image of the power equipment to obtain a plurality of edge domains and a core domain decoded image; the feature extraction module extracts features from the edge domain and core domain decoded images respectively through a preset feature extraction network; the image enhancement module constructs an edge domain and core domain image enhancement reconstruction network for image enhancement.

[0011] The technical effects and advantages of the image enhancement processing method and system based on deep learning of the present invention are as follows: 1. The present invention obtains an image of power equipment, encodes and decodes the image of power equipment, obtains several edge domains and one core domain decoded images, extracts features from the edge domain and core domain decoded images respectively through a preset feature extraction network, and finally constructs an edge domain and core domain image enhancement and reconstruction network for image enhancement. First, the image is divided into an edge domain image and a core domain image, which can better optimize the features of different regions. The edge domain image usually contains more details and edge information. Through special processing by the feature extraction network, the clarity and detail performance of the image can be effectively enhanced. The core domain image contains more overall information, and special processing of the core domain can ensure the overall structure and consistency of the image. In addition, the application of the feature extraction network and the image enhancement and reconstruction network enables the method to fully utilize the advantages of deep learning technology. Through the preset feature extraction network, high-level feature information can be extracted from the edge domain and core domain images, providing a rich feature basis for subsequent image enhancement. The image enhancement and reconstruction network can effectively fuse and reconstruct the extracted feature information into a high-quality image to ensure the naturalness and consistency of the enhancement effect.

[0012] 2. The present invention uses deep learning technology to perform image enhancement on the collected images of power equipment, which can effectively improve the clarity and contrast of the image, so that low-quality images taken in complex environments are significantly improved, thereby improving the accuracy of subsequent image recognition; secondly, the deep learning algorithm can adaptively process images of different scenes and equipment, has strong generalization capabilities, and can effectively cope with diverse power equipment and complex environments; at the same time, image enhancement technology can also reduce the recognition difficulties caused by factors such as insufficient lighting and occlusion, and improve the overall efficiency and safety of power inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a structural schematic diagram of the image enhancement processing method based on deep learning of the present invention.

[0014] Figure 2 This is a schematic diagram of the structure of the image enhancement processing system based on deep learning of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Embodiment 1, Figure 1 The present invention provides an image enhancement processing method and system based on deep learning, comprising the following steps: S1, obtaining an image of a power device, encoding and decoding the image of the power device, and obtaining a plurality of edge domains and a core domain decoded image; S2, extracting features from the edge domain and core domain decoded images through a preset feature extraction network respectively; S3, construct edge domain and core domain image enhancement reconstruction network for image enhancement.

[0017] In this example, the power equipment image is specifically: The power equipment images include appearance images of equipment such as transformers, circuit breakers, and cables.

[0018] In this example, the power equipment image is encoded and decoded to obtain a plurality of edge domains and a core domain decoded image, specifically: Segment the power equipment image into an edge domain image and a core domain image; Encoding the edge domain image and the core domain image; The encoded edge domain and core domain images are decoded to obtain edge domain and core domain decoded images.

[0019] In this example, the power equipment image is segmented into an edge domain image and a core domain image, specifically: Convert the power equipment image into grayscale to obtain a grayscale image of the power equipment; Based on an edge detection algorithm, edge detection is performed on the grayscale image of the electric power equipment to obtain an edge detection result; Based on the edge detection result, the edge part and the core part in the grayscale image of the power equipment are extracted to form an edge domain image and a core domain image.

[0020] In this example, the grayscale conversion of the power equipment image to obtain the power equipment grayscale image is specifically as follows: The RGB value of the image to be enhanced is converted into a gray value by the weighted average method, where the weighted average calculation formula is:

[0021] Among them, R is the red channel value, G is the green channel value, B is the blue channel value, and g is the grayscale value. The range of each channel value is between 0 and 255.

[0022] It should be noted that the grayscale image of power equipment has only one channel, while the color image has three channel values. The grayscale image of power equipment is obtained by grayscale conversion, which emphasizes the contrast of brightness and is conducive to highlighting the texture, edge and other features of the image.

[0023] In this example, the edge detection algorithm is specifically: The Canny edge detection algorithm is used to detect the edges in the power equipment image while reducing the influence of noise. The main steps of the algorithm are as follows: Perform Gaussian filtering on the grayscale image of power equipment to reduce the impact of noise on edge detection; Use the Sobel operator to calculate the gradient intensity and direction of the grayscale image of the power equipment; For each pixel, according to its gradient direction, check whether it is a local maximum in the neighborhood of its gradient strength. If the gradient strength of the pixel is not a local maximum, set its gradient strength to 0. Double threshold processing is used to divide the gradient intensity into strong edge, weak edge and core areas; Starting from the strong edge, the weak edge is traced along the gradient direction, the weak edges connected to the strong edge are retained as the real edge, and the weak edges not connected to the strong edge are suppressed as the core domain.

[0024] In this example, dual threshold processing is used to divide the gradient intensity into strong edge, weak edge and core area, specifically: Set a high threshold and a low threshold. Pixels with gradient strength greater than the high threshold are strong edges, pixels with gradient strength between the low threshold and the high threshold are weak edges, and pixels with gradient strength less than the low threshold are core areas.

[0025] It should be noted that the Canny edge detection algorithm is a powerful and robust edge detection method. It can effectively detect edges in images through steps such as smoothing, gradient calculation, non-maximum suppression, double threshold processing and edge connection, and can maintain good performance in the presence of noise.

[0026] In this example, encoding the edge domain image and the core domain image is specifically performed as follows: Based on the encoding method, the edge domain image and the core domain image are input into the encoder to generate encoding features; The coding features are fused to obtain the final coding.

[0027] In this example, the encoded edge domain and core domain images are decoded to obtain edge domain and core domain decoded images, specifically: The generated encoded features are input into the decoder, which restores the encoded features to the original image.

[0028] In this example, the edge domain and core domain decoded images are respectively passed through a preset feature extraction network, specifically: Extract features from several edge domain decoded images through a preset feature extraction network to obtain several edge domain low-resolution convolution feature maps; Extracting features from the core domain decoded image through a preset feature extraction network to obtain a core domain low-resolution convolution feature map; In this example, the feature extraction is performed by a preset feature extraction network, specifically: Performing a convolution block operation on the edge domain and core domain decoded images to convert the decoded images into convolution feature values, wherein the convolution block operation includes convolution operation, batch normalization, and activation function calculation; Constructing a residual convolution block to extract features from the convolution eigenvalues ​​in turn, and performing channel weighted average fusion of the extracted eigenvalues ​​several times to obtain fused eigenvalues, wherein the residual convolution block includes convolution operation, batch normalization, activation function calculation, and skip connection; The fused feature value and the convolution feature value are fused channel-wise by weighted averaging to obtain a low-resolution convolution feature map.

[0029] In this example, the convolution block operation includes convolution operation, batch normalization and activation function calculation, specifically: The convolution operation extracts the local features of the image through several convolution kernels, and performs sliding window operations on the edge domain and core domain decoded images through the convolution kernel to generate a feature map; batch normalization standardizes the output features of each batch to make them have zero mean and unit variance, thereby speeding up model training and reducing gradient vanishing and explosion problems; activation function calculation can solve the gradient vanishing problem.

[0030] In this example, the residual convolution block is constructed to extract features from the convolution feature values ​​in sequence, specifically: The preset feature extraction network includes several layers of convolution blocks, residual convolution blocks and convolution layers; Using the convolution feature value output by the convolution block of the current layer as the input of the residual convolution block and the convolution layer of the same layer; Linearly sum the output of the residual convolution block and the convolution layer of the same layer as the input of the convolution block of the next layer; Linearly sum the output of the residual convolution block and the convolution layer of the last layer as a low-resolution convolution feature map; In this example, the residual convolution block includes convolution operation, batch normalization, activation function calculation and skip connection, specifically: The residual convolution block includes a basic convolution block and a residual connection. The specific expression is:

[0031] in, is the output of the residual convolution block, is the output of the convolutional block, is the weight parameter of the convolutional layer, and x is the input.

[0032] It should be noted that the residual convolution block solves the gradient vanishing and gradient exploding problems in deep network training by introducing residual connections, making it possible to train deeper neural networks. It simplifies the optimization problem, accelerates training convergence, improves network performance, and has been widely used in many fields of deep learning.

[0033] In this example, the edge domain and core domain image enhancement reconstruction network is constructed to perform image enhancement, specifically: Constructing a mean absolute error loss function as a loss function in the image enhancement reconstruction network; Inputting the obtained edge domain low-resolution convolution feature map into a plurality of convolution blocks connected in series and introducing skip connections to obtain an edge domain reconstruction feature map; Inputting the edge domain reconstruction feature into the upsampling convolution layer to obtain an edge domain decoding enhanced image; Fusing several edge domain low-resolution convolution feature maps obtained to obtain an edge domain fused feature map; The obtained edge domain fusion feature map and the core domain low-resolution feature map are concatenated along the channel dimension to obtain the core domain reconstructed feature map.

[0034] In this example, the mean absolute error loss function is constructed as the loss function in the image enhancement reconstruction network, specifically:

[0035] in, , , They are the first edge domain decoding image enhancement loss, the core domain decoding image enhancement loss, and the second edge domain decoding image enhancement loss. and are the reconstructed features of the first edge domain and the i-th pixel of the decoded image of the first edge domain, respectively. and are the reconstructed feature map of the core domain and the i-th pixel of the decoded image of the core domain, respectively. and are the i-th pixel of the second edge domain reconstructed feature map and the second edge domain decoded image, respectively, n is the total number of pixels of the power equipment image, is the weight of the marginal domain loss function, is the weight of the core domain loss function.

[0036] It should be noted that the mean absolute error loss function provides a simple, direct and robust evaluation indicator by measuring the absolute error between the power equipment image and the target image. It can effectively maintain image details, is insensitive to noise and outliers, and provides a stable gradient during the training process, so that the model can be gradually optimized and improve the image enhancement effect.

[0037] In this example, the fused feature value and the convolution feature value are fused by channel weighted average to obtain a low-resolution convolution feature map, specifically:

[0038]

[0039]

[0040] in, is the convolutional feature map after passing through the i-th edge domain feature extraction network, Decode the image for the ith edge region, is the mapping function of the edge domain feature extraction network, yes and Linear combination of feature images, is the convolutional feature map after the core domain feature extraction network, Decode the image for the core domain, Mapping function for the core domain feature extraction network.

[0041] The obtained edge domain low-resolution convolution feature map is input into several convolution blocks connected in series and introducing skip connections to obtain an edge domain reconstruction feature map, specifically: The introduction of the jump connection is to reduce the number of network parameters and facilitate the propagation of the gradient direction. The specific formula is:

[0042] Where x is the input, To add the input x directly to the output after being processed by several layers, y is the output.

[0043] It should be noted that in deep neural networks, gradient vanishing and gradient exploding are common problems, which makes it difficult to train deep networks. Skip connections alleviate the problems of gradient vanishing and gradient exploding by providing an untransformed path so that gradients can be back-propagated more easily.

[0044] Embodiment 2, Figure 2 The present invention provides an image enhancement processing system based on deep learning, including a data acquisition module, a feature extraction module, and an image enhancement module: A data acquisition module acquires an image of the power equipment, encodes and decodes the image of the power equipment, and obtains a plurality of edge domains and a core domain decoded image; The feature extraction module extracts features from the edge domain and core domain decoded images through a preset feature extraction network respectively; The image enhancement module constructs edge domain and core domain image enhancement reconstruction networks for image enhancement.

[0045] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0046] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0047] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0048] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0049] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0050] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image enhancement processing method based on deep learning, characterized in that: The following steps are involved: Acquire an image of a power device, encode and decode the image of the power device, and obtain a plurality of edge domains and a core domain decoded image; The edge domain and core domain decoded images are respectively subjected to feature extraction through a preset feature extraction network; Construct edge domain and core domain image enhancement and reconstruction networks for image enhancement.

2. The image enhancement processing method based on deep learning according to claim 1, characterized in that: The power equipment image is encoded and decoded to obtain a number of edge domains and a core domain decoded image, specifically: Segment the power equipment image into an edge domain image and a core domain image; Encoding the edge domain image and the core domain image; The encoded edge domain and core domain images are decoded to obtain edge domain and core domain decoded images.

3. The image enhancement processing method based on deep learning according to claim 2, characterized in that: The power equipment image is segmented into an edge domain image and a core domain image, specifically: Convert the power equipment image into grayscale to obtain a grayscale image of the power equipment; Based on an edge detection algorithm, edge detection is performed on the grayscale image of the electric power equipment to obtain an edge detection result; Based on the edge detection result, the edge part and the core part in the grayscale image of the power equipment are extracted to form an edge domain image and a core domain image.

4. The image enhancement processing method based on deep learning according to claim 3 is characterized in that: The edge domain and core domain decoded images are respectively subjected to feature extraction through a preset feature extraction network, specifically: Extract features from several edge domain decoded images through a preset feature extraction network to obtain several edge domain low-resolution convolution feature maps; The core domain decoded image is passed through a preset feature extraction network for feature extraction to obtain a low-resolution convolutional feature map of the core domain.

5. The image enhancement processing method based on deep learning according to claim 4 is characterized in that: The edge domain and core domain decoded images are respectively subjected to feature extraction through a preset feature extraction network, specifically: Performing a convolution block operation on the edge domain and core domain decoded images to convert the decoded images into convolution feature values, wherein the convolution block operation includes convolution operation, batch normalization, and activation function calculation; Constructing a residual convolution block to extract features from the convolution eigenvalues ​​in turn, and performing channel weighted average fusion of the extracted eigenvalues ​​several times to obtain fused eigenvalues, wherein the residual convolution includes convolution operation, batch normalization, activation function calculation, and skip connection; The fused feature value and the convolution feature value are fused channel-wise by weighted averaging to obtain a low-resolution convolution feature map.

6. The image enhancement processing method based on deep learning according to claim 5, characterized in that: The residual convolution block is constructed to extract features from the convolution feature values ​​in turn, specifically: The preset feature extraction network includes several layers of convolution blocks, residual convolution blocks and convolution layers; Using the convolution feature value output by the convolution block of the current layer as the input of the residual convolution block and the convolution layer of the same layer; Linearly sum the output of the residual convolution block and the convolution layer of the same layer as the input of the convolution block of the next layer; The residual convolution block of the last layer and the output of the convolution layer are linearly summed as the low-resolution convolution feature map.

7. The image enhancement processing method based on deep learning according to claim 6, characterized in that: The edge domain and core domain image enhancement reconstruction network is constructed to perform image enhancement, specifically: Constructing a mean absolute error loss function as a loss function in the image enhancement reconstruction network; Inputting the obtained edge domain low-resolution convolution feature map into a plurality of convolution blocks connected in series and introducing skip connections to obtain an edge domain reconstruction feature map; Inputting the edge domain reconstruction feature into the upsampling convolution layer to obtain an edge domain decoding enhanced image; Fusing several edge domain low-resolution convolution feature maps obtained to obtain an edge domain fused feature map; The obtained edge domain fusion feature map and the core domain low-resolution feature map are concatenated along the channel dimension to obtain the core domain reconstructed feature map.

8. The image enhancement processing method based on deep learning according to claim 7, characterized in that: The fusion feature value and the convolution feature value are fused by channel weighted average to obtain a low-resolution convolution feature map, specifically: in, is the convolutional feature map after the i-th edge domain feature extraction network, Decode the image for the ith edge region, is the mapping function of the edge domain feature extraction network, yes and Linear combination of feature images, is the convolutional feature map after the core domain feature extraction network, Decode the image for the core domain, Mapping function for the core domain feature extraction network.

9. The image enhancement processing method based on deep learning according to claim 8, characterized in that: The mean absolute error loss function is constructed as the loss function in the image enhancement reconstruction network, and the specific formula is as follows: in, , , They are the first edge domain decoding image enhancement loss, the core domain decoding image enhancement loss, and the second edge domain decoding image enhancement loss. and are the reconstructed features of the first edge domain and the i-th pixel of the decoded image of the first edge domain, respectively. and are the reconstructed feature map of the core domain and the i-th pixel of the decoded image of the core domain, respectively. and are the i-th pixel of the second edge domain reconstructed feature map and the second edge domain decoded image, respectively, n is the total number of pixels of the power equipment image, is the weight of the marginal domain loss function, is the weight of the core domain loss function.

10. A system using the deep learning-based image enhancement processing method according to any one of claims 1 to 9, characterized in that: Including data acquisition module, feature extraction module, image enhancement module: The data acquisition module acquires the power equipment image, encodes and decodes the power equipment image, and obtains a plurality of edge domains and a core domain decoded image; The feature extraction module extracts features from the edge domain and core domain decoded images respectively through a preset feature extraction network; The image enhancement module constructs edge domain and core domain image enhancement reconstruction networks to perform image enhancement.

Images