Power distribution line image enhancement method and system based on scale and brightness flow correlation

Abstract

The present application relates to a kind of distribution line image enhancement method and system based on scale and brightness prevalence association, belong to image enhancement technical field, solve the problem of lack of image enhancement mode for distribution line target inspection in prior art.The method comprises: constructing high-resolution image training set, test set, and low-resolution image training set, test set, and construct scale space set of image high-low resolution combination accordingly;Scale enhancement network is trained using scale space set;Each low-resolution image in low-resolution image training set and each image block thereof are respectively input into scale enhancement network trained through, and scale enhancement high-resolution image and its image block are obtained;Brightness enhancement network is trained using each high-resolution image and its image block, and corresponding scale enhancement high-resolution image and its image block;Scale enhancement network and brightness enhancement network are sequentially connected to construct image comprehensive enhancement network, to realize image enhancement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image enhancement, in particular to a power distribution line image enhancement method and system based on scale and brightness flow correlation. BACKGROUND

[0002] Image enhancement refers to a technology that uses computers and artificial intelligence techniques to make fuzzy images clear or deepen their pixels. Image enhancement has always been an important research focus and has been widely used in many energy industries. For example, during power distribution inspection, it is difficult to inspect the circuit due to extreme weather (rain, snow, fog), at which time image enhancement can be used to assist circuit inspection.

[0003] Traditional image enhancement methods usually require manual intervention, which is time-consuming and labor-intensive, and prone to errors. Automatic image enhancement technology based on artificial intelligence algorithms can effectively solve this problem, improve production efficiency and reduce costs. Convolutional neural networks (CNN) are a common artificial intelligence algorithm that simulates the way neurons in the brain connect. Convolutional layers, as the core of convolutional neural networks, can extract features from input data and automatically classify and recognize. In automatic image enhancement technology, convolutional neural networks can directly process raw images without any preprocessing. By training the neural network, it can learn different types of fuzzy features and correctly classify and recognize different fuzzy types.

[0004] Although the existing image enhancement method based on convolutional neural network model has achieved good performance, due to the different structures of different types of convolutional neural networks, the image enhancement effect is not the same. The power distribution line image is affected by environmental factors such as shooting angle, viewing distance and light intensity, making it difficult to detect power distribution line abnormalities. How to design an effective enhancement method for power distribution lines to reduce the difficulty of power distribution line target inspection is a technical problem that needs to be solved in the field of power distribution line image enhancement. SUMMARY

[0005] In view of the above analysis, the embodiments of the present application aim to provide a power distribution line image enhancement method and system based on scale and brightness flow correlation to solve the problem of lack of image enhancement method for power distribution line target inspection in the prior art.

[0006] In one aspect, the present application discloses a power distribution line image enhancement method based on scale and brightness flow correlation, which comprises:

[0007] Collect original high-resolution power distribution line images under different power distribution line scenes and pre-process, use the obtained high-resolution images to construct high-resolution image training set and test set, use the low-resolution images obtained by gradually degrading the high-resolution images to construct low-resolution image training set and test set;

[0008] Block each image in the high-resolution image training set and the low-resolution image training set, respectively obtain the neighbor block of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set, respectively obtain the neighbor block of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set, and combine the high-resolution image and the low-resolution image in the training set, the image block and the corresponding neighbor block of each image block to obtain a scale space set of high-resolution and low-resolution combined images;

[0009] Train the scale enhancement network using the scale space set;

[0010] Input each low-resolution image and its image blocks in the low-resolution image training set into the scale enhancement network trained, to obtain the corresponding scale-enhanced high-resolution image and its image blocks; train the brightness enhancement network using the corresponding high-resolution image and its image blocks, and the corresponding scale-enhanced high-resolution image and its image blocks of each low-resolution image and its image blocks;

[0011] Connect the scale enhancement network and the brightness enhancement network in sequence to obtain an image comprehensive enhancement network; and use the obtained image comprehensive enhancement network to perform image enhancement on the real-time collected low-resolution power distribution line image.

[0012] On the basis of the above scheme, the application also makes the following improvements:

[0013] Further, the scale enhancement network is composed of a feature extraction sub-model and a super-resolution sub-model; in the process of training the scale enhancement network using the scale space set, first train the feature extraction sub-model by performing the following operations:

[0014] Train the feature extraction sub-model using the high-resolution images, the image blocks of the high-resolution images and their neighbor blocks in the scale space set, and the low-resolution images, the image blocks of the low-resolution images and their neighbor blocks, to obtain the trained feature extraction sub-model.

[0015] Further, the trained feature extraction sub-model performs:

[0016] The high-resolution image, its individual image patches, and their nearest neighbor patches are used as input to the feature extraction sub-model. The feature extraction sub-model extracts features separately, determines the error between the local features of each image patch in the high-resolution image and the local features of its nearest neighbor patches, and the error between the overall features of the high-resolution image and the overall features reconstructed from the local features of each image patch in the high-resolution image. The model parameters of the feature extraction sub-model are then trained.

[0017] Meanwhile, the low-resolution image, each image patch of the low-resolution image and its nearest neighbor patches are also used as input to the feature extraction sub-model. The feature extraction sub-model performs feature extraction separately, judges the error between the local features of each image patch of the low-resolution image and the local features of its nearest neighbor patches, and the error between the overall features of the low-resolution image and the overall features reconstructed from the local features of each image patch of the low-resolution image, and trains the model parameters of the feature extraction sub-model.

[0018] This yields a successfully trained feature extraction sub-model.

[0019] Furthermore, during the training of the scale-enhancing network using the scale-space set, the super-resolution sub-model is trained in the following manner:

[0020] The super-resolution sub-model is trained by taking a low-resolution image and its patches as input and the corresponding high-resolution image and its patches as the fitting output of the super-resolution sub-model. The model parameters of the super-resolution sub-model are trained by adjusting the error between the overall and local features of the high-resolution image and its patches obtained by the feature extraction sub-model and the overall and local features of the corresponding high-resolution image and its patches obtained by the feature extraction sub-model.

[0021] This results in a successfully trained super-resolution sub-model.

[0022] Furthermore, the loss function of scale-enhanced networks Represented as:

[0023] (1)

[0024] in, This represents the overall features of each high-resolution image and its patch within the high-resolution image training set. ; , Represents a high-resolution image training set. They represent The first to A collection of image patches; This represents a low-resolution training set. They represent The first to A set of image patches This represents the total number of image blocks into which each image is divided; This involves enhancing high-resolution images and their patches at various scales for each low-resolution image in the low-resolution training set through a super-resolution sub-model. Enhance the overall features of high-resolution images and their image patches at various scales; and The low- and high-resolution images in the image correspond one-to-one. It is Gaussian noise. These are the network parameters of the super-resolution sub-model. Indicates vector insertion loss calculation. , These represent the loss functions between the local and global aspects of high-resolution and low-resolution images, respectively.

[0025] Furthermore, the loss function between local and global aspects of high-resolution images. Represented as:

[0026] (2)

[0027] Loss function between local and global aspects of low-resolution images Represented as:

[0028] (3)

[0029] in, These represent high-resolution image test sets, These represent the low-resolution image test set; Represents the first high-resolution image in the training set. A high-resolution image, Represents the first image in the low-resolution image training set. Low-resolution image; This represents the total number of high-resolution images in the high-resolution image training set. This represents the total number of low-resolution images in the training set. Represents image blocks High-resolution image test set The first in One nearest neighbor block; Represents image blocks Low-resolution image test set The first in One nearest neighbor block; Indicates to From 1 to of Local features of an image patch the overall feature after local feature reconstruction; denotes the constraint weight, denotes the model parameter of the feature extraction submodel; the local feature of the image block; the overall feature after local feature reconstruction; denotes the constraint weight, denotes the model parameter of the feature extraction submodel; denotes the model parameter of the feature extraction submodel; denotes the model parameter of the feature extraction submodel; .

[0030] Further, the brightness enhancement network is trained in the following manner:

[0031] each low-resolution image in the low-resolution image training set and the image blocks thereof are respectively input into the super-resolution submodel in the scale enhancement network that has passed the training to obtain the corresponding scale enhancement high-resolution image and the image blocks thereof;

[0032] the pixel values of the scale enhancement high-resolution image and the image blocks thereof are respectively taken as the input of the brightness enhancement network, the pixel values of the high-resolution image and the image blocks thereof corresponding to the low-resolution image and the image blocks thereof are taken as the output fitted by the brightness enhancement network, and the brightness enhancement network is trained to obtain the brightness enhancement network that has passed the training.

[0033] Further, the loss function of the brightness enhancement network is denoted as:

[0034] (4)

[0035] wherein, is a constant, and the range is set to ; denotes the vector composed of the scale enhancement high-resolution image and the image blocks thereof corresponding to the i-th low-resolution image and the image blocks thereof in the low-resolution image training set, denotes the vector composed of the brightness enhancement high-resolution image and the image blocks thereof predicted and output by the brightness enhancement network, denotes the data distribution vector of the pixel values of denotes the data distribution vector of the pixel values of ; , ; ; , denote the Gaussian distribution of the pixel values of , ;denotes the threshold of the brightness manifold strong correlation.

[0036] ​Further, the constructed image comprehensive enhancement network is used to perform image enhancement on a real-time collected low-resolution power distribution line image, and execution is performed:

[0037] The real-time collected low-resolution power distribution line image is image blocked to obtain each image block of the low-resolution power distribution line image;

[0038] The low-resolution power distribution line image and each image block thereof are input into a super-resolution sub-model in the trained scale enhancement network to obtain corresponding scale-enhanced power distribution line images and image blocks thereof;

[0039] Pixel values of the scale-enhanced power distribution line images and image blocks thereof are input into the trained brightness enhancement network, and pixel values of brightness-enhanced power distribution line images and image blocks thereof are predicted and output by the brightness enhancement network;

[0040] According to the image blocking manner, the pixel values of the brightness-enhanced power distribution line images and image blocks thereof are weighted and fused to obtain an enhanced high-resolution power distribution line image.

[0041] On the other hand, the application also provides a power distribution line image enhancement system based on scale and brightness correlation, the system comprising:

[0042] An image acquisition and preprocessing module is configured to acquire original high-resolution power distribution line images under different power distribution line scenes and to pre-process the images, to construct a high-resolution image training set and a test set using the obtained high-resolution images, and to construct a low-resolution image training set and a test set using low-resolution images obtained by gradually degrading the high-resolution images;

[0043] A scale space set construction module is configured to obtain, for each high-resolution image in the high-resolution image training set and the low-resolution image training set, a neighbor block in the high-resolution image test set and the low-resolution image test set, and to combine the high-resolution image training set and the low-resolution image training set to construct a scale space set of the high-resolution image and the low-resolution image combination;

[0044] A model training module is configured to train a scale enhancement network using the scale space set, to input each low-resolution image in the low-resolution image training set and each image block thereof into the trained scale enhancement network to obtain corresponding scale-enhanced high-resolution images and image blocks thereof, to train a brightness enhancement network using the corresponding high-resolution images and image blocks thereof and the corresponding scale-enhanced high-resolution images and image blocks thereof, and to sequentially connect the trained scale enhancement network and the brightness enhancement network to construct an image comprehensive enhancement network;

[0045] An image enhancement module is configured to use the constructed image comprehensive enhancement network to perform image enhancement on a real-time collected low-resolution power distribution line image.

[0046] Compared with the prior art, the present application can achieve at least one of the following beneficial effects:

[0047] The power distribution line image enhancement method and system based on scale and brightness flow correlation provided by the present application have the following beneficial effects:

[0048] Firstly, the method fully considers that the power distribution line image is affected by environmental factors such as shooting angle, sight distance and illumination brightness, adopts an image enhancement method based on scale and brightness flow correlation, realizes image enhancement of the power distribution line image, and well solves the problem of lacking an image enhancement method for power distribution line target inspection in the prior art.

[0049] Secondly, on the basis of constructing a data set, the method respectively acquires the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set and the low-resolution image training set, combines the high-resolution image training set and the low-resolution image training set, and constructs a scale space set of high-resolution and low-resolution image combinations; the scale enhancement network is trained by using the scale space set; the scale enhancement network is composed of a feature extraction subnetwork and a super-resolution submodel, the feature extraction submodel capable of comprehensively considering the correlation degree of local and global feature extraction of high-resolution and low-resolution images is first trained. Then, the low-resolution and high-resolution image mapping conversion is performed by using the super-resolution submodel, and the scale enhancement effect of the scale enhancement high-resolution image output by the super-resolution submodel is verified by using the feature extraction subnetwork to ensure the training accuracy of the super-resolution submodel. The training method of the scale enhancement network proposed in the present application can effectively ensure the scale enhancement effect in the conversion process from low-resolution image to high-resolution image.

[0050] Thirdly, after the scale enhancement ends, the brightness enhancement is performed on the scale enhancement high-resolution image to weaken the influence of environmental factors such as illumination brightness on image enhancement as much as possible, so that a more accurate image enhancement effect is obtained.

[0051] Fourthly, the scale enhancement network and the brightness enhancement network that pass the training are sequentially connected to construct an image comprehensive enhancement network; then, the constructed image comprehensive enhancement network can be used to effectively perform image enhancement on the real-time collected low-resolution power distribution line image, fundamentally realizes the power distribution line image enhancement based on scale and brightness flow correlation, and achieves a good image enhancement effect, which is adapted to the requirements of scale and brightness enhancement of the power distribution line image.

[0052] The technical solutions in the present application can be combined with each other to realize more preferred combination solutions. Other features and advantages of the present application will be described in the following description, and some advantages will become apparent from the description, or will be understood by those skilled in the art through implementation of the present application. The objects and other advantages of the present application can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated herein and constitute a part of the detailed description. The same reference numbers in different drawings refer to the same elements throughout the drawings.

[0054] Figure 1 A flowchart of the power distribution line image enhancement method based on scale and brightness popular structure correlation provided for the specific embodiment 1 of the present application is shown in the figure.

[0055] Figure 2 A structural framework of the image comprehensive enhancement network based on scale and brightness popular structure correlation provided for the embodiment of the present application is shown in the figure.

[0056] Figure 3 A structural schematic diagram of the power distribution line image enhancement system based on scale and brightness popular correlation provided for the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0057] The preferred embodiments of the present application will be specifically described below in combination with the drawings, wherein the drawings constitute a part of the present application, and are used to explain the principles of the embodiments of the present application, and are not used to limit the scope of the present application.

[0058] In the actual power distribution line inspection application scenario, due to the existence of extreme weather (rain, fog, snow, etc.), the scale and brightness of the photographed image are seriously affected, which makes it difficult to obtain the target image that can effectively reflect the real scene in the power distribution line inspection process. In the actual power distribution line inspection process, if the influence of scale and brightness on the power distribution line image is not considered, the detection and recognition accuracy of the specific target in the power distribution line image will be seriously affected. Therefore, in order to solve such problems, the present embodiment provides an image enhancement method and system based on scale and brightness popular correlation. The specific description is as follows.

[0059] In a specific embodiment 1 of the present application, a power distribution line image enhancement method based on scale and brightness popular correlation is disclosed, and a flowchart of the method is shown in the figure. Figure 1 A structural framework of the image comprehensive enhancement network based on scale and brightness popular structure correlation is shown in the figure. Figure 2 The specific description is as follows.

[0060] Step S1: Collect original high-resolution power distribution line images under different power distribution line scenes and pre-process, use the obtained high-resolution images to construct high-resolution image training set and test set, use the low-resolution images obtained by gradually degrading the high-resolution images to construct low-resolution image training set and test set.

[0061] In this embodiment, the original high-resolution power distribution line images under different power distribution line scenes can be obtained by using a high-resolution camera. The original high-resolution power distribution line images obtained by shooting cover normal power distribution line images under different power distribution line scenes, and power distribution line images of different abnormal types, such as cracks, protrusions, and damages. Then, the collected original high-resolution power distribution line images are pre-processed, including removing outliers and noise, random cropping, scaling, enhancement, and twisting, etc. operations, so as to obtain corresponding high-resolution images, so as to enhance the diversity of the images, expand the sample set, and enhance the robustness of the images and the generalization ability of the model. Then, the high-resolution images obtained by pre-processing can be divided into high-resolution image training set and test set.

[0062] It should be noted that each high-resolution image in the high-resolution image training set is gradually degraded (for example, the resolution of the image can be gradually reduced by down-sampling), to obtain corresponding low-resolution images, so as to construct a low-resolution image training set. Therefore, the high-resolution images in the high-resolution image training set correspond one-to-one to the low-resolution images in the low-resolution image training set.

[0063] Step S2: Block each image in the high-resolution image training set and the low-resolution image training set and test set; respectively obtain the neighbor block of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set, respectively obtain the neighbor block of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set, and combine the high-resolution image and its image block, the neighbor block corresponding to each image block in the training set to obtain a scale space set of image high-low resolution combination;

[0064] In step S2, the neighbor block is determined by the following method.

[0065] Step S21: The images in the high-resolution image training set and test set, and the low-resolution image training set and test set are respectively divided into blocks according to the same division method, and the labels of the image blocks with the same position in different images are the same.

[0066] Step S22: For each image patch of a high-resolution image in the high-resolution image training set, obtain the K nearest neighbor patches from the image patches with the same label in the high-resolution image test set; for each image patch of a low-resolution image in the low-resolution image training set, obtain the K nearest neighbor patches from the image patches with the same label in the low-resolution image test set.

[0067] Step S23: Construct a scale space set of high and low resolution images by combining each high-resolution image in the high-resolution image training set, each image patch of the high-resolution image and its nearest neighbor, and each low-resolution image in the low-resolution image training set, each image patch of the low-resolution image and its nearest neighbor.

[0068] Specifically, in this embodiment, it is assumed that the high-resolution image training set... ,in, Represents the first high-resolution image in the training set. A high-resolution image, This represents the total number of high-resolution images in the high-resolution image training set. High-resolution image test set. , Represents the first high-resolution image in the test set. A high-resolution image, This represents the total number of high-resolution images in the high-resolution image test set. Additionally, it is assumed that the low-resolution image training set... ,in, Represents the first image in the low-resolution image training set. A low-resolution image, This represents the total number of low-resolution images in the training set. Low-resolution image test set. ,in, Represents the first image in the low-resolution image test set. A low-resolution image, This represents the total number of low-resolution images in the low-resolution image test set. The images in both the high-resolution image training and test sets, and the low-resolution image training and test sets, are uniformly divided using the same partitioning method. Image blocks located at the same position in different images have the same label; the image block label is set to... .

[0069] At this time, the high-resolution image training set Chinese label Image patch collection High-resolution image test set Chinese label Image patch collection Low-resolution image training set Chinese label Image patch collection Low-resolution image test set Chinese label Image patch collection .

[0070] Then, following step S22, for each image patch in the high-resolution image training set, the K nearest neighbor patches with the same label are obtained from the image patches with the same label in the high-resolution image test set; for each image patch in the low-resolution image training set, the K nearest neighbor patches with the same label are obtained from the image patches with the same label in the low-resolution image test set, thereby realizing the distribution of manifold associations in the scale space of high and low resolution images. Specifically, for the high-resolution image training set... Chinese label Image blocks Image patches are calculated using Euclidean distance. With high-resolution image test set Chinese label The distances between image patches in the image patch set are used to obtain the K nearest neighbor patches, which are labeled as... , Represents image blocks High-resolution image test set The first in Nearest neighbor blocks. For low-resolution image training sets. Chinese label Image blocks Image patches are calculated using Euclidean distance. With low-resolution image test set Chinese label The distances between image patches in the image patch set are used to obtain the K nearest neighbor patches, which are labeled as... , Represents image blocks Low-resolution image test set The first in The nearest neighbor blocks. Traverse. From 1 to ,Establish In and The relationship between them; at the same time, establish In and The relationships between the images are determined, and high- and low-resolution image training sets are combined to construct a scale space set of combined high- and low-resolution images.

[0071] Step S3: Train the scale augmentation network using the scale space set.

[0072] The scale enhancement network is composed of a feature extraction submodel and a super-resolution submodel. Specifically, in the present embodiment, the scale enhancement network is trained in the following manner.

[0073] Step S31: training the feature extraction submodel using the high-resolution images in the scale space set, the image blocks of the high-resolution images and their neighboring blocks, and the low-resolution images, the image blocks of the low-resolution images and their neighboring blocks, to obtain a feature extraction submodel that passes the training.

[0074] The specific process is described as follows: taking the high-resolution images, the image blocks of the high-resolution images and their neighboring blocks as inputs of the feature extraction submodel, performing feature extraction by the feature extraction submodel respectively, judging the errors between the local features of the image blocks of the high-resolution images and the local features of their neighboring blocks, and the errors between the overall features of the high-resolution images and the overall features reconstructed from the local features of the image blocks of the high-resolution images, and training the model parameters of the feature extraction submodel; at the same time, taking the low-resolution images, the image blocks of the low-resolution images and their neighboring blocks as inputs of the feature extraction submodel, performing feature extraction by the feature extraction submodel respectively, judging the errors between the local features of the image blocks of the low-resolution images and the local features of their neighboring blocks, and the errors between the overall features of the low-resolution images and the overall features reconstructed from the local features of the image blocks of the low-resolution images, and training the model parameters of the feature extraction submodel; obtaining a feature extraction submodel that passes the training.

[0075] It should be noted that in the present embodiment, the feature extraction submodel mainly considers the correlation between the local and overall feature extraction of the high-resolution and low-resolution images; preferably, the feature extraction submodel can be implemented based on a Resnet50 or a Transformer network.

[0076] In the feature extraction submodel of the present embodiment, the loss function between the local and overall features of the high-resolution images is represented as:

[0077] (1)

[0078] The loss function between the local and overall features of the low-resolution images is represented as:

[0079] (2)

[0080] wherein, represents the overall feature reconstructed from the local features of the image blocks from 1 to ​​​​denotes the constraint weight, from 1 to the local features of the image blocks the overall features after local feature reconstruction; denotes the constraint weight, denote the model parameters of the feature extraction sub-model; denote the feature vectors obtained after the feature extraction sub-model processes the input information; denote the high and low .

[0081] Step S32: Take the low-resolution image and its image blocks as the input of the super-resolution sub-model, take the corresponding high-resolution image and its image blocks as the output of the fitting of the super-resolution sub-model, input the scale high-resolution image and its image blocks predicted and output by the super-resolution sub-model into the feature extraction sub-model to obtain the overall features and local features, input the high-resolution image and its image blocks into the feature extraction sub-model to obtain the overall features and local features, calculate the error between the overall features and local features corresponding to the scale high-resolution image and its image blocks and the overall features and local features corresponding to the high-resolution image and its image blocks, and adjust the model parameters of the super-resolution sub-model according to the error; obtain the super-resolution sub-model that passes the training.

[0082] It should be noted that in this embodiment, the super-resolution sub-model mainly considers the mapping relationship in the conversion process from the low-resolution image to the high-resolution image. The super-resolution sub-model can be implemented by using a high-low resolution image reconstruction model (Maximum-Entropy Capital Asset Pricing Model, MECAPM), that is, a commonly used super-resolution network structure, to super-resolve the low-resolution image into a high-resolution image, and calculate the loss between the features and the original high-resolution image to measure the performance of the super-resolution reconstruction.

[0083] Specifically, in this embodiment, the overall loss function of the scale enhancement network can be calculated in the following manner, that is, the loss function of the scale enhancement network is expressed as:

[0084] (3)

[0085] wherein, is the overall feature of each high-resolution image and its image blocks in the high-resolution image training set (that is, the feature vector output after the high-resolution image and its image blocks are processed by the feature extraction sub-model); ; , denote the high-resolution image training set, denote the high and low a set of 1st to image blocks; represent low resolution training set, respectively represent a set of 1st to image blocks; each scale enhanced high resolution image and its image blocks output by the super-resolution sub-model for each low resolution image in the low resolution training set, each overall feature of the scale enhanced high resolution image and its image blocks (i.e. the feature vector output by the feature extraction sub-model after processing the scale enhanced high resolution image and its image blocks); correspond to the low and high resolution images in one-to-one; is Gaussian noise, is the network parameter of the super-resolution sub-model, represents vector interpolation loss calculation, , respectively represent the loss function between the local and overall of the high and low resolution images.

[0086] Step S4: input each low resolution image in the low resolution image training set and its image blocks into the scale enhancement network trained to pass, to obtain the corresponding scale enhanced high resolution image and its image blocks; train the brightness enhancement network using the high resolution image and its image blocks corresponding to each low resolution image and its image blocks, and the corresponding scale enhanced high resolution image and its image blocks.

[0087] Step S41: input each low resolution image in the low resolution image training set and its image blocks into the super-resolution sub-model in the scale enhancement network trained to pass, to obtain the corresponding scale enhanced high resolution image and its image blocks.

[0088] At this time, after the scale enhancement network is trained, the feature extraction sub-network no longer functions.

[0089] Step S42: take the pixel values of the scale enhanced high resolution image and its image blocks as the input of the brightness enhancement network, take the pixel values of the high resolution image and its image blocks corresponding to the low resolution image and its image blocks as the output fitted by the brightness enhancement network, train the brightness enhancement network, and obtain the brightness enhancement network trained to pass.

[0090] Preferably, in the present embodiment, the loss function of the brightness enhancement network is represented as:

[0091] (4)

[0092] wherein, is a constant, setting the range as ; represents the vector composed of the th low-resolution image in the low-resolution image training set and its image blocks and the scale-enhanced high-resolution image corresponding to the image blocks, represents the vector composed of the brightness-enhanced high-resolution image predicted by the brightness enhancement network and its image blocks; represents the data distribution vector of the pixel value of ; represents the data distribution vector of the pixel value of ; , ; , respectively represent the Gaussian distribution of the pixel value of , ; represents the threshold of the strong correlation of the scale and brightness manifold.

[0093] Step S5: sequentially connecting the scale enhancement network and the brightness enhancement network passed by the training to construct an image comprehensive enhancement network; and using the constructed image comprehensive enhancement network to perform image enhancement on the low-resolution power distribution line image collected in real time.

[0094] Preferably, in the embodiment, the sequentially connected scale enhancement network and brightness enhancement network constitute the image comprehensive enhancement network in the embodiment, and the loss function of the image comprehensive enhancement network is represented as:

[0095] (5)

[0096] wherein is used to balance the effects of the scale enhancement network and the brightness enhancement network. By calculating , the entire image enhancement of the scale and brightness manifold correlation is completed.

[0097] In step S5, the following operations are specifically performed.

[0098] Step S51: performing image blocking on the low-resolution power distribution line image collected in real time to obtain each image block of the low-resolution power distribution line image. The process of image blocking is described in step S21.

[0099] Step S52: inputting the low-resolution power distribution line image and each image block thereof into the super-resolution sub-model in the scale enhancement network passed by the training to obtain the corresponding scale-enhanced power distribution line image and its image blocks.

[0100] Step S53: input the pixel values of the scale-enhanced power distribution line image and its image blocks into the brightness enhancement network trained and passed, and predict and output the pixel values of the brightness-enhanced power distribution line image and its image blocks by the brightness enhancement network.

[0101] Step S54: weight and fuse the pixel values of the brightness-enhanced power distribution line image and its image blocks in a manner of image blocking to obtain the enhanced high-resolution power distribution line image.

[0102] Specifically, the pixel values of each image block predicted and output by the brightness enhancement network are spliced in a manner of image blocking to reconstruct a reference brightness-enhanced image; since the reconstructed reference brightness-enhanced image is equivalent to a complete high-resolution image and has similar enhancement characteristics to the brightness-enhanced power distribution line image, the reconstructed reference brightness-enhanced image and the brightness-enhanced power distribution line image can be weight and fused in terms of pixel values at corresponding pixel positions, so as to obtain the enhanced high-resolution power distribution line image.

[0103] In the specific implementation process, considering that the brightness-enhanced power distribution line image directly predicted and output by the brightness enhancement network is a direct and complete image, a higher weight can be considered for the brightness-enhanced power distribution line image, and a lower weight can be considered for the reconstructed reference brightness-enhanced image as an auxiliary reference. Exemplarily, the reconstructed reference brightness-enhanced image and the brightness-enhanced power distribution line image can be weight and fused in terms of pixel values at corresponding pixel positions in a ratio of 3:7 or 2:8.

[0104] Specific embodiment 2 of the present application also discloses a power distribution line image enhancement system based on scale and brightness flow correlation, a structural schematic diagram of which is shown in Figure 3 The system comprises an image acquisition and preprocessing module, a scale space set construction module, a model training module and an image enhancement module; wherein:

[0105] The image acquisition and preprocessing module is used for acquiring original high-resolution power distribution line images under different power distribution line scenes and pre-processing, constructing a high-resolution image training set and a test set by using the obtained high-resolution images, and constructing a low-resolution image training set and a test set by using low-resolution images obtained by gradually degrading the high-resolution images.

[0106] The scale space set construction module is used for acquiring the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set and the low-resolution image test set, respectively, and combining the high-resolution image training set and the low-resolution image training set to construct a scale space set of high-resolution and low-resolution combined images.

[0107] The model training module is configured to train the scale enhancement network using the scale space set; input each low-resolution image in the low-resolution image training set and each image block thereof into the scale enhancement network that has passed the training, to obtain a corresponding scale-enhanced high-resolution image and image blocks thereof; train the brightness enhancement network using the high-resolution image and image blocks thereof corresponding to each low-resolution image and image blocks thereof, and the corresponding scale-enhanced high-resolution image and image blocks thereof; and sequentially connect the scale enhancement network and the brightness enhancement network that have passed the training, to obtain the image comprehensive enhancement network.

[0108] The image enhancement module is configured to perform image enhancement on the low-resolution power distribution line image collected in real time, using the obtained image comprehensive enhancement network.

[0109] The specific implementation process of the system embodiment of the present application can be referred to the above method embodiment, which will not be described here again. Since the principle of the present embodiment is the same as that of the above method embodiment, the present system also has the corresponding technical effects of the above method embodiment.

[0110] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. The computer readable storage medium includes a magnetic disk, an optical disk, a read-only memory, a random access memory, etc.

[0111] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A power distribution line image enhancement method based on scale and brightness flow correlation, characterized in that, The method comprises: Collecting original high-resolution power distribution line images under different power distribution line scenes and preprocessing, using the obtained high-resolution images to construct a high-resolution image training set and a test set, using low-resolution images obtained by gradually degrading the high-resolution images to construct a low-resolution image training set and a test set; Block each image in the high-resolution image training set and the low-resolution image training set; obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set, and obtain the neighboring blocks of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set; combine the high-resolution images and the low-resolution images in the training set, the image blocks, and the corresponding neighboring blocks of each image block to obtain a scale space set of high-resolution and low-resolution combined images; Train the scale enhancement network using the scale space set; Input each low-resolution image and its image blocks in the low-resolution image training set into the scale enhancement network trained, to obtain the corresponding scale-enhanced high-resolution images and their image blocks; train the brightness enhancement network using the corresponding high-resolution images and their image blocks, and the corresponding scale-enhanced high-resolution images and their image blocks of each low-resolution image and its image blocks; Connect the scale enhancement network and the brightness enhancement network in sequence to construct an image comprehensive enhancement network; and use the constructed image comprehensive enhancement network to perform image enhancement on the real-time collected low-resolution power distribution line images; Train the brightness enhancement network in the following manner: Input each low-resolution image and its image blocks in the low-resolution image training set into the super-resolution sub-model in the scale enhancement network trained, to obtain the corresponding scale-enhanced high-resolution images and their image blocks; Use the pixel values of the scale-enhanced high-resolution images and their image blocks as the input of the brightness enhancement network, use the pixel values of the corresponding high-resolution images and their image blocks as the output fitted by the brightness enhancement network, train the brightness enhancement network, and obtain the brightness enhancement network trained; Use the constructed image comprehensive enhancement network to perform image enhancement on the real-time collected low-resolution power distribution line images, and perform: Block the real-time collected low-resolution power distribution line images to obtain image blocks of the low-resolution power distribution line images; Input the low-resolution power distribution line images and their image blocks into the super-resolution sub-model in the scale enhancement network trained, to obtain the corresponding scale-enhanced power distribution line images and their image blocks; Input the pixel values of the scale-enhanced power distribution line images and their image blocks into the brightness enhancement network trained, and use the brightness enhancement network to predict and output the pixel values of the brightness-enhanced power distribution line images and their image blocks; According to the image blocking manner, weight fuse the pixel values of the brightness-enhanced power distribution line images and their image blocks to obtain the enhanced high-resolution power distribution line images.

2. The power line image enhancement method based on scale and intensity flow association according to claim 1, characterized in that, The scale enhancement network is composed of a feature extraction sub-model and a super-resolution sub-model; in the process of training the scale enhancement network by using the scale space set, the feature extraction sub-model is first trained by performing the following operations: The feature extraction sub-model is trained by using the high-resolution images, the image blocks and the neighboring blocks of the high-resolution images in the scale space set, and the low-resolution images, the image blocks and the neighboring blocks of the low-resolution images, to obtain a trained feature extraction sub-model.

3. The power line image enhancement method based on scale and intensity flow correlation of claim 2, wherein, The trained feature extraction sub-model is executed by performing the following operations: The high-resolution images, the image blocks and the neighboring blocks of the high-resolution images are taken as inputs of the feature extraction sub-model, and the feature extraction sub-model is used to perform feature extraction on the inputs, respectively, to judge the errors between the local features of the image blocks of the high-resolution images and the local features of the neighboring blocks, and the errors between the overall features of the high-resolution images and the overall features reconstructed from the local features of the image blocks of the high-resolution images, and to train the model parameters of the feature extraction sub-model; Meanwhile, the low-resolution images, the image blocks and the neighboring blocks of the low-resolution images are also taken as inputs of the feature extraction sub-model, and the feature extraction sub-model is used to perform feature extraction on the inputs, respectively, to judge the errors between the local features of the image blocks of the low-resolution images and the local features of the neighboring blocks, and the errors between the overall features of the low-resolution images and the overall features reconstructed from the local features of the image blocks of the low-resolution images, and to train the model parameters of the feature extraction sub-model; Thus, the trained feature extraction sub-model is obtained.

4. The power line image enhancement method based on scale and intensity flow association according to claim 3, characterized in that, In the process of training the scale enhancement network by using the scale space set, the super-resolution sub-model is trained by performing the following operations: The low-resolution images and the image blocks thereof are taken as inputs of the super-resolution sub-model, and the corresponding high-resolution images and the image blocks thereof are taken as fitting outputs of the super-resolution sub-model, and the model parameters of the super-resolution sub-model are trained by adjusting the errors between the overall features and the local features of the high-resolution images and the image blocks thereof obtained by processing the scale high-resolution images and the image blocks thereof predicted by the super-resolution sub-model via the feature extraction sub-model, and the overall features and the local features of the high-resolution images and the image blocks thereof obtained by processing the high-resolution images and the image blocks thereof via the feature extraction sub-model; Thus, the trained super-resolution sub-model is obtained.

5. The power line image enhancement method based on scale and brightness flow correlation of claim 4, wherein, Loss function of a scale enhancement network is represented as: (1) wherein, is the overall feature of each high-resolution image and its image patches in the high-resolution image training set, ; , denotes the high-resolution image training set, denotes the set of image patches of the 1stto thimage in the high-resolution image training set, respectively; denotes the low-resolution training set, denotes the set of image patches of the 1stto thimage in the low-resolution training set, respectively, denotes the total number of image patches divided for each image; is each scale-up high-resolution image and its image patches output by the super-resolution sub-model for each low-resolution image in the low-resolution training set, is the overall feature of each scale-up high-resolution image and its image patches; corresponds to each low-resolution image and each high-resolution image in one-to-one; is the Gaussian noise, is the network parameter of the super-resolution sub-model, denotes the vector insertion loss calculation, , denote the loss function between the local and overall of the high-resolution image and the low-resolution image, respectively.

6. The power line image enhancement method based on scale and brightness flow correlation of claim 5, wherein, Loss function between local and global of high resolution image is represented as: (2) Loss function between low resolution image local and global is represented as: (3) in, These represent high-resolution image test sets, These represent the low-resolution image test set; Represents the first high-resolution image in the training set. A high-resolution image, Represents the first image in the low-resolution image training set. Low-resolution image; This represents the total number of high-resolution images in the high-resolution image training set. This represents the total number of low-resolution images in the training set. Represents image blocks High-resolution image test set The first in One nearest neighbor block; Represents image blocks Low-resolution image test set The first in One nearest neighbor block; Indicates to From 1 to of Local features of an image patch The overall features after reconstructing local features; Indicates to From 1 to of Local features of an image patch The overall features after reconstructing local features; Indicates the constraint weight. Represents the model parameters of the feature extraction sub-model; This represents the feature vector obtained after the feature extraction sub-model processes the input information; These represent the overall features of high-resolution and low-resolution images, respectively.

7. The power line image enhancement method based on scale and intensity flow correlation of claim 6, wherein, Loss function of a brightness enhancement network is expressed as: (4) wherein, is a constant, set in the range ; represents the i-th low-resolution image in the low-resolution image training set and its image patches, represents the i-th low-resolution image in the low-resolution image training set and its image patches, represents the data distribution vector of pixel values of represents the data distribution vector of pixel values of , ; , represents the Gaussian distribution of pixel values of , represents the threshold of the strong correlation of the brightness manifold.​​​​ 8. A power distribution line image enhancement system based on scale and brightness flow correlation, characterized by, The system is implemented based on the power line image enhancement method based on scale and brightness flow correlation according to any one of claims 1-7, and the system comprises: An image acquisition and preprocessing module, which is configured to acquire original high-resolution power line images under different power line scenes and pre-process the original high-resolution power line images, construct a high-resolution image training set and a high-resolution image test set by using the obtained high-resolution images, and construct a low-resolution image training set and a low-resolution image test set by using low-resolution images obtained by gradually degrading the high-resolution images; A scale space set construction module, which is configured to obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set and the low-resolution image training set, and combine the high-resolution image training set and the low-resolution image training set to construct a scale space set of the high-resolution images and the low-resolution images; The model training module is configured to train the scale enhancement network by using the scale space set; input each low-resolution image in the low-resolution image training set and each image block thereof into the scale enhancement network trained to pass, to obtain a corresponding scale-enhanced high-resolution image and image blocks thereof; train the brightness enhancement network by using the high-resolution image and image blocks thereof corresponding to each low-resolution image and image blocks thereof, and the corresponding scale-enhanced high-resolution image and image blocks thereof; and sequentially connect the scale enhancement network trained to pass and the brightness enhancement network to obtain an image comprehensive enhancement network. The image enhancement module is configured to perform image enhancement on a low-resolution power distribution line image collected in real time by using the image comprehensive enhancement network obtained.

Images