Distribution line image enhancement method and system based on scale and brightness manifold association

Abstract

The invention relates to a distribution line image enhancement method and system based on scale and brightness popularity association, belongs to the technical field of image enhancement, and solves the problem of lack of an image enhancement mode for distribution line target inspection in the prior art. The method comprises the following steps: constructing a high-resolution image training set, a high-resolution image test set, a low-resolution image training set and a low-resolution image test set, and constructing a scale space set of image high-resolution and low-resolution combination; training a scale enhancement network by using the scale space set; respectively inputting each low-resolution image in the low-resolution image training set and each image block thereof into the trained scale enhancement network to obtain a scale-enhanced high-resolution image and an image block thereof; training a brightness enhancement network by using the high-resolution images and the image blocks thereof, and the corresponding scale-enhanced high-resolution images and the image blocks thereof; and sequentially connecting the scale enhancement network and the brightness enhancement network to construct an image comprehensive enhancement network so as to realize image enhancement.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a distribution line image enhancement method and system based on scale and brightness popular correlation. Background Art

[0002] Image enhancement refers to a technology that uses computers and artificial intelligence technology to make blurry images clear or deepen their pixels. Image enhancement has always been an important research hotspot and has been widely used in many energy industries. For example, during the power distribution inspection process in the power industry, circuit inspection is difficult due to extreme weather (rain, snow, fog). At this time, image enhancement can be used to assist circuit inspection.

[0003] Traditional image enhancement methods usually require manual work, which is time-consuming and labor-intensive, and prone to errors. Automated image enhancement technology based on artificial intelligence algorithms can effectively solve this problem, improve production efficiency and reduce costs. As a common artificial intelligence algorithm, the working method of convolutional neural network (CNN) imitates the connection between neurons in the brain. As the core of convolutional neural network, the convolutional layer can extract features from input data and automatically realize classification and recognition. In automated image enhancement technology, convolutional neural network can directly process the original image without any preprocessing. By training the neural network, it can learn different types of fuzzy features and correctly classify and identify different types of fuzzy.

[0004] Although the existing image enhancement methods based on convolutional neural network models have achieved good performance, the image enhancement effects vary due to the different structures of different types of convolutional neural networks. The distribution line images are affected by environmental factors such as shooting angle, viewing distance and light brightness, which makes it difficult to detect distribution line anomalies. How to design an effective enhancement method for distribution lines and reduce the difficulty of distribution line target inspection is a technical problem that needs to be solved in the field of distribution line image enhancement. Summary of the invention

[0005] In view of the above analysis, the embodiments of the present invention aim to provide a distribution line image enhancement method and system based on scale and brightness popular correlation, so as to solve the problem of lack of image enhancement methods for distribution line target inspection in the prior art.

[0006] On the one hand, the present invention discloses a distribution line image enhancement method based on scale and brightness popular correlation, the method comprising:

[0007] The original high-resolution distribution line images in different distribution line scenarios are collected and preprocessed, and the obtained high-resolution images are used to construct high-resolution image training sets and test sets, and the low-resolution images after the high-resolution images are gradually degraded are used to construct low-resolution image training sets and test sets;

[0008] Each image in the high-resolution and low-resolution image training sets and test sets is divided into blocks; the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set are obtained respectively, and the neighboring blocks of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set are obtained respectively, and the high-resolution and low-resolution images and their image blocks and the neighboring blocks corresponding to each image block in the training set are combined to obtain a scale space set of high-resolution and low-resolution image combinations;

[0009] Scale-augmented networks are trained using scale-space sets;

[0010] Input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and image block; train the brightness enhancement network using the high-resolution image and image block corresponding to each low-resolution image and image block, and the corresponding scale-enhanced high-resolution image and image block;

[0011] The trained scale enhancement network and brightness enhancement network are connected sequentially to construct an image comprehensive enhancement network; and the constructed image comprehensive enhancement network is used to perform image enhancement on the low-resolution distribution line images collected in real time.

[0012] On the basis of the above scheme, the present invention also makes the following improvements:

[0013] Furthermore, the scale enhancement network is composed of a feature extraction sub-model and a super-resolution sub-model; in the process of training the scale enhancement network using the scale space set, the feature extraction sub-model is first trained by performing the following operations:

[0014] The feature extraction sub-model is trained using the high-resolution image, each image block of the high-resolution image and its neighboring blocks, as well as the low-resolution image, each image block of the low-resolution image and its neighboring blocks in the scale space concentration to obtain a trained feature extraction sub-model.

[0015] Further, the training feature extraction sub-model executes:

[0016] The high-resolution image, each image block of the high-resolution image and its neighboring blocks are used as inputs of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, determines the error between the local features of each image block of the high-resolution image and the local features of its neighboring blocks, and the error between the overall features of the high-resolution image and the overall features reconstructed from the local features of each image block of the high-resolution image, and trains the model parameters of the feature extraction sub-model;

[0017] At the same time, the low-resolution image, each image block of the low-resolution image and its neighboring blocks are used as inputs of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, judges the error between the local features of each image block of the low-resolution image and the local features of its neighboring blocks, and the error between the overall features of the low-resolution image and the overall features reconstructed from the local features of each image block of the low-resolution image, and trains the model parameters of the feature extraction sub-model;

[0018] Thus, a trained feature extraction sub-model is obtained.

[0019] Furthermore, in the process of training the scale-enhanced network using the scale-space set, the super-resolution sub-model is trained in the following way:

[0020] The low-resolution image and its image blocks are used as the input of the super-resolution sub-model, and the corresponding high-resolution image and its image blocks are used as the output of the fitting of the super-resolution sub-model. The model parameters of the super-resolution sub-model are trained by adjusting the error between the overall features and local features of the scale high-resolution image and its image blocks processed by the feature extraction sub-model and the overall features and local features of the corresponding high-resolution image and its image blocks processed by the feature extraction sub-model as output of the super-resolution sub-model prediction;

[0021] Thus, a trained super-resolution sub-model is obtained.

[0022] Furthermore, the loss function L of the scale-enhanced network is (X→D) It is expressed as:

[0023]

[0024] in, is the feature vector of each high-resolution image and its image patch in the high-resolution image training set, I H represents the high-resolution image training set, I H The set of image blocks from 1 to M in I L represents the low-resolution training set, I LThe set of image blocks 1 to M in , where M represents the total number of image blocks divided into each image; For each low-resolution image in the low-resolution training set, the high-resolution images and their image patches at each scale are enhanced by the super-resolution sub-model. Enhance the feature vectors of high-resolution images and their image patches at each scale; and The low and high resolution images in the image correspond to each other; s is Gaussian noise, θ * are the network parameters of the super-resolution sub-model, Represents vector insertion loss calculation, L c , L d Represent the loss functions between the local and the whole of high and low resolution images respectively.

[0025] Furthermore, the loss function L between the local and the whole of the high-resolution image c It is expressed as:

[0026]

[0027] The loss function L between the local and the whole of the low-resolution image d It is expressed as:

[0028]

[0029]

[0030] Among them, I X Respectively represent the high-resolution image test set, I T Represent the low-resolution image test set; I h represents the hth high-resolution image in the high-resolution image training set, I l represents the lth low-resolution image in the low-resolution image training set; P represents the total number of high-resolution images in the high-resolution image training set, and N represents the total number of low-resolution images in the low-resolution image training set; Represents an image block On the high-resolution image test set I X The kth nearest neighbor block in ; Represents an image block On the low-resolution image test set I T The kth nearest neighbor block in ; Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; λ represents the constraint weight, θ represents the model parameters of the feature extraction sub-model; f(.,θ) represents the feature vector obtained after the feature extraction sub-model processes the input information; f(I h ,θ)、f(I l ,θ) represent the overall characteristics of high and low resolution images respectively.

[0031] Furthermore, the brightness enhancement network is trained as follows:

[0032] Each low-resolution image and each image block in the low-resolution image training set are input into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and its image block;

[0033] The pixel values ​​of the scale-enhanced high-resolution image and its image blocks are used as the input of the brightness enhancement network respectively, and the pixel values ​​of the high-resolution image and its image blocks corresponding to the low-resolution image and its image blocks are used as the output of the brightness enhancement network fitting. The brightness enhancement network is trained to obtain a trained brightness enhancement network.

[0034] Furthermore, the loss function l of the brightness enhancement network bp It is expressed as:

[0035] l bp =∑ l |α-I e |(f b (y(l))-f b (y′(l))) (4)

[0036] Wherein, α is a constant, and the setting range is 0<α<0.3; y(l) represents the vector composed of the scale-enhanced high-resolution image and its image blocks corresponding to the l-th low-resolution image and its image blocks in the low-resolution image training set, and y′(l) represents the vector composed of the brightness-enhanced high-resolution image and its image blocks predicted and output by the scale brightness enhancement network; f b (y(l)) represents the data distribution vector of the pixel value of y(l), f b (y′(l)) represents the data distribution vector of the pixel value of y′(l); e=G(y(l))-G(y′(l))-H b ; G(y(l)) and G(y′(l)) represent the Gaussian distribution of the pixel values ​​of y(l) and y′(l), respectively, and H b Indicates the threshold for strong correlation of the brightness manifold.

[0037] Furthermore, the constructed image comprehensive enhancement network is used to enhance the low-resolution distribution line images collected in real time, and the following operations are performed:

[0038] Dividing the low-resolution distribution line image collected in real time into blocks to obtain image blocks of the low-resolution distribution line image;

[0039] Inputting the low-resolution distribution line image and each image block thereof into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale enhanced distribution line image and image blocks thereof;

[0040] Inputting the pixel values ​​of the scale-enhanced distribution line image and its image blocks into the trained brightness enhancement network, the brightness enhancement network predicts and outputs the pixel values ​​of the brightness-enhanced distribution line image and its image blocks;

[0041] According to the image block method, the pixel values ​​of the brightness enhanced distribution line image and its image blocks are weighted fused to obtain an enhanced high-resolution distribution line image.

[0042] On the other hand, the present invention also provides a distribution line image enhancement system based on scale and brightness popular association, the system comprising:

[0043] The image acquisition and preprocessing module is used to acquire and preprocess the original high-resolution distribution line images in different distribution line scenarios, and use the obtained high-resolution images to construct a high-resolution image training set and a test set, and use the low-resolution images after the high-resolution images are gradually degraded to construct a low-resolution image training set and a test set;

[0044] The scale space set construction module is used to obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution and low-resolution image training sets in the high-resolution and low-resolution image test sets, and combine the high-resolution and low-resolution image training sets to construct a scale space set of high-resolution and low-resolution image combinations;

[0045] The model training module is used to train the scale enhancement network using the scale space set; input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and its image block; train the brightness enhancement network using the high-resolution image and its image block corresponding to each low-resolution image and its image block, and the corresponding scale-enhanced high-resolution image and its image block; sequentially connect the trained scale enhancement network and the brightness enhancement network to construct an image comprehensive enhancement network;

[0046] The image enhancement module is used to enhance the low-resolution distribution line images collected in real time by using the constructed image comprehensive enhancement network.

[0047] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0048] The distribution line image enhancement method and system based on scale and brightness popular correlation provided by the present application have the following beneficial effects:

[0049] First, this method fully considers the impact of environmental factors such as shooting angle, viewing distance and light brightness on distribution line images, and adopts an image enhancement method based on scale and brightness popular correlation to achieve image enhancement of distribution line images, which effectively solves the problem of lack of image enhancement methods for distribution line target inspection in the existing technology.

[0050] Second, on the basis of constructing a good data set, the method obtains the neighboring blocks of each image block of each high-resolution image in the high-resolution and low-resolution image training sets in the high-resolution and low-resolution image test sets, and combines the high-resolution and low-resolution image training sets to construct a scale space set of high- and low-resolution image combinations; the scale space set is used to train the scale enhancement network; the scale enhancement network is composed of a feature extraction subnetwork and a super-resolution submodel, and firstly, the feature extraction submodel that can comprehensively consider the correlation between the local and overall feature extraction of high-resolution and low-resolution images is trained. Then, the super-resolution submodel is used to perform mapping conversion of low-resolution and high-resolution images, and the feature extraction subnetwork is used to verify the scale enhancement effect of the scale-enhanced high-resolution image output by the super-resolution submodel to ensure the training accuracy of the super-resolution submodel. The training method of the scale enhancement network proposed in the present invention can effectively ensure the scale enhancement effect in the process of converting low-resolution images to high-resolution images.

[0051] Third, after the scale enhancement is completed, the brightness of the scale enhanced high-resolution image is enhanced to minimize the impact of environmental factors such as light brightness on the image enhancement, thereby obtaining a more accurate image enhancement effect.

[0052] Fourth, the trained scale enhancement network and brightness enhancement network are connected in sequence to construct an image comprehensive enhancement network; then the constructed image comprehensive enhancement network can be used to effectively enhance the low-resolution distribution line images collected in real time, fundamentally realizing the distribution line image enhancement based on the popular correlation between scale and brightness, achieving a good image enhancement effect, and adapting to the needs of scale and brightness enhancement of distribution line images.

[0053] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0055] Figure 1 A flow chart of a distribution line image enhancement method based on scale and brightness popular association provided in a specific embodiment 1 of the present invention;

[0056] Figure 2 The structural framework of the image comprehensive enhancement network based on scale and brightness popular structure association provided by the embodiment of the present invention;

[0057] Figure 3 A schematic diagram of the structure of a distribution line image enhancement system based on scale and brightness popular correlation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0059] In actual distribution line inspection application scenarios, due to the existence of extreme weather (rain, fog, snow, etc.), the scale and brightness of the captured image are seriously affected, resulting in difficulty in obtaining target images that can effectively reflect the real scene during the distribution line inspection process. In the actual distribution line inspection process, if the impact of scale and brightness on the distribution line image is not considered, the detection and recognition accuracy of specific targets in the distribution line image will be seriously affected. Therefore, in order to solve such problems, this embodiment provides an image enhancement method and system based on scale and brightness popular association. The specific description is as follows.

[0060] Specific embodiment 1 of the present invention discloses a distribution line image enhancement method based on scale and brightness popular correlation, the flow chart of the method is as follows: Figure 1 As shown in the figure, the structural framework of the image comprehensive enhancement network based on scale and brightness popular structure association is as follows Figure 2 The specific instructions are as follows.

[0061] Step S1: collect and preprocess original high-resolution distribution line images in different distribution line scenarios, use the obtained high-resolution images to construct a high-resolution image training set and a test set, and use the low-resolution images after the high-resolution images are gradually degraded to construct a low-resolution image training set and a test set.

[0062] In this embodiment, a high-resolution camera can be used to capture original high-resolution distribution line images in different distribution line scenes. The captured original high-resolution distribution line images include normal distribution line images in different distribution line scenes, and distribution line images of different abnormal types, such as cracks, protrusions, and damage. Afterwards, the collected original high-resolution distribution line images are preprocessed, including: removing outliers and noise, random cropping, scaling, enhancement, and twisting operations, so as to obtain corresponding high-resolution images, which are used to enhance the diversity of images, expand the sample set, and enhance the robustness of images and the generalization ability of models. Afterwards, the high-resolution images obtained by preprocessing can be divided into high-resolution image training sets and test sets.

[0063] It should be noted that each high-resolution image in the high-resolution image training set is gradually degraded (for example, the resolution of the image can be gradually reduced by downsampling or other methods) to obtain a corresponding low-resolution image to construct the low-resolution image training set. Therefore, the high-resolution images in the high-resolution image training set correspond to the low-resolution images in the low-resolution image training set.

[0064] Step S2: Divide each image in the high-resolution and low-resolution image training sets and test sets into blocks; respectively obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set, respectively obtain the neighboring blocks of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set, and respectively combine the high-resolution and low-resolution images and their image blocks in the training set, and the neighboring blocks corresponding to each image block to obtain a scale space set of high-resolution and low-resolution combinations of images;

[0065] In step S2, neighbor blocks are determined in the following manner.

[0066] Step S21: the images in the high-resolution image training set and test set, and the images in the low-resolution image training set and test set are divided into M blocks evenly according to the same division method, and the labels of the image blocks in the same position in different images are the same.

[0067] Step S22: for each image block of each high-resolution image in the high-resolution image training set, obtain the closest K neighboring blocks from the image blocks with the same label in the high-resolution image test set; for each image block of each low-resolution image in the low-resolution image training set, obtain the closest K neighboring blocks from the image blocks with the same label in the low-resolution image test set.

[0068] Step S23: construct each high-resolution image in the high-resolution image training set, each image block of the high-resolution image and its neighboring blocks, and each low-resolution image in the low-resolution image training set, each image block of the low-resolution image and its neighboring blocks into a scale space set of high- and low-resolution combinations of images.

[0069] Specifically, in this embodiment, it is assumed that the high-resolution image training set Among them, I h represents the hth high-resolution image in the high-resolution image training set, and P represents the total number of high-resolution images in the high-resolution image training set. I x represents the xth high-resolution image in the high-resolution image test set, and Q represents the total number of high-resolution images in the high-resolution image test set. In addition, assuming that the low-resolution image training set Among them, I l represents the lth low-resolution image in the low-resolution image training set, and N represents the total number of low-resolution images in the low-resolution image training set. Low-resolution image test set Among them, I t represents the t-th low-resolution image in the low-resolution image test set, and T represents the total number of low-resolution images in the low-resolution image test set. The images in the high-resolution image training set and test set, and the images in the low-resolution image training set and test set are evenly divided into M blocks in the same way. The labels of image blocks with the same position in different images are the same, and the image block labels are set to {i=1,2,3,...,M}.

[0070] At this time, the high-resolution image training set I H The set of image patches with label i in High Resolution Image Test Set I X The set of image patches with label i in Low-resolution image training set I L The set of image patches with label i in Low-resolution image test set I T The set of image patches with label i in

[0071] After that, according to step S22, for each image block in the high-resolution image training set, the closest K neighboring blocks are obtained from the image blocks with the same label in the high-resolution image test set; for each image block in the low-resolution image training set, the closest K neighboring blocks are obtained from the image blocks with the same label in the low-resolution image test set, thereby realizing the distribution of the manifold association of the high-resolution and low-resolution image scale spaces. Specifically, for the high-resolution image training set I H The image patch with label i in Calculate image blocks by Euclidean distance With high-resolution image test set I X The distance between each image block in the image block set with label i in the image block is obtained, and K nearest neighbor blocks are marked as Represents an image block On the high-resolution image test set I X For the low-resolution image training set I L The image patch with label i in Calculate image blocks by Euclidean distance With low-resolution image test set I T The distance between each image block in the image block set with label i in the image block is obtained, and K nearest neighbor blocks are marked as Represents an image block On the low-resolution image test set I T The kth nearest neighbor block in . Traverse i from 1 to M and establish In and At the same time, establish In and The relationship between them is established, and the high-resolution and low-resolution image training sets are combined to construct a scale space set of high- and low-resolution combinations of images.

[0072] Step S3: Train the scale enhancement network using the scale space set.

[0073] The scale enhancement network is composed of a feature extraction sub-model and a super-resolution sub-model. Specifically, in this embodiment, the scale enhancement network is trained in the following manner.

[0074] Step S31: Use the high-resolution image, each image block of the high-resolution image and its neighboring blocks, as well as the low-resolution image, each image block of the low-resolution image and its neighboring blocks in the scale space concentration to train the feature extraction sub-model to obtain a trained feature extraction sub-model.

[0075] The specific process is described as follows: the high-resolution image, each image block of the high-resolution image and its neighboring blocks are used as the input of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, and the error between the local features of each image block of the high-resolution image and the local features of its neighboring blocks, as well as the error between the overall features of the high-resolution image and the overall features reconstructed from the local features of each image block of the high-resolution image, is judged, and the model parameters of the feature extraction sub-model are trained; at the same time, the low-resolution image, each image block of the low-resolution image and its neighboring blocks are used as the input of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, and the error between the local features of each image block of the low-resolution image and the local features of its neighboring blocks, as well as the error between the overall features of the low-resolution image and the overall features reconstructed from the local features of each image block of the low-resolution image, is judged, and the model parameters of the feature extraction sub-model are trained; and a trained feature extraction sub-model is obtained.

[0076] It should be noted that, in this embodiment, the feature extraction sub-model focuses on the correlation between local and overall feature extraction of high- and low-resolution images; preferably, the feature extraction sub-model can be implemented based on Resnet50 or Transformer network.

[0077] In the feature extraction sub-model of this embodiment, the loss function L between the local and the whole high-resolution image is c It is expressed as:

[0078]

[0079] The loss function L between the local and the whole of the low-resolution image d It is expressed as:

[0080]

[0081] in, Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; λ represents the constraint weight, θ represents the model parameters of the feature extraction sub-model; f(.,θ) represents the feature vector obtained after the feature extraction sub-model processes the input information; f(I h ,θ)、f(I l ,θ) represent the overall characteristics of high and low resolution images respectively.

[0082] Step S32: using the low-resolution image and its image blocks as the input of the super-resolution sub-model, using the corresponding high-resolution image and its image blocks as the output of the fitting of the super-resolution sub-model, inputting the scaled high-resolution image and its image blocks predicted and output by the super-resolution sub-model into the feature extraction sub-model for processing to obtain overall features and local features, inputting the high-resolution image and its image blocks into the feature extraction sub-model for processing to obtain overall features and local features, calculating the errors between the overall features and local features corresponding to the scaled high-resolution image and its image blocks and the overall features and local features corresponding to the high-resolution image and its image blocks, adjusting the model parameters of the super-resolution sub-model according to the errors, and obtaining a trained super-resolution sub-model.

[0083] It should be noted that, in this embodiment, the super-resolution sub-model focuses on the mapping relationship in the process of converting low-resolution images to high-resolution images. The super-resolution sub-model can be implemented using a high-low resolution image reconstruction model (Maximum-Entropy CapitalAsset Pricing Model, MECAPM), which is a commonly used super-resolution network structure, which super-divides low-resolution images into high-resolution images and calculates the loss between features with the original high-resolution image to measure the performance of super-resolution reconstruction.

[0084] Specifically, in this embodiment, the overall loss function of the scale-enhanced network can be calculated as follows: (X→D) It is expressed as:

[0085]

[0086] Among them, among them, is the feature vector of each high-resolution image and its image block in the high-resolution image training set (i.e., the feature vector outputted by the high-resolution image and its image block after being processed by the feature extraction sub-model); I H represents the high-resolution image training set, I H The set of image blocks from 1 to M in I L represents the low-resolution training set, I L The set of image blocks 1 to M in ; For each low-resolution image in the low-resolution training set, the high-resolution images and their image patches at each scale are enhanced by the super-resolution sub-model. is the feature vector of each scale-enhanced high-resolution image and its image block (i.e., the feature vector outputted by the scale-enhanced high-resolution image and its image block after being processed by the feature extraction sub-model); and The low and high resolution images in the image correspond to each other; s is Gaussian noise, θ * are the network parameters of the super-resolution sub-model, Represents vector insertion loss calculation, L c , L d Represent the loss functions between the local and the whole of high and low resolution images respectively.

[0087] Step S4: input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and image block; use the high-resolution image and image block corresponding to each low-resolution image and image block, and the corresponding scale-enhanced high-resolution image and image block to train the brightness enhancement network.

[0088] Step S41: input each low-resolution image and its image blocks in the low-resolution image training set into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale enhanced high-resolution image and its image blocks.

[0089] At this point, after the scale enhancement network training is completed, the feature extraction sub-network no longer works.

[0090] Step S42: The pixel values ​​of the scale-enhanced high-resolution image and its image blocks are used as inputs of the brightness enhancement network, and the pixel values ​​of the high-resolution image and its image blocks corresponding to the low-resolution image and its image blocks are used as outputs of the brightness enhancement network fitting. The brightness enhancement network is trained to obtain a trained brightness enhancement network.

[0091] Preferably, in this embodiment, the loss function l of the brightness enhancement network is bp It is expressed as:

[0092] l bp =Σ l |α-I e |(f b (y(l))-f b (y′(l))) (4)

[0093] Wherein, α is a constant, and the setting range is 0<α<0.3; y(l) represents the vector composed of the scale-enhanced high-resolution image and its image blocks corresponding to the l-th low-resolution image and its image blocks in the low-resolution image training set, and y′(l) represents the vector composed of the brightness-enhanced high-resolution image and its image blocks predicted and output by the scale brightness enhancement network; f b (y(l)) represents the data distribution vector of the pixel value of y(l), f b (y′(l)) represents the data distribution vector of the pixel value of y′(l); e=G(y(l))-G(y′(l))-H b ; G(y(l)) and G(y′(l)) represent the Gaussian distribution of the pixel values ​​of y(l) and y′(l), respectively, and H b Indicates the threshold for strong correlation of the brightness manifold.

[0094] Step S5: sequentially connect the trained scale enhancement network and brightness enhancement network to construct an image comprehensive enhancement network; and use the constructed image comprehensive enhancement network to perform image enhancement on the low-resolution distribution line image collected in real time.

[0095] Preferably, in this embodiment, the scale enhancement network and the brightness enhancement network connected in sequence constitute the image comprehensive enhancement network in this embodiment, and the loss function L of the image comprehensive enhancement network is expressed as:

[0096] L=(1-λ bp )L (X→D) +λ bp L bp (5)

[0097] Among them, λ bp It is a weight control parameter used to balance the effects of the scale enhancement network and the brightness enhancement network. By calculating L, the image enhancement associated with the entire scale and brightness popular structure is completed.

[0098] In step S5, the following operations are specifically performed.

[0099] Step S51: Divide the low-resolution distribution line image collected in real time into blocks to obtain image blocks of the low-resolution distribution line image. The process of dividing the image into blocks refers to step S21.

[0100] Step S52: inputting the low-resolution distribution line image and each image block thereof into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale enhancement distribution line image and image blocks thereof.

[0101] Step S53: inputting the pixel values ​​of the scale-enhanced distribution line image and its image blocks into the trained brightness enhancement network, and the brightness enhancement network predicts and outputs the pixel values ​​of the brightness-enhanced distribution line image and its image blocks.

[0102] Step S54: weighted fusion is performed on the brightness enhanced distribution line image and the pixel values ​​of each image block thereof in a manner of image block division to obtain an enhanced high-resolution distribution line image.

[0103] Specifically, according to the image block method, the pixel values ​​of each image block predicted and output by the brightness enhancement network are spliced ​​to reconstruct a reference brightness enhanced image; since the reconstructed reference brightness enhanced image is equivalent to a complete high-resolution image and has similar enhancement characteristics to the brightness enhanced distribution line image, the pixel values ​​of the reconstructed reference brightness enhanced image and the brightness enhanced distribution line image at corresponding pixel positions can be weightedly fused to obtain an enhanced high-resolution distribution line image.

[0104] In the specific implementation process, considering that the brightness enhancement network directly predicts and outputs a brightness enhancement distribution line image that is a direct and complete image, it can be considered to be assigned a higher weight, while the reconstructed reference brightness enhancement image is used as an auxiliary reference, and a lower weight can be considered to be assigned to it. For example, the pixel values ​​of the corresponding pixel positions of the reconstructed reference brightness enhancement image and the brightness enhancement distribution line image can be weighted fused in a ratio of 3:7 or 2:8.

[0105] Specific embodiment 2 of the present invention also discloses a distribution line image enhancement system based on scale and brightness popular correlation, the structural schematic diagram is as follows: Figure 3 As shown, the system includes an image acquisition and preprocessing module, a scale space set construction module, a model training module and an image enhancement module; wherein:

[0106] The image acquisition and preprocessing module is used to acquire and preprocess the original high-resolution distribution line images in different distribution line scenarios, and use the obtained high-resolution images to construct a high-resolution image training set and a test set, and use the low-resolution images after the high-resolution images are gradually degraded to construct a low-resolution image training set and a test set;

[0107] The scale space set construction module is used to obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution and low-resolution image training sets in the high-resolution and low-resolution image test sets, and combine the high-resolution and low-resolution image training sets to construct a scale space set of high-resolution and low-resolution image combinations;

[0108] The model training module is used to train the scale enhancement network using the scale space set; input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and its image block; train the brightness enhancement network using the high-resolution image and its image block corresponding to each low-resolution image and its image block, and the corresponding scale-enhanced high-resolution image and its image block; sequentially connect the trained scale enhancement network and the brightness enhancement network to construct an image comprehensive enhancement network;

[0109] The image enhancement module is used to enhance the low-resolution distribution line images collected in real time by using the constructed image comprehensive enhancement network.

[0110] The specific implementation process of the system embodiment of the present invention can be referred to the above method embodiment, and this embodiment will not be repeated here. Since the principle of this embodiment is the same as that of the above method embodiment, the system also has the corresponding technical effects of the above method embodiment.

[0111] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0112] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A distribution line image enhancement method based on scale and brightness popular correlation, characterized in that: The method comprises: The original high-resolution distribution line images in different distribution line scenarios are collected and preprocessed, and the obtained high-resolution images are used to construct high-resolution image training sets and test sets, and the low-resolution images after the high-resolution images are gradually degraded are used to construct low-resolution image training sets and test sets; Each image in the high-resolution and low-resolution image training sets and test sets is divided into blocks; the neighboring blocks of each image block of each high-resolution image in the high-resolution image training set in the high-resolution image test set are obtained respectively, and the neighboring blocks of each image block of each low-resolution image in the low-resolution image training set in the low-resolution image test set are obtained respectively, and the high-resolution and low-resolution images and their image blocks and the neighboring blocks corresponding to each image block in the training set are combined to obtain a scale space set of high-resolution and low-resolution image combinations; Scale-augmented networks are trained using scale-space sets; Input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and image block; train the brightness enhancement network using the high-resolution image and image block corresponding to each low-resolution image and image block, and the corresponding scale-enhanced high-resolution image and image block; The trained scale enhancement network and brightness enhancement network are connected sequentially to construct an image comprehensive enhancement network; and the constructed image comprehensive enhancement network is used to perform image enhancement on the low-resolution distribution line images collected in real time.

2. The method for enhancing the distribution line image based on scale and brightness manifold correlation according to claim 1, characterized in that: The scale enhancement network is composed of a feature extraction sub-model and a super-resolution sub-model. In the process of training the scale enhancement network using the scale space set, the feature extraction sub-model is first trained by performing the following operations: The feature extraction sub-model is trained using the high-resolution image, each image block of the high-resolution image and its neighboring blocks, as well as the low-resolution image, each image block of the low-resolution image and its neighboring blocks in the scale space concentration to obtain a trained feature extraction sub-model.

3. The method for enhancing the distribution line image based on scale and brightness popular association according to claim 2, characterized in that: The training feature extraction sub-model executes: The high-resolution image, each image block of the high-resolution image and its neighboring blocks are used as inputs of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, determines the error between the local features of each image block of the high-resolution image and the local features of its neighboring blocks, and the error between the overall features of the high-resolution image and the overall features reconstructed from the local features of each image block of the high-resolution image, and trains the model parameters of the feature extraction sub-model; At the same time, the low-resolution image, each image block of the low-resolution image and its neighboring blocks are used as inputs of the feature extraction sub-model, and the feature extraction sub-model performs feature extraction respectively, judges the error between the local features of each image block of the low-resolution image and the local features of its neighboring blocks, and the error between the overall features of the low-resolution image and the overall features reconstructed from the local features of each image block of the low-resolution image, and trains the model parameters of the feature extraction sub-model; Thus, a trained feature extraction sub-model is obtained.

4. The method for enhancing the distribution line image based on scale and brightness manifold association according to claim 3, characterized in that: In the process of training the scale-enhanced network using the scale-space set, the super-resolution sub-model is trained in the following way: The low-resolution image and its image blocks are used as the input of the super-resolution sub-model, and the corresponding high-resolution image and its image blocks are used as the output of the fitting of the super-resolution sub-model. The model parameters of the super-resolution sub-model are trained by adjusting the error between the overall features and local features of the scale high-resolution image and its image blocks processed by the feature extraction sub-model and the overall features and local features of the corresponding high-resolution image and its image blocks processed by the feature extraction sub-model as output of the super-resolution sub-model prediction; Thus, a trained super-resolution sub-model is obtained.

5. The method for enhancing the distribution line image based on scale and brightness manifold association according to claim 4, characterized in that: The loss function L of the scale-enhanced network (X→D) It is expressed as: in, is the feature vector of each high-resolution image and its image patch in the high-resolution image training set, I H represents the high-resolution image training set, I H The set of image blocks from 1 to M in I L represents the low-resolution training set, I L The set of image blocks 1 to M in , where M represents the total number of image blocks divided into each image; For each low-resolution image in the low-resolution training set, the high-resolution images and their image patches at each scale are enhanced by the super-resolution sub-model. Enhance the feature vectors of high-resolution images and their image patches at each scale; and The low and high resolution images in the image correspond to each other; s is Gaussian noise, θ * are the network parameters of the super-resolution sub-model, Represents vector insertion loss calculation, L c , L d Represent the loss functions between the local and the whole of high and low resolution images respectively.

6. The method for power distribution line image enhancement based on scale and brightness manifold correlation according to claim 5, characterized in that: The loss function L between the local and the whole of the high-resolution image c It is expressed as: The loss function L between the local and the whole of the low-resolution image d It is expressed as: Among them, I X Respectively represent the high-resolution image test set, I T Represent the low-resolution image test set; I h represents the hth high-resolution image in the high-resolution image training set, I l represents the lth low-resolution image in the low-resolution image training set; P represents the total number of high-resolution images in the high-resolution image training set, and N represents the total number of low-resolution images in the low-resolution image training set; Represents an image block On the high-resolution image test set I X The kth nearest neighbor block in ; Represents an image block On the low-resolution image test set I T The kth nearest neighbor block in ; Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; Represents the local features of M image blocks with i ranging from 1 to M The overall features after local feature reconstruction; λ represents the constraint weight, θ represents the model parameters of the feature extraction sub-model; f(.,θ) represents the feature vector obtained after the feature extraction sub-model processes the input information; f(I h ,θ)、f(I l ,θ) represent the overall characteristics of high and low resolution images respectively.

7. The method for enhancing the distribution line image based on scale and brightness manifold correlation according to claim 6, characterized in that: Train the brightness enhancement network as follows: Each low-resolution image and each image block in the low-resolution image training set are input into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and its image block; The pixel values ​​of the scale-enhanced high-resolution image and its image blocks are used as the input of the brightness enhancement network respectively, and the pixel values ​​of the high-resolution image and its image blocks corresponding to the low-resolution image and its image blocks are used as the output of the brightness enhancement network fitting. The brightness enhancement network is trained to obtain a trained brightness enhancement network.

8. The method for enhancing the distribution line image based on scale and brightness manifold correlation according to claim 7, characterized in that: The loss function l of the brightness enhancement network bp It is expressed as: l bp =∑ l |α-I e |(f b (y(l))-f b (y′(l))) (4) Wherein, α is a constant, and the setting range is 0<α<0.3; y(l) represents the vector composed of the scale-enhanced high-resolution image and its image blocks corresponding to the l-th low-resolution image and its image blocks in the low-resolution image training set, and y′(l) represents the vector composed of the brightness-enhanced high-resolution image and its image blocks predicted and output by the scale brightness enhancement network; f b (y(l)) represents the data distribution vector of the pixel value of y(l), f b (y′(l)) represents the data distribution vector of the pixel value of y′(l); e=G(y(l))-G(y′(l))-H b ; G(y(l)) and G(y′(l)) represent the Gaussian distribution of the pixel values ​​of y(l) and y′(l), respectively, and H b Indicates the threshold for strong correlation of the brightness manifold.

9. The method for enhancing the distribution line image based on scale and brightness manifold association according to claim 8, characterized in that: The constructed image comprehensive enhancement network is used to enhance the low-resolution distribution line images collected in real time, and the following operations are performed: Dividing the low-resolution distribution line image collected in real time into blocks to obtain image blocks of the low-resolution distribution line image; Inputting the low-resolution distribution line image and each image block thereof into the super-resolution sub-model in the trained scale enhancement network to obtain the corresponding scale enhanced distribution line image and image blocks thereof; Inputting the pixel values ​​of the scale-enhanced distribution line image and its image blocks into the trained brightness enhancement network, the brightness enhancement network predicts and outputs the pixel values ​​of the brightness-enhanced distribution line image and its image blocks; According to the image block method, the pixel values ​​of the brightness enhanced distribution line image and its image blocks are weighted fused to obtain an enhanced high-resolution distribution line image.

10. A distribution line image enhancement system based on scale and brightness popular correlation, characterized in that: The system comprises: The image acquisition and preprocessing module is used to acquire and preprocess the original high-resolution distribution line images in different distribution line scenarios, and use the obtained high-resolution images to construct a high-resolution image training set and a test set, and use the low-resolution images after the high-resolution images are gradually degraded to construct a low-resolution image training set and a test set; The scale space set construction module is used to obtain the neighboring blocks of each image block of each high-resolution image in the high-resolution and low-resolution image training sets in the high-resolution and low-resolution image test sets, and combine the high-resolution and low-resolution image training sets to construct a scale space set of high-resolution and low-resolution image combinations; The model training module is used to train the scale enhancement network using the scale space set; input each low-resolution image and each image block in the low-resolution image training set into the trained scale enhancement network to obtain the corresponding scale-enhanced high-resolution image and its image block; train the brightness enhancement network using the high-resolution image and its image block corresponding to each low-resolution image and its image block, and the corresponding scale-enhanced high-resolution image and its image block; sequentially connect the trained scale enhancement network and the brightness enhancement network to construct an image comprehensive enhancement network; The image enhancement module is used to enhance the low-resolution distribution line images collected in real time by using the constructed image comprehensive enhancement network.

Images