Exposure anomaly correction method for power tower images based on improved MDIN network

Abstract

The present invention discloses a method for correcting abnormal exposure of power tower images based on an improved MDIN network. A data set of power tower images with abnormal exposure is created, images are preprocessed, and an improved MDIN network model is built; a targeted loss function is established, and training is performed with the minimization as the goal; the power tower image with abnormal exposure to be processed is input into the trained improved MDIN network model, and a corrected power tower image is output. The present invention realizes automatic correction of overexposed or underexposed power inspection tower photos, which can not only reduce the operation and maintenance time lag problem of reshooting, but also has high processing efficiency and convenient calculation, so that power inspection can achieve more efficient and intelligent operation.

Description

Technical Field

[0001] The present invention relates to an image exposure optimization processing method in the field of deep learning technology, and specifically to a method for correcting abnormal exposure of power tower images based on an improved MDIN network. Background Art

[0002] In order to ensure the normal operation of my country's growing power transmission lines, drone-based intelligent inspections have now accounted for the vast majority of power in the operation and maintenance of power transmission lines. Compared with traditional manual inspections, drone-based intelligent inspections are more efficient and safer, and can better meet the current requirements for intelligent inspections and lean management.

[0003] During the intelligent inspection process of drones, detailed images and videos of various parts of power towers can be taken efficiently, safely and comprehensively in the harsh environment where the transmission lines are located. The operation and maintenance team no longer needs to go there manually to take pictures, which provides great convenience for the operation and maintenance personnel of power lines. However, with the increasing popularity and application of intelligent inspection by drones, each inspection line will bring a huge amount of data, and the technology for intelligent processing of huge image data is still immature. For massive image data, operation and maintenance personnel need to manually organize the images, including screening and eliminating low-quality photos, classifying and naming photos, etc. This requires manual selection of unusable low-quality photos, classifying each tower, and then checking, analyzing and renaming them one by one before starting to formulate subsequent management and inspection plans.

[0004] The immaturity of intelligent image data processing technology has a huge impact on the intelligent inspection of drones, greatly reducing the efficiency of power inspections. The manual sorting part first needs to screen low-quality photos, which include blurred pictures, deviations in shooting parts, and overexposure or underexposure of the pictures. For abnormal exposure of the power tower images, it is usually necessary to first establish a discrimination model for abnormal photos, then record the flight control information of each photo, let the drone go back to shoot, and then reclassify the photos. This not only consumes a lot of time and energy, but also has a large lag time, and may also cause problems such as repeated shooting. Summary of the invention

[0005] This section aims to summarize the implementation scheme of the present invention for realizing the method for automatically correcting abnormal exposure of power tower images. In order to reduce the problems such as large lag time caused by the existing low-quality photo processing scheme, the purpose of this application is to provide a method for correcting abnormal exposure of power tower images based on an improved MDIN network. By using the deep learning method, an improved MDIN network model for correcting image exposure abnormalities is built by optimizing the MDIN network model, which realizes real-time correction and output of images with abnormal exposure, improves the time lag problem caused by the re-shooting scheme, and makes the intelligent inspection more efficient.

[0006] In order to achieve the above purpose, the present application method comprises the following steps:

[0007] Step 1: Obtain power pole tower images and create a power pole tower image dataset that generates exposure anomalies;

[0008] Step 2: Perform image preprocessing on the images in the power tower image dataset;

[0009] Step 3: Build an improved MDIN network model for correcting image exposure anomalies;

[0010] Step 4: According to the improved MDIN network model, a targeted loss function is established, and the power tower image dataset is input into the improved MDIN network model for training with the goal of minimizing the loss function;

[0011] Step 5: Input the power pole tower image with abnormal exposure to be processed into the trained improved MDIN network model, and output the corrected power pole tower image.

[0012] In the step 1: a plurality of images of power towers with normal lighting in different locations and weather conditions are collected, the collected images are converted from RGB space to HSV space, a Gamma transformation is performed on the V channel in the HSV space to obtain a new V' channel, while keeping the other two channels of the H channel and the S channel in the HSV space unchanged, and the images are converted back to RGB space to obtain an image of a power tower with abnormal exposure;

[0013] The power pole tower images with abnormal exposure and their corresponding power pole tower images with normal illumination are combined into image pairs, and each image pair constitutes a power pole tower image dataset.

[0014] In the step 2: the images in the power tower image data set are converted into images of fixed size, and normalized, and the pixel information of the processed images is converted into tensor information as the input of the improved MDIN network model.

[0015] In step 3:

[0016] The improved MDIN network model adopts an encoding-decoding structure, which is mainly composed of an encoding part and a decoding part connected in sequence;

[0017] The encoding part is mainly composed of three consecutive encoders connected in sequence. The first encoder is composed of three convolution modules, and the second and third encoders are composed of four convolution modules. Each convolution module is composed of a convolution layer and an activation function connected in sequence. Among them, the three convolution modules of the first encoder, the first three convolution modules of the second encoder, and the first three convolution modules of the third encoder are all connected in a dense connection module manner.

[0018] The decoding part is mainly composed of three consecutive decoders connected in sequence. The three decoders are composed of a deconvolution module and two consecutive convolution modules connected in sequence. Each deconvolution module is composed of a deconvolution layer and an activation function connected in sequence.

[0019] The connection method of the densely connected module is specifically as follows: three convolution modules are connected in sequence, and the input of the first convolution module and the output of the second convolution module are input to the third convolution module after channel superposition, and the input and output of the first convolution module and the output of the third convolution module are output after channel superposition.

[0020] In step 4, the loss function is set to a combination of the mean square error (MSE) and the SmoothL1 norm loss function, specifically:

[0021]

[0022]

[0023] Among them, I i represents the i-th power tower image with abnormal exposure trained; J i Represents the image I with the i-th abnormal illumination i The corresponding image of the power tower with normal lighting; F represents the last activation function of the improved MDIN network model; ω represents the parameters of the improved MDIN network model; Smooth L1 () represents the formula of the smoothed L1 loss function; N represents the number of normal and abnormal power tower image sample pairs during training; α and β represent the first and second Gamma transformation parameters.

[0024] In the step 5: a batch of power pole tower images with abnormal exposure are collected by using a drone, and after the images are preprocessed as described in step 2, they are input into the improved MDIN network model after training in batches, and finally the corrected power pole tower images with normal illumination are output.

[0025] The improved MDIN network model of the present invention is specifically a convolutional neural network model.

[0026] Compared with the existing abnormal exposure image processing solutions, the method of this application has the following beneficial effects:

[0027] The embodiment of the present application provides a method for correcting exposure anomalies of power tower images based on an improved MDIN network. By optimizing the MDIN network model, an optimized improved MDIN network model is constructed to achieve automatic correction of overexposed or underexposed power inspection tower photos.

[0028] Compared with the existing power inspection, it is necessary to record the location information of each inspection photo with abnormal exposure and re-shoot it. This not only reduces the operation and maintenance time lag problem of re-shooting, but also has high processing efficiency and convenient calculation, making the power inspection more efficient and intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the method of the present invention;

[0030] Figure 2 Schematic diagram of a densely connected module in the method of the present invention.

[0031] Figure 3 Schematic diagram of the structure of the improved MDIN network model in the method of the present invention. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention are described in further detail below.

[0033] The specific embodiments of the present invention and their implementation process are as follows:

[0034] like Figure 1 As shown, the method comprises the following steps:

[0035] Step 1: Obtain power pole tower images and create a power pole tower image dataset that generates exposure anomalies;

[0036] The collected images include several power pole tower images with normal lighting in different locations and weather conditions. The collected images are converted from RGB space to HSV space, and the V channel in the HSV space is subjected to Gamma transformation to obtain a new V' channel. At the same time, the other two channels of the HSV space, the H channel and the S channel, are kept unchanged and converted back to the RGB space to obtain the power pole tower images with abnormal exposure. The power pole tower images with abnormal exposure and their corresponding power pole tower images with normal lighting are combined into image pairs, and each image pair constitutes a power pole tower image dataset.

[0037] Specifically, 500 original power tower images were collected. For each image with normal illumination, 8 different sets of Gamma transformation parameters were randomly selected, of which 4 sets of parameter gamma values ​​were greater than 1, and the other 4 sets of parameter gamma values ​​were less than 1. The original power tower images were processed to generate 4 under-exposed and 4 over-exposed power tower images. Finally, a dataset of 4,000 power tower images with abnormal exposure was formed, and each abnormal exposure image could correspond to an image with normal illumination.

[0038] Step 2: Perform image preprocessing on the images in the power tower image dataset;

[0039] The images in the power tower image dataset are converted into 224*224 fixed-size images and normalized. The pixel information of the processed images is converted into tensor information as the input of the improved MDIN network model.

[0040] Step 3: Build an improved MDIN network model for correcting image exposure anomalies;

[0041] like Figure 3 As shown, the improved MDIN network model is mainly composed of an encoding part and a decoding part connected in sequence;

[0042] The encoding part is mainly composed of three consecutive encoders connected in sequence. The first encoder is composed of three convolution modules, and the second and third encoders are composed of four convolution modules. Each convolution module is composed of a convolution layer and an activation function connected in sequence. Among them, the three convolution modules of the first encoder, the first three convolution modules of the second encoder, and the first three convolution modules of the third encoder are all connected in a dense connection module manner.

[0043] The decoding part is mainly composed of three consecutive decoders connected in sequence. The three decoders are composed of a deconvolution module and two consecutive convolution modules connected in sequence. Each deconvolution module is composed of a deconvolution layer and an activation function connected in sequence.

[0044] like Figure 2 As shown in the figure, the connection method of the densely connected modules is as follows: the three convolution modules are connected in sequence, and the input of the first convolution module and the output of the second convolution module are input to the third convolution module after channel superposition, and the input and output of the first convolution module and the output of the third convolution module are output after channel superposition, so that the input of each convolution layer of the network will receive the output of all the previous convolution layers.

[0045] The present invention optimizes the encoding part of the MDIN network model, introduces a dense connection module into the encoding part, and changes the connection mode between the original convolutional layers.

[0046] Starting from the output of the first encoder encoder1 in the encoder, the convolution modules in the subsequent second encoder encoder2 and the third encoder encoder3 blocks are connected using dense connection modules, where each encoder contains 3 convolution layers of different sizes but the same number of channels.

[0047] The dense connection module connects the channels between modules and superimposes them by element-wise addition. The output formula after channel superposition is:

[0048]

[0049] Among them, X i ,Y i Represents different input feature maps, i represents the ordinal number of the number of channels of the input feature map; c represents the number of channels; K i Indicates the number of channels of the i-th feature map; the * sign indicates convolution.

[0050] Specific implementation: After the dense connection module is introduced, the number of channels of the network increases. The improved MDIN network adds a 1×1 convolution module after the last layer of the second encoder encoder2 and the third encoder encoder3 blocks, respectively, to restore the number of channels to the original number.

[0051] The final improved MDIN network is a more encoding-less decoding structure, consisting of 3 encoders and 3 decoders. The first encoder encoder1 uses 1 9*9 and 2 3*3 convolutional layers from shallow to deep; the second encoder encoder2 is composed of 1 densely connected 3*3 and two 1*1 convolutional layers, and 1×1 convolutional layers with varying number of channels; the third encoder encoder3 is composed of 3 densely connected 1*1 convolutional layers and 1×1 convolutional layers with varying number of channels, as the encoding part of the network. The decoding part of the network is composed of 3 decoders with the same parameters, decoder1, decoder2, and decoder3, stacked together, where each decoder decoder uses 1 1*1 deconvolution and 2 1*1 convolutional layers in sequence.

[0052] In the above convolutional neural network operation process, each convolution layer performs convolution on the image to extract features of different dimensions. After each layer, a nonlinear activation function is used for processing. After the response result is obtained, it continues to be input into the next convolution layer to extract features. The convolution layer gradually extracts and learns the semantic information of the image from shallow to deep.

[0053] Step 4: According to the improved MDIN network model, a targeted loss function is established, and the power tower image dataset is input into the improved MDIN network model for training with the goal of minimizing the loss function;

[0054] The loss function is set to a combination of the mean square error MSE and the SmoothL1 norm loss function, which can converge faster than the original loss function without reducing the quality of the output corrected image. Specifically:

[0055]

[0056]

[0057] According to the improved MDIN network model and the given loss function formula, with the goal of minimizing the loss function, the data set created in step 1 is input for training to obtain the trained improved MDIN network model.

[0058] Step 5: Input the power pole tower image with abnormal exposure to be processed into the trained improved MDIN network model, and output the corrected power pole tower image.

[0059] A batch of power pole tower images with abnormal exposure were collected using drones. After the images were preprocessed in step 2, they were input into the trained improved MDIN network model in batches, and finally the corrected power pole tower images with normal illumination were output.

[0060] In summary, the embodiment of the present application optimizes the MDIN network model to build an optimized improved MDIN network model, thereby realizing automatic correction of overexposed or underexposed photos of power inspection towers. Compared with the existing power inspection, it is necessary to record the location information of each abnormally exposed inspection photo and re-shoot it. This can not only reduce the operation and maintenance time lag problem of re-shooting, but also has high processing efficiency and convenient calculation, so that power inspection can achieve more efficient and intelligent operation.

[0061] Finally, it should be pointed out that the above embodiments and the proposed control method are only representative examples of the present invention. Obviously, the technical solution of the present invention is not limited to the above embodiments and the proposed control method, and there may be many variations. A person of ordinary skill in the art can directly derive or make several improvements from the disclosure of the present invention, and these improvements should also be within the scope of protection of the claims of the present application.

Claims

1. A method for correcting exposure anomaly of power tower images based on an improved MDIN network, characterized in that: The method comprises the following steps: Step 1: Obtain power pole tower images and create a power pole tower image dataset that generates exposure anomalies; Step 2: Perform image preprocessing on the images in the power tower image dataset; Step 3: Build an improved MDIN network model for correcting image exposure anomalies; Step 4: According to the improved MDIN network model, a targeted loss function is established, and the power tower image dataset is input into the improved MDIN network model for training with the goal of minimizing the loss function; Step 5: Input the power pole tower image with abnormal exposure to be processed into the trained improved MDIN network model, and output the corrected power pole tower image; In step 3: The improved MDIN network model is mainly composed of an encoding part and a decoding part connected in sequence; The encoding part is mainly composed of three consecutive encoders connected in sequence. The first encoder is composed of three convolution modules, and the second and third encoders are composed of four convolution modules. Each convolution module is composed of a convolution layer and an activation function connected in sequence. Among them, the three convolution modules of the first encoder, the first three convolution modules of the second encoder, and the first three convolution modules of the third encoder are all connected in a dense connection module manner. The decoding part is mainly composed of three consecutive decoders connected in sequence. The three decoders are composed of a deconvolution module and two consecutive convolution modules connected in sequence. Each deconvolution module is composed of a deconvolution layer and an activation function connected in sequence. In step 4, the loss function is set to a combination of the mean square error (MSE) and the SmoothL1 norm loss function, specifically: Among them, I i represents the i-th power tower image with abnormal exposure trained; J i Represents the image I with the i-th abnormal illumination i The corresponding image of the power tower with normal lighting; F represents the last activation function of the improved MDIN network model; ω represents the parameters of the improved MDIN network model; Smooth L1 () represents the formula of the smoothed L1 loss function; N represents the number of normal and abnormal power tower image sample pairs during training; α and β represent the first and second Gamma transformation parameters.

2. The method for correcting exposure anomaly of power tower images based on an improved MDIN network according to claim 1, characterized in that: In the step 1: a plurality of images of power towers with normal lighting in different locations and weather conditions are collected, the collected images are converted from RGB space to HSV space, a Gamma transformation is performed on the V channel in the HSV space to obtain a new V' channel, while keeping the other two channels of the H channel and the S channel in the HSV space unchanged, and the images are converted back to RGB space to obtain an image of a power tower with abnormal exposure; The power pole tower images with abnormal exposure and their corresponding power pole tower images with normal illumination are combined into image pairs, and each image pair constitutes a power pole tower image dataset.

3. The method for correcting exposure anomaly of power tower images based on an improved MDIN network according to claim 1, characterized in that: In the step 2: the images in the power tower image data set are converted into images of fixed size, and normalized, and the pixel information of the processed images is converted into tensor information as the input of the improved MDIN network model.

4. The method for correcting exposure anomaly of power tower images based on an improved MDIN network according to claim 1, characterized in that: The connection method of the densely connected module is specifically as follows: three convolution modules are connected in sequence, and the input of the first convolution module and the output of the second convolution module are input to the third convolution module after channel superposition, and the input and output of the first convolution module and the output of the third convolution module are output after channel superposition.

5. The method for correcting exposure anomaly of power tower images based on an improved MDIN network according to claim 1, characterized in that: In the step 5: a batch of power pole tower images with abnormal exposure are collected by using a drone, and after the images are preprocessed as described in step 2, they are input into the improved MDIN network model after training in batches, and finally the corrected power pole tower images with normal illumination are output.

Images