Automatic exposure compensation method and device for monitoring image of overhead transmission line and medium

Abstract

The invention relates to an automatic exposure compensation method and device for a monitoring image of an overhead transmission line and a medium. The method comprises the following steps: screening a to-be-processed monitoring image, and performing Laplacian pyramid decomposition on the to-be-processed monitoring image to obtain L levels of Laplacian pyramid feature maps; and providing each level of Laplacian pyramid feature map of the to-be-processed monitoring image to a corresponding codec in a pre-trained exposure compensation neural network model generator in an inverted order, wherein the cascaded codec generates a corresponding exposure compensation image based on each level of Laplacian pyramid feature map. In the process of reconstructing the normally exposed image, the codec at each level gradually introduces the exposure compensation features from large scale to fine scale on the basis of the Laplacian pyramid feature map of the unusual exposed image, compared with a conventional GAN architecture generator, the generator is graded in cooperation with the Laplacian pyramid feature map, and the generation efficiency is improved. Exposure compensation features are better generated to support high-quality reconstruction exposure normal images.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal image exposure processing, and in particular to a method, device and medium for automatic exposure compensation of an overhead transmission line monitoring image. Background Art

[0002] Abnormal exposure during the acquisition of images by fixed overhead line monitoring is an important factor causing loss of effective image information. Exposure problems can be divided into two categories: one is overexposure, where the camera exposure time is too long, resulting in overbrightness and color distortion in the image area; the other is underexposure, where the exposure time is too short, resulting in overdarkness in the image area. Both overexposure and underexposure are not conducive to subsequent image analysis.

[0003] In the prior art, the exposure image is preprocessed by image de-exposure (or exposure correction) methods. Traditional exposure processing methods generally include: Histogram equalization: By adjusting the grayscale distribution of image pixels, the contrast is increased to alleviate the exposure problem. Gamma correction: By adjusting the gamma value of the image to optimize the brightness and contrast, it is suitable for mild exposure problems. Local contrast enhancement: Enhance the local contrast of the image through block processing to improve the overly bright or dark areas. Filtering technology: Use Gaussian filtering and other means to remove highlight interference in the image. These methods are effective for simple exposure problems, but their effects are limited for complex lighting scenes, especially scenes where highlights and shadows coexist. Summary of the invention

[0004] In order to solve the above technical problem or at least partially solve the above technical problem, the present invention provides a method, device and medium for automatic exposure compensation of overhead transmission line monitoring images.

[0005] In a first aspect, the present invention provides an automatic exposure compensation method for an overhead transmission line monitoring image, comprising: screening a monitoring image I to be processed in sRGB of a current overhead transmission line, performing Laplacian pyramid decomposition on the monitoring image I to be processed to obtain a Laplacian pyramid feature map I with L levels of multi-resolution l .l∈[1,L];

[0006] The Laplacian pyramid feature maps of each level of the monitored image to be processed are provided in reverse order to the corresponding codecs in the pre-trained exposure compensation neural network model generator, and the cascaded codecs generate corresponding exposure compensation images based on the Laplacian pyramid feature maps of each level;

[0007] The training of the exposure compensation neural network model adopts an adversarial generation architecture, including a generator and a discriminator. In the inference stage, only the generator works. The generator is composed of L-level cascaded codecs. The 1st to L-1th codecs are followed by transposed convolutions with a step size of 2 and an output channel of 3. Upsampling operations are performed through transposed convolutions with a step size of 2. The 2nd to Lth codecs are jump-chained in parallel, respectively, so that the coarser-grained compensation features generated by the previous stage can be retained as the depth increases.

[0008] Furthermore, the process of performing Laplacian pyramid decomposition to obtain L levels of multi-resolution Laplacian pyramid feature maps is as follows:

[0009] The L+1-level Gaussian pyramid feature map of the surveillance image I is obtained by iterative Gaussian blurring and downsampling:

[0010]

[0011] Among them, g n*n Represents the convolution kernel of Gaussian blur, n*n is the kernel size, and DownSample() represents the downsampling operation;

[0012] According to the Gaussian pyramid feature map G at any level l Subtract the next level of Gaussian pyramid feature map G l+1 The upsampling result after Gaussian blurring obtains the Laplacian pyramid feature map of this level.

[0013] Among them, UpSample() represents the upsampling operation.

[0014] Furthermore, the codec comprises: a cascaded encoder unit, a cascaded decoder unit and an output network, wherein each encoder unit comprises two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a maximum pooling layer for downsampling effect; each decoder unit comprises: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a transposed convolutional layer for upsampling effect, and the transposed convolutional layer is followed by a LeakyReLU activation function; the output network comprises: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a maximum pooling layer for downsampling effect. A 1×1 convolutional layer is used to fuse channel features; the output of the encoder unit of the previous level is used as the input of the encoder unit of the next level, and the output of the encoder unit of the last level is used as the input of the first-level decoder unit; the output of the last LeakyReLU activation function in the last-level encoder unit and the output of the first-level decoder unit are concatenated as the input of the second-level decoder unit; the output of the last LeakyReLU activation function in the second-to-last decoder unit and the output of the second-level decoder unit are concatenated as the input of the third-level decoder unit, and so on. The output of the last LeakyReLU activation function in the first-level encoder unit and the output of the last-level decoder unit are concatenated as the input of the output network.

[0015] Furthermore, the discriminator is a convolutional neural network whose output uses a Sigmoid activation function. The convolutional neural network extracts image features, and the Sigmoid activation function converts the image features into probabilities for distinguishing image types.

[0016] Furthermore, the exposure compensation neural network model includes the following steps in the training process:

[0017] When the imaging conditions are the same except for the different exposure conditions, pairs of abnormal exposure images and normal exposure reference images are collected to form a data set;

[0018] The abnormal exposure image and the normal exposure reference image in the dataset are decomposed by Laplacian pyramid to obtain L multi-resolution Laplacian pyramid feature maps. are all Laplacian pyramid feature maps of the nth exposure abnormal image, 1:L represents the slices of 1, 2, ... L, is the complete Laplacian pyramid feature map of the nth exposed normal reference image, and N is the size of the dataset;

[0019] The first-level codec receives the L-th level Laplacian pyramid feature map of any exposure anomaly image in the dataset The first-level codec and transposed convolution are expected to be based on the L-th level Laplacian pyramid feature map of the exposure anomaly image. Generate the L-1th level Laplacian pyramid feature map of the normal reference image Matching results;

[0020] The second-level codec receives the L-1th level Laplacian pyramid feature map of the exposure abnormality image. The sum of the output F1 of the transposed convolution after the first-level codec, the expected second-level codec and transposed convolution are based on the L-1th level Laplacian pyramid feature map of the exposure abnormal image The sum of F1 and L-2 level Laplacian pyramid feature map generated by the normal exposure reference image The results of matching the lighting data;

[0021] Recursively, the L-th level codec receives the first level Laplacian pyramid feature map of the exposure abnormality image and the output F of the transposed convolution after the L-1th level encoder-decoder L-1 The Lth level codec is expected to generate a properly exposed image that matches the properly exposed reference image.

[0022] Furthermore, by exposing the L-1th level Laplacian pyramid feature map of the normal reference image and the difference between the first stage encoder / decoder and the transposed convolution output As a loss term to constrain the training of the first-stage encoder and decoder, ED1() represents the first-stage encoder and decoder, Tc() represents the transposed convolution; let

[0023] By exposing the L-2th level Laplacian pyramid feature map of the normal reference image Difference between the second stage encoder / decoder and the transposed convolution output As the loss term to constrain the training of the second-stage codec, where ED2() represents the second-stage codec, let

[0024] Recursively, the difference between the normal exposure reference image and the predicted normal exposure image is As the loss term to constrain the training of the L-th level encoder-decoder, where ED L () is the L-th level codec, n I data_T is the normal exposure reference image, let is the predicted exposure normal image;

[0025] The exposure normal image predicted by the L-th level codec n I preProvided to the discriminator, the discriminator identifies whether the generated image is a generated image or a reference image with normal exposure. When the image generated by the generator is very close to the reference image with normal exposure, the discriminator cannot identify whether the generated image is a generated image or a reference image with normal exposure. The discriminator's recognition result is the most confusing. n I pre )) Combined with the loss function of the preamble generator, the Adam optimizer is used to minimize the loss function.

[0026] Furthermore, the loss function of the discriminator is -logD( n I data_T )-log(1-D( n I pre )), -logD( n I data _T ) refers to the loss of the discriminator in identifying the reference image with normal exposure, -log(1-D( n I pre )) is the loss of the discriminator in identifying the predicted exposure normal image. The loss function of the discriminator constrains the discriminator to distinguish the predicted exposure normal image from the exposure normal reference image as much as possible.

[0027] Furthermore, for the collected surveillance images, surveillance images to be processed are screened out based on the image histogram or color level distribution.

[0028] In a second aspect, the present invention provides an automatic exposure compensation device for an overhead transmission line monitoring image, comprising: at least one processing unit, the processing unit being connected to a storage unit and an acquisition unit via a bus unit, the storage unit storing a computer program, and when the computer program is executed by the processing unit, the automatic exposure compensation method for the overhead transmission line monitoring image is implemented.

[0029] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the automatic exposure compensation method for overhead transmission line monitoring images is implemented.

[0030] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0031] This application uses Laplace pyramid decomposition to retain more global color features in higher-level Laplace feature maps, and more detailed color features in lower-level Laplace feature maps. In the process of reconstructing a normally exposed image, the codecs at all levels gradually introduce exposure compensation features from large scale to fine scale based on the Laplace pyramid feature map of the abnormally exposed image. Compared with the conventional GAN ​​architecture generator, this application uses the Laplace pyramid feature map to grade the generator, and better generate exposure compensation features to support high-quality reconstruction of normally exposed images and exposure reconstruction in complex lighting scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1 A flowchart of an automatic exposure compensation method for an overhead transmission line monitoring image provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of an exposure compensation neural network model provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of a codec in a generator provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a discriminator provided by an embodiment of the present invention;

[0038] Figure 5 A flowchart of Laplacian pyramid decomposition provided by an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of an automatic exposure compensation device for overhead transmission line monitoring images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0042] Example 1

[0043] like Figure 1 As shown, the technology of the present invention implements an automatic exposure compensation method for overhead transmission line monitoring images, including:

[0044] The monitoring image of the current overhead transmission line, the height of the monitoring image is H, the width is W, and the channels are 3, corresponding to the R channel, the G channel, and the B channel respectively. Filter the monitoring image I to be processed in the sRGB format of the current overhead transmission line. In the specific implementation process, for the collected monitoring image, the monitoring image to be processed is filtered out based on the image histogram or the color level distribution. Taking the image histogram as an example, the horizontal axis of the histogram represents the brightness value, and the vertical axis represents the number of pixels under the brightness value. If the peak of the histogram is biased to the left, it means that the image is underexposed; if the peak is biased to the right, it means that the photo is overexposed. The histogram peak position, the preset image underexposure histogram peak position threshold and the image overexposure histogram peak position threshold are compared to determine whether the monitoring image needs to be processed.

[0045] The surveillance image I to be processed is decomposed into a Laplacian pyramid to obtain L multi-resolution Laplacian pyramid feature maps I l ,l∈[1,L];

[0046] like Figure 5 As shown, the process of obtaining the Laplacian pyramid feature map is as follows:

[0047] The L+1-level Gaussian pyramid feature map of the surveillance image I is obtained by iterative Gaussian blurring and downsampling. The formula is as follows:

[0048]

[0049] Among them, g n*n Represents the convolution kernel of Gaussian blur, n*n is the kernel size, and DownSample() represents the downsampling operation;

[0050] Traverse all Gaussian pyramid feature maps, and the Gaussian pyramid feature map G of any level traversed will be l Subtract the next level of Gaussian pyramid feature map G l+1 The upsampling result after Gaussian blurring obtains the Laplacian pyramid feature map of this level:

[0051]

[0052] Among them, UpSample() represents the upsampling operation.

[0053] Through Laplace pyramid decomposition, more global color features are retained in Laplace feature maps of higher levels, and more detailed color features are retained in Laplace feature maps of lower levels.

[0054] The Laplacian pyramid feature maps of each level of the monitored image to be processed are provided in reverse order to the corresponding codec in the pre-trained exposure compensation neural network model generator, and the cascaded codec generates the corresponding exposure compensation image based on the Laplacian pyramid feature maps of each level.

[0055] In the specific implementation process, Figure 2 As shown, the training of the exposure compensation neural network model adopts an adversarial generation architecture, including a generator and a discriminator. In the inference stage, only the generator works. The generator is composed of L-level cascaded codecs. The 1st to L-1th codecs are followed by transposed convolutions with a step size of 2 and an output channel of 3. Upsampling operations are performed through transposed convolutions with a step size of 2. The 2nd to Lth codecs are jump-chained in parallel, respectively, so that the coarser-grained compensation features generated by the previous stage can be retained as the depth increases.

[0056] like Figure 3As shown, the codec includes: a cascaded encoder unit, a cascaded decoder unit and an output network, wherein each encoder unit includes two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a maximum pooling layer that has a downsampling effect; each decoder unit includes: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a transposed convolutional layer that has an upsampling effect, and the transposed convolutional layer is followed by a LeakyReLU activation function; the output network includes: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a 1×1 convolutional layer for fusion channel features. The output of the encoder unit of the previous level is used as the input of the encoder unit of the next level, and the output of the encoder unit of the last level is used as the input of the decoder unit of the first level. The output of the last LeakyReLU activation function in the last-level encoder unit and the output of the first-level decoder unit are concatenated as the input of the second-level decoder unit; the output of the last LeakyReLU activation function in the second-to-last-level decoder unit and the output of the second-level decoder unit are concatenated as the input of the third-level decoder unit, and so on. The output of the last LeakyReLU activation function in the first-level encoder unit and the output of the last-level decoder unit are concatenated as the input of the output network.

[0057] The discriminator is a convolutional neural network whose output uses a Sigmoid activation function. The convolutional neural network extracts image features, and the Sigmoid activation function converts the image features into the probability of distinguishing the image type.

[0058] The exposure compensation neural network model includes:

[0059] When all other imaging conditions are the same except for the different exposure conditions, pairs of abnormal exposure images and normal exposure reference images are collected to form a data set.

[0060] The abnormal exposure image and the normal exposure reference image in the dataset are decomposed by Laplacian pyramid to obtain L multi-resolution Laplacian pyramid feature maps. are all Laplacian pyramid feature maps of the nth exposure abnormal image, and 1:L represents the slices of 1, 2, ...L. are all Laplacian pyramid feature maps of the nth exposed normal reference image, and N is the size of the dataset.

[0061] The training task of this application is to train the generator to gradually generate corresponding normal exposure images based on all Laplacian pyramid feature maps of abnormal exposure images in the data set under the supervision of all Laplacian pyramid feature maps of normal exposure reference images. In the specific implementation process, the first-level codec receives the L-th level Laplacian pyramid feature map of any abnormal exposure image in the data set. The first-level codec and transposed convolution are expected to be based on the L-th level Laplacian pyramid feature map of the exposure anomaly image. Generate the L-1th level Laplacian pyramid feature map of the normal reference image The matching result is thus obtained by exposing the L-1th level Laplacian pyramid feature map of the normal reference image. and the difference between the first stage encoder / decoder and the transposed convolution output As a loss term to constrain the training of the first-stage encoder and decoder, ED1() represents the first-stage encoder and decoder, Tc() represents the transposed convolution; let

[0062] The second-level codec receives the L-1th level Laplacian pyramid feature map of the exposure abnormality image. The sum of the output F1 of the transposed convolution after the first-level codec, the expected second-level codec and transposed convolution are based on the L-1th level Laplacian pyramid feature map of the exposure abnormal image The sum of F1 and L-2 level Laplacian pyramid feature map generated by the normal exposure reference image The illumination data is matched to the result, therefore, the L-2th level Laplacian pyramid feature map of the exposed normal reference image is obtained Difference between the second stage encoder / decoder and the transposed convolution output As the loss term to constrain the training of the second-stage codec, where ED2() represents the second-stage codec, let

[0063] The third-level codec receives the L-2th level Laplacian pyramid feature map of the exposure abnormality image. The sum of the output F2 of the transposed convolution after the second-level codec, the expected third-level codec and transposed convolution are based on the L-2-level Laplacian pyramid feature map of the exposure abnormal image The sum of F2 and L-3 level Laplacian pyramid feature map generated by the normal exposure reference image The illumination data is matched to the result, therefore, the L-3th level Laplacian pyramid feature map of the exposed normal reference image is obtained. and the difference between the third-stage encoder-decoder and the transposed convolution output As the loss term to constrain the training of the second-level codec, where ED3() represents the third-level codec, let

[0064] Recursively, the L-th level codec receives the first level Laplacian pyramid feature map of the exposure abnormality image and the output F of the transposed convolution after the L-1th level encoder-decoder L-1 The L-th level codec is expected to generate a properly exposed image that matches the properly exposed reference image. Therefore, the difference between the properly exposed reference image and the predicted properly exposed image is As the loss term to constrain the training of the L-th level encoder-decoder, where ED L () is the L-th level codec, n I data_T is the normal exposure reference image, let is the predicted normal exposure image, let

[0065] In the process of reconstructing the normally exposed image, the codecs at each level gradually introduce exposure compensation features from large scale to fine scale based on the Laplacian pyramid feature map of the abnormally exposed image.

[0066] The exposure normal image predicted by the L-th level codec n I pre Provided to the discriminator, the discriminator identifies whether the generated image is a generated image or a reference image with normal exposure. The loss function of the discriminator is -logD( n I data_T )-log(1-D( n I pre )), -logD( n I data_T ) refers to the loss of the discriminator in identifying the reference image with normal exposure, -log(1-D( n I pre )) is the loss of the discriminator in identifying the predicted exposure normal image. The loss function of the discriminator constrains the discriminator to distinguish the predicted exposure normal image from the exposure normal reference image as much as possible.

[0067] When the image generated by the generator is very close to the reference image with normal exposure, the discriminator cannot recognize whether the generated image is a generated image or a reference image with normal exposure. The discriminator's recognition result is the most confusing, and the corresponding -log(D( n I pre )) is the smallest. During training, -log(D( n I pre )) Combined with the loss function of the preamble generator, the Adam optimizer is used to minimize the loss function.

[0068] Example 2

[0069] See also Figure 6 As shown, an embodiment of the present invention provides an automatic exposure compensation device for monitoring images of overhead transmission lines, comprising: at least one processing unit, the processing unit is connected to a storage unit and a collection unit through a bus unit, the storage unit is a computer-readable storage medium, and can be used to store software programs, computer executable programs and modules, such as the software programs, computer executable programs and modules corresponding to an automatic exposure compensation method for monitoring images of overhead transmission lines in an embodiment of the present invention, which can be used to store images collected by the collection unit. The processing unit implements the above-mentioned automatic exposure compensation method for monitoring images of overhead transmission lines by running the software programs, computer executable programs and modules stored in the storage unit, comprising:

[0070] Get the sRGB monitoring image I of the current overhead transmission line, and perform Laplacian pyramid decomposition on the monitoring image I to obtain L levels of multi-resolution Laplacian pyramid feature map I l .l∈[1,L];

[0071] The Laplacian pyramid feature maps of each level of the monitored image to be processed are provided in reverse order to the corresponding codecs in the pre-trained exposure compensation neural network model generator, and the cascaded codecs generate corresponding exposure compensation images based on the Laplacian pyramid feature maps of each level;

[0072] The training of the exposure compensation neural network model adopts an adversarial generation architecture, including a generator and a discriminator. In the inference stage, only the generator works. The generator is composed of L-level cascaded codecs. The 1st to L-1th codecs are followed by transposed convolutions with a step size of 2 and an output channel of 3. Upsampling operations are performed through transposed convolutions with a step size of 2. The 2nd to Lth codecs are jump-chained in parallel, respectively, so that the coarser-grained compensation features generated by the previous stage can be retained as the depth increases.

[0073] Of course, the computer program stored in the storage unit of the device for implementing the method for automatic exposure compensation of overhead transmission line monitoring images provided by an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the method for automatic exposure compensation of overhead transmission line monitoring images provided by any embodiment of the present invention.

[0074] An exemplary acquisition unit is a fixed camera installed at a monitoring point near a transmission line, which is used to capture images of overhead transmission lines. The camera supports all-weather monitoring and has high-resolution imaging capabilities.

[0075] An exemplary processing unit is an edge computing device: responsible for running the exposure correction algorithm, such as an NVIDIA Jetson series processor or an industrial computer.

[0076] Example 3

[0077] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the automatic exposure compensation method for monitoring images of overhead transmission lines is implemented, including:

[0078] Get the sRGB monitoring image I of the current overhead transmission line, and perform Laplacian pyramid decomposition on the monitoring image I to obtain L levels of multi-resolution Laplacian pyramid feature map I l .l∈[1,L];

[0079] The Laplacian pyramid feature maps of each level of the monitored image to be processed are provided in reverse order to the corresponding codecs in the pre-trained exposure compensation neural network model generator, and the cascaded codecs generate corresponding exposure compensation images based on the Laplacian pyramid feature maps of each level;

[0080] The training of the exposure compensation neural network model adopts an adversarial generation architecture, including a generator and a discriminator. In the inference stage, only the generator works. The generator is composed of L-level cascaded codecs. The 1st to L-1th codecs are followed by transposed convolutions with a step size of 2 and an output channel of 3. Upsampling operations are performed through transposed convolutions with a step size of 2. The 2nd to Lth codecs are jump-chained in parallel, respectively, so that the coarser-grained compensation features generated by the previous stage can be retained as the depth increases.

[0081] A computer-readable storage medium provided in an embodiment of the present invention stores a computer program which is not limited to the method operations described above, but can also execute related operations in an automatic exposure compensation method for overhead transmission line monitoring images provided in any embodiment of the present invention.

[0082] In the embodiments provided by the present invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for automatic exposure compensation of an overhead transmission line monitoring image, characterized in that: include: Filter the sRGB monitoring image I to be processed of the current overhead transmission line, and perform Laplacian pyramid decomposition on the monitoring image I to be processed to obtain a Laplacian pyramid feature map I with L levels of multi-resolution l .l∈[1,L]; The Laplacian pyramid feature maps of each level of the monitored image to be processed are provided in reverse order to the corresponding codecs in the pre-trained exposure compensation neural network model generator, and the cascaded codecs generate corresponding exposure compensation images based on the Laplacian pyramid feature maps of each level; The training of the exposure compensation neural network model adopts an adversarial generation architecture, including a generator and a discriminator. In the inference stage, only the generator works. The generator is composed of L-level cascaded codecs. The 1st to L-1th codecs are followed by transposed convolutions with a step size of 2 and an output channel of 3. Upsampling operations are performed through transposed convolutions with a step size of 2. The 2nd to Lth codecs are jump-chained in parallel, respectively, so that the coarser-grained compensation features generated by the previous stage can be retained as the depth increases.

2. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 1, characterized in that: The process of performing Laplacian pyramid decomposition to obtain L levels of multi-resolution Laplacian pyramid feature maps is as follows: The L+1-level Gaussian pyramid feature map of the surveillance image I is obtained by iterative Gaussian blurring and downsampling: Among them, g n*n Represents the convolution kernel of Gaussian blur, n*n is the kernel size, and DownSample() represents the downsampling operation; According to the Gaussian pyramid feature map G at any level l Subtract the next level of Gaussian pyramid feature map G l+1 The upsampling result after Gaussian blurring obtains the Laplacian pyramid feature map of this level. Among them, UpSample() represents the upsampling operation.

3. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 1, characterized in that: The codec comprises: a cascaded encoder unit, a cascaded decoder unit and an output network, wherein each encoder unit comprises two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a maximum pooling layer for downsampling effect; each decoder unit comprises: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a transposed convolutional layer for upsampling effect, and the transposed convolutional layer is followed by a LeakyReLU activation function; the output network comprises: two stacked convolutional layers and a LeakyReLU activation function, and the last LeakyReLU activation function is followed by a fusion 1×1 convolutional layer for channel features; the output of the encoder unit of the previous level is used as the input of the encoder unit of the next level, and the output of the encoder unit of the last level is used as the input of the first-level decoder unit; the output of the last LeakyReLU activation function in the last-level encoder unit and the output of the first-level decoder unit are concatenated as the input of the second-level decoder unit; the output of the last LeakyReLU activation function in the second-to-last decoder unit and the output of the second-level decoder unit are concatenated as the input of the third-level decoder unit, and so on. The output of the last LeakyReLU activation function in the first-level encoder unit and the output of the last-level decoder unit are concatenated as the input of the output network.

4. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 1, characterized in that: The discriminator is a convolutional neural network whose output uses a Sigmoid activation function. The convolutional neural network extracts image features, and the Sigmoid activation function converts the image features into the probability of distinguishing the image type.

5. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 1, characterized in that: The exposure compensation neural network model includes the following steps in the training process: When the imaging conditions are the same except for the different exposure conditions, pairs of abnormal exposure images and normal exposure reference images are collected to form a data set; The abnormal exposure image and the normal exposure reference image in the dataset are decomposed by Laplacian pyramid to obtain L multi-resolution Laplacian pyramid feature maps. are all Laplacian pyramid feature maps of the nth exposure abnormal image, 1:L represents the slices of 1, 2, ... L, is the complete Laplacian pyramid feature map of the nth exposed normal reference image, and N is the size of the dataset; The first-level codec receives the L-th level Laplacian pyramid feature map of any exposure anomaly image in the dataset The first-level codec and transposed convolution are expected to be based on the L-th level Laplacian pyramid feature map of the exposure anomaly image. Generate the L-1th level Laplacian pyramid feature map of the normal reference image Matching results; The second-level codec receives the L-1th level Laplacian pyramid feature map of the exposure abnormality image. The sum of the output F1 of the transposed convolution after the first-level codec, the expected second-level codec and transposed convolution are based on the L-1th level Laplacian pyramid feature map of the exposure abnormal image The sum of F1 and L-2 level Laplacian pyramid feature map generated by the normal exposure reference image The results of matching the lighting data; Recursively, the L-th level codec receives the first level Laplacian pyramid feature map of the exposure abnormality image and the output F of the transposed convolution after the L-1th level encoder and decoder L-1 The Lth level codec is expected to generate a properly exposed image that matches the properly exposed reference image.

6. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 5, characterized in that: By exposing the L-1th level Laplacian pyramid feature map of the normal reference image and the difference between the first stage encoder / decoder and the transposed convolution output As the loss term to constrain the training of the first-stage codec, where ED1( ) represents the first-stage codec, Tc( ) represents the transposed convolution; let By exposing the L-2th level Laplacian pyramid feature map of the normal reference image Difference between the second stage encoder / decoder and the transposed convolution output As the loss term to constrain the training of the second-stage codec, where ED2( ) represents the second-stage codec, let Recursively, the difference between the normal exposure reference image and the predicted normal exposure image is As the loss term to constrain the training of the L-th level encoder-decoder, where ED L ( ) is the L-th level codec, n I data_T is the normal exposure reference image, let is the predicted exposure normal image; The exposure normal image predicted by the L-th level codec n I pre Provided to the discriminator, the discriminator identifies whether the generated image is a generated image or a reference image with normal exposure. When the image generated by the generator is very close to the reference image with normal exposure, the discriminator cannot identify whether the generated image is a generated image or a reference image with normal exposure. The discriminator's recognition result is the most confusing. n I pre )) Combined with the loss function of the preamble generator, the Adam optimizer is used to minimize the loss function.

7. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 5, characterized in that: The loss function of the discriminator is -logD( n I data_T )-log(1-D( n I pre )), -logD( n I data_T ) refers to the loss of the discriminator in identifying the reference image with normal exposure, -log(1-D( n I pre )) is the loss of the discriminator in identifying the predicted exposure normal image. The loss function of the discriminator constrains the discriminator to distinguish the predicted exposure normal image from the exposure normal reference image as much as possible.

8. The automatic exposure compensation method for monitoring images of overhead power transmission lines according to claim 1, characterized in that: For the collected surveillance images, the surveillance images to be processed are screened out based on the image histogram or color level distribution.

9. An automatic exposure compensation device for monitoring images of overhead power transmission lines, characterized in that: include: At least one processing unit, the processing unit is connected to a storage unit via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, the automatic exposure compensation method for overhead transmission line monitoring images as described in any one of claims 1-8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the automatic exposure compensation method for overhead transmission line monitoring images as described in any one of claims 1 to 8 is implemented.

Images