A distribution line inspection image enhancement method and system

Abstract

The present invention discloses a distribution line inspection image enhancement method and system, comprising: extracting color information and grayscale information from an acquired distribution line inspection image, and obtaining an image quality score based on the color information and grayscale information using a trained image quality evaluation network; screening the distribution line inspection image according to the image quality score, performing color space conversion on the screening distribution line inspection image, and extracting a brightness component; performing brightness enhancement on the brightness component based on a pre-built enhancement model based on an attention residual module, and performing color space conversion on the enhanced brightness component, the original hue component, and the original saturation component to obtain an enhanced distribution line inspection image. First, image quality evaluation is performed to filter low-quality images, and then the brightness information of the low-illuminance inspection image is enhanced, and at the same time, detail information is enhanced through bilateral filtering to solve the problem of poor image quality in a low-illuminance environment of a distribution network scene.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and in particular to a method and system for enhancing distribution line inspection images. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The intelligent inspection system for distribution lines uses visible light cameras to collect inspection images. However, when the visible light camera detects distribution line equipment faults in a low-light environment, the output image has problems such as unclearness, serious loss of edge detail information, insufficient brightness, and poor contrast, which affects the visual effect of the image and cannot reflect the image features, which is not conducive to the location, analysis, and identification of the fault point. In addition, the image data volume is large and the image quality is uneven. Although the automatic exposure mechanism of the visible light camera, such as ISO, shutter, flash, etc., can enhance the image brightness, it will also produce other effects, such as blur, oversaturation, etc. These problems are not conducive to subsequent equipment identification, defect diagnosis, temperature detection and other processes. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a distribution line inspection image enhancement method and system, which performs image quality evaluation before image enhancement, filters low-quality inspection images according to the evaluation results, and then enhances the brightness information of the low-illumination inspection images through a U-shaped deep learning model. At the same time, the detail information is enhanced through bilateral filtering to solve the problem of poor image quality in low-illumination environments in distribution network scenarios.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a method for enhancing a distribution line inspection image, comprising:

[0007] Extract color information and grayscale information from the obtained distribution line inspection image, and use the trained image quality evaluation network to obtain the image quality score based on the color information and grayscale information;

[0008] The distribution line inspection images are screened according to the image quality scores, and the screened distribution line inspection images are converted into color space to extract the brightness component;

[0009] The brightness component is enhanced based on a pre-built enhancement model based on the attention residual module, and the enhanced brightness component, the original hue component and the original saturation component are converted into color space to obtain an enhanced distribution line inspection image.

[0010] As an optional implementation, the image quality assessment network includes an Inception module to perform feature dimensionality reduction through a 1*1 convolution kernel, and perform multi-scale feature extraction on the inspection image after dimensionality reduction through convolution kernels of different sizes.

[0011] As an optional implementation, the image quality assessment network is trained using an image quality assessment dataset, where the image quality assessment dataset is constructed by manually classifying historical inspection images.

[0012] As an optional implementation, in the process of obtaining the image quality score based on color information and grayscale information using a trained image quality assessment network, features related to image distortion are extracted based on color information and grayscale information using a trained image quality assessment network. After feature extraction, the image quality assessment score is obtained through the fully connected layer of the image quality assessment network.

[0013] As an optional implementation, the color information is the H component in the HSV space; and the grayscale information is an image grayscale map.

[0014] As an optional implementation, the process of constructing the enhancement model based on the attention residual module includes: constructing an enhancement model of a U-shaped network structure based on a channel attention mechanism and a residual module, and performing brightness enhancement on the brightness component through forward propagation using the enhancement model.

[0015] As an optional implementation, the power distribution line inspection image enhancement method further includes: performing bilateral filtering on the enhanced power distribution line inspection image to obtain a power distribution line inspection image with enhanced details.

[0016] In a second aspect, the present invention provides a distribution line inspection image enhancement system, comprising:

[0017] A quality evaluation module is configured to extract color information and grayscale information from the acquired distribution line inspection image, and obtain an image quality score based on the color information and grayscale information using a trained image quality evaluation network;

[0018] A component extraction module is configured to screen the distribution line inspection image according to the image quality score, perform color space conversion on the distribution line inspection image obtained after the screening, and extract the brightness component;

[0019] The brightness enhancement module is configured to perform brightness enhancement on the brightness component based on a pre-built enhancement model based on the attention residual module, and perform color space conversion on the enhanced brightness component, the original hue component and the original saturation component to obtain an enhanced distribution line inspection image.

[0020] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] According to the characteristics of inspection images in distribution network scenarios, the present invention uses deep learning technology to intelligently evaluate the quality of distribution line inspection images, filters low-quality inspection data according to the evaluation results, and ensures that high-quality images of the equipment are collected.

[0024] In order to solve the problem of poor image quality in low-illuminance environment of distribution network scenario, the present invention proposes a low-illuminance inspection image enhancement method for distribution line scenario, which adjusts the brightness information of low-illuminance image through a U-shaped deep learning model, and enhances the detail information of low-illuminance image through bilateral filtering technology, thereby improving the quality of low-illuminance inspection image.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 A flow chart of a method for enhancing distribution line inspection images provided in Example 1 of the present invention;

[0028] Figure 2 A schematic diagram of an image quality assessment network provided in Example 1 of the present invention;

[0029] Figure 3 A schematic diagram of the Inception module provided in Example 1 of the present invention;

[0030] Figure 4 A U-shaped network structure diagram provided in Example 1 of the present invention;

[0031] Figure 5(a)-5(b) Schematic diagram of the residual module and channel attention mechanism CA provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0034] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0036] Example 1

[0037] like Figure 1 As shown, this embodiment provides a distribution line inspection image enhancement method, including:

[0038] (1) Extract color information and grayscale information from the obtained distribution line inspection image, and use the trained image quality evaluation network to obtain the image quality score based on the color information and grayscale information;

[0039] In this embodiment, the image quality of the acquired distribution line inspection images is first evaluated to filter out inspection images with poor quality, specifically including:

[0040] Firstly, an image quality evaluation dataset is constructed. Based on the inspection images collected by the distribution network vehicle-mounted intelligent inspection system, the inspection images are manually classified according to their quality, thus constructing a distribution network scene image quality evaluation dataset.

[0041] Then, a deep learning-based image quality evaluation network is constructed, and feature dimensionality reduction is performed through a 1*1 convolution kernel. Multi-scale feature extraction is performed on the reduced-dimensional image through convolution kernels of different sizes.

[0042] Preferably, if Figure 2 As shown in the figure, the image quality assessment network consists of two convolutional layers, two pooling layers, an Inception module and a fully connected layer;

[0043] Preferably, since the traditional CNN network has high computational complexity and poor generalization effect, an Inception module is added to the image quality evaluation network, such as Figure 3 As shown in the figure, compared with conventional networks, the Inception module first uses a 1*1 convolution kernel to achieve feature dimensionality reduction, and then performs multi-scale feature extraction, thereby reducing network parameters and reducing the computational complexity of the model.

[0044] In this embodiment, the color information and grayscale information of the extracted inspection image are used as the input of the image quality evaluation network to extract features related to image distortion. After the feature extraction is completed, the image quality evaluation score is obtained through the fully connected layer of the image quality evaluation network.

[0045] Since the existing deep learning-based no-reference image quality assessment method only considers the information of the grayscale image and ignores the distortion information contained in the color component of the image, this embodiment uses the image color information and grayscale information together as the input of the CNN model, thereby enhancing the feature extraction capability of the quality assessment model.

[0046] Preferably, the color information is the H component in the HSV space of the image; and the grayscale information is the grayscale map of the image.

[0047] (2) Screening the distribution line inspection images according to the image quality scores, performing color space conversion on the screened distribution line inspection images, and extracting the brightness component;

[0048] In this embodiment, according to the image quality score and a preset score threshold, the inspection images with a score lower than the score threshold are filtered.

[0049] In this embodiment, image enhancement is performed on the inspection images that are screened out and have a score higher than the threshold, and the inspection images of the distribution lines obtained after screening are converted from the RGB color space to the HSI color space, and the I component is extracted in the HSI color space of the image;

[0050] Preferably, the low-light inspection image data set is composed of low-light images of different lines and the corresponding high-quality clear images.

[0051] (3) The brightness component is enhanced based on a pre-built enhancement model based on the attention residual module, and the enhanced brightness component, the original hue component and the original saturation component are converted into a color space to obtain an enhanced distribution line inspection image.

[0052] In this embodiment, if Figure 4As shown, the construction process of the enhancement model based on the attention residual module includes: constructing a deep learning model of a U-type network structure based on a channel attention mechanism (CA) and a residual block, inputting the extracted I component into the deep learning network of the U-type structure, and obtaining the brightness enhanced I component through forward propagation.

[0053] Preferably, if Figure 5(a)-5(b) As shown in the figure, the channel attention mechanism (CA) and the residual module (residualmodule) constitute the attention residual module (CA_Residual module), and the U-shaped network structure contains multiple attention residual modules (CA_Residual module), which can better preserve the detailed features of the image.

[0054] In this embodiment, the enhanced brightness component, the original hue component and the original saturation component are converted into an RGB color space to obtain an RGB image with enhanced brightness.

[0055] In this embodiment, the method further includes: performing bilateral filtering on the distribution line inspection RGB image with enhanced brightness to obtain an RGB image with enhanced details.

[0056] Example 2

[0057] This embodiment provides a distribution line inspection image enhancement system, including:

[0058] A quality evaluation module is configured to extract color information and grayscale information from the acquired distribution line inspection image, and obtain an image quality score based on the color information and grayscale information using a trained image quality evaluation network;

[0059] A component extraction module is configured to screen the distribution line inspection image according to the image quality score, perform color space conversion on the distribution line inspection image obtained after the screening, and extract the brightness component;

[0060] The brightness enhancement module is configured to perform brightness enhancement on the brightness component based on a pre-built enhancement model based on the attention residual module, and perform color space conversion on the enhanced brightness component, the original hue component and the original saturation component to obtain an enhanced distribution line inspection image.

[0061] It should be noted that the above modules correspond to the steps described in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0062] In further embodiments, there is also provided:

[0063] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, it will not be described in detail here.

[0064] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0065] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0066] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in Example 1 is completed.

[0067] The method in Example 1 can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it is not described in detail here.

[0068] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0069] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A distribution line inspection image enhancement method, characterized in that: include: Extract color information and grayscale information from the obtained distribution line inspection image, and use the trained image quality evaluation network to obtain the image quality score based on the color information and grayscale information; The color information is the H component in the HSV space; the grayscale information is the image grayscale map; The distribution line inspection images are screened according to the image quality score, and the inspection images below the score threshold are filtered according to the image quality score and the preset score threshold. The screened inspection images above the score threshold are enhanced, and the distribution line inspection images obtained after screening are converted from the RGB color space to the HSI color space. In the HSI color space of the image, the I component is extracted; The brightness component is enhanced based on a pre-built enhancement model based on the attention residual module, and the enhanced brightness component, the original hue component and the original saturation component are converted into a color space to obtain an enhanced distribution line inspection image; The construction process of the enhancement model based on the attention residual module includes: constructing a deep learning model of a U-shaped network structure based on a channel attention mechanism and a residual module, inputting the extracted I component into the deep learning network of the U-shaped structure, and obtaining the brightness enhanced I component through forward propagation.

2. A distribution line inspection image enhancement method as claimed in claim 1, characterized in that: The image quality evaluation network includes an Inception module, which performs feature dimensionality reduction through a 1*1 convolution kernel, and performs multi-scale feature extraction on the inspection image after dimensionality reduction through convolution kernels of different sizes.

3. A distribution line inspection image enhancement method as claimed in claim 1, characterized in that: The image quality assessment network is trained using an image quality assessment data set, and the image quality assessment data set is constructed by manually classifying historical inspection images.

4. A distribution line inspection image enhancement method as claimed in claim 1, characterized in that: In the process of obtaining the image quality score based on color information and grayscale information using the trained image quality assessment network, features related to image distortion are extracted based on color information and grayscale information using the trained image quality assessment network. After feature extraction, the image quality assessment score is obtained through the fully connected layer of the image quality assessment network.

5. A distribution line inspection image enhancement method as claimed in claim 1, characterized in that: The power distribution line inspection image enhancement method further includes: performing bilateral filtering on the enhanced power distribution line inspection image to obtain a power distribution line inspection image with enhanced details.

6. A distribution line inspection image enhancement system, characterized in that: include: A quality evaluation module is configured to extract color information and grayscale information from the acquired distribution line inspection image, and obtain an image quality score based on the color information and grayscale information using a trained image quality evaluation network; The color information is the H component in the HSV space; the grayscale information is the image grayscale map; The component extraction module is configured to screen the inspection images of the distribution line according to the image quality score, filter the inspection images below the score threshold according to the image quality score and a preset score threshold, perform image enhancement on the screened inspection images above the score threshold, convert the distribution line inspection images obtained after screening from the RGB color space to the HSI color space, and extract the I component in the HSI color space of the image; A brightness enhancement module is configured to perform brightness enhancement on the brightness component based on a pre-built enhancement model based on the attention residual module, and perform color space conversion on the enhanced brightness component, the original hue component and the original saturation component to obtain an enhanced distribution line inspection image; The construction process of the enhancement model based on the attention residual module includes: constructing a deep learning model of a U-shaped network structure based on a channel attention mechanism and a residual module, inputting the extracted I component into the deep learning network of the U-shaped structure, and obtaining the brightness enhanced I component through forward propagation.

7. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 4 is completed.

8. An electronic device as claimed in claim 7, characterized in that: The enhanced distribution line inspection image is subjected to bilateral filtering to obtain a distribution line inspection image with enhanced details.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 5.

Images