Image optimization method and device for power facility inspection

By building an image optimization model of multi-layer convolutional layer and activation function, the problems of artifacts and noise in images in complex environments of power facilities inspection equipment are solved, high-quality image optimization and noise cancellation are achieved, and recognition capabilities are improved .

CN118657676BActive Publication Date: 2024-11-05SICHUAN UNIVERSITY OF SCIENCE AND ENGINEERING
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411140756.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-05
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

There are a lot of artifacts and noise in images taken in complex environments, making it difficult to accurately identify defects, and existing algorithms do not perform well in practical applications.

Method used

The image optimization model is constructed using multi-layer convolutional layer and activation function. Through the combination of the first extraction module, the second extraction module and the third extraction module, the jump connection and deformable convolutional layer are used to realize feature extraction and reconstruction, reducing pseudo-reconstruction shadow probability and noise impact.

Benefits of technology

Effectively extract important information in the image, reduce the probability of artifacts generated by the output image, improve image optimization quality, accurately identify and eliminate noise, and improve the recognition ability of power facility inspection equipment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118657676B_ABST
    Figure CN118657676B_ABST
Patent Text Reader

Abstract

The present invention provides an image optimization method and device for power facility inspection, which belongs to the field of image data processing. The method comprises: acquiring an image, using a first convolution layer to obtain a first feature map; using multiple first extraction modules to extract features from the first feature map to obtain two different feature maps; using a second extraction module to process the feature map of the last first extraction module to obtain a processed feature map; using a first third extraction module to process the feature map processed by the second extraction module and the feature map input by the first extraction module to obtain an output feature map input to the next third extraction module; using a reconstruction module to process the output feature map of the last third extraction module to obtain an output image, thereby completing image optimization. The present invention achieves more accurate elimination of the influence of noise in the input image, reduces the probability of artifacts in the output image, and improves the quality of the output image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image data processing, and in particular relates to an image optimization method and device for power facility inspection. Background Art

[0002] Due to the extremely wide spatial distribution of power facilities, inspection of power facilities has long been a time-consuming, labor-intensive, and dangerous task. In recent years, the use of drones and robots for inspection operations has greatly overcome the distance limitation and improved work efficiency and safety. However, the inspection environment is complex and changeable, especially on rainy days. There are a large number of rain streaks in the target images collected by the inspection equipment, making it difficult to identify defects. However, although the existing advanced algorithms have achieved good results in tests on public data sets, they have performed disappointingly when applied to images taken during actual inspections. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides an image optimization method and device for power facility inspection, which solves the problems that power facility inspection equipment is greatly affected by the environment, the probability of output image artifacts is high, and it is difficult to accurately eliminate noise.

[0004] In order to achieve the above objectives, the technical solution adopted by the present invention is: an image optimization method for power facility inspection, comprising the following steps:

[0005] S1. Obtain an image of the power facility inspection, and use the first convolutional layer to extract shallow feature information in the image to obtain a first feature map;

[0006] S2, according to the feature information of the first feature map, the feature information is sequentially subjected to feature extraction by multiple first extraction modules, and a feature map is obtained after feature extraction by each first extraction module. And feature map , where the feature map The feature map is input to the third extraction module corresponding to the first extraction module through the jump connection. Input to the next first extraction module, and the last first extraction module converts the feature map input to the second extraction module, each of the first extraction modules is connected in sequence, and each of the third extraction modules is connected in sequence;

[0007] S3, based on the feature map of the last first extraction module , the feature map is obtained by processing with the second extraction module , and input to the first third extraction module;

[0008] S4: The first third extraction module processes the input feature map And the feature map corresponding to the first extraction module Get the output feature map , and the output feature map The third extraction module is input to the next third extraction module, and the previous third extraction module of the current third extraction module except the first third extraction module is defined as the upstream third extraction module. Each third extraction module except the first third extraction module sequentially converts the feature map output by the upstream third extraction module into And the feature map corresponding to the first extraction module As input;

[0009] S5. Use the reconstruction module to process the output feature map of the last third extraction module , get the output image and complete the image optimization of power facility inspection.

[0010] The beneficial effects of the present invention are as follows: the present invention constructs an image optimization model by establishing a first extraction module, a second extraction module and a third extraction module, which can effectively extract important information from the image and effectively remove noise from the image of the power facility inspection, thereby reducing the probability of artifacts in the output optimized image.

[0011] Furthermore, the first extraction module and the third extraction module are symmetrically arranged front to back, the first first extraction module is connected to the last third extraction module, and the last first extraction module is respectively connected to the second extraction module and the first third extraction module.

[0012] Furthermore, the structures of the first extraction modules are the same, and include:

[0013] The second convolutional layer and the first activation function are used to process the feature map input to the first extraction module in sequence to obtain the feature map ;

[0014] The third convolutional layer and the second activation function are used to process the feature map input into the first extraction module in sequence to obtain the feature map ;

[0015] The fourth convolutional layer and the third activation function are used to process the feature map input into the first extraction module in sequence to obtain the feature map ;

[0016] The fifth convolutional layer and the fourth activation function are used to map the feature map Subtract feature map The feature maps after processing are processed in sequence, and the processed feature maps and feature maps are Do the element-by-element product to get the feature map , the feature map and feature map Add, and add the feature map and feature map Do the element-by-element product to get the indexed feature map;

[0017] The sixth convolutional layer and the fifth activation function are used to process the indexed feature maps in sequence to obtain the feature maps ;

[0018] The seventh convolutional layer and the sixth activation function are used to map the feature map and feature map The added feature maps are processed in sequence to obtain the feature maps .

[0019] The beneficial effect of the above further scheme is: the present invention can not only better alleviate information loss by symmetrically inputting the information learned by the first extraction module into the third extraction module, but also realize the establishment of information interaction between multiple perception areas of different ranges, so that the neural network can more comprehensively capture the mapping relationship between pixels, thereby more accurately identifying the scope and effect of local noise in the image based on the learned cross-regional image features, and can more accurately eliminate the influence of noise in the process of reconstructing the image.

[0020] Furthermore, the second extraction module includes:

[0021] Deformable convolution layer and the seventh activation function are used to transform the input feature map Process them sequentially to obtain the conversion feature map;

[0022] Integrate the convolution layer and the eighth activation function to process the conversion feature map in sequence to obtain the feature map .

[0023] The beneficial effect of the above further scheme is: the present invention adopts deformable convolution in the second extraction module, so that the image features output from the second extraction module and input into the subsequent third extraction module have better flexibility, reduce the non-similar structural interference between the first extraction module and the third extraction module, and further improve the quality of the output image.

[0024] Furthermore, the structures of the third extraction modules are the same, and include:

[0025] The eighth convolutional layer and the ninth activation function are used to sequentially process the feature maps input from the upstream end to the third extraction module to obtain feature maps ;

[0026] The ninth convolutional layer and the tenth activation function are used to sequentially process the feature maps input from the upstream end to the third extraction module to obtain feature maps ; Input the feature map of the third extraction module Adding the feature map input from the upstream end to the third extraction module to obtain a cross-domain feature map;

[0027] The tenth convolutional layer and the eleventh activation function are used to process the cross-domain feature maps in sequence to obtain the feature maps ;

[0028] The eleventh convolutional layer is used to map the feature map Subtract feature map The convolution operation is performed on the feature map after the correction to obtain the correction feature map. The twelfth activation function is used to convolve the correction feature map and the feature map Activate the feature map after element-wise multiplication to get the feature map ;

[0029] The twelfth convolution layer and the thirteenth activation function are used to map the feature map and feature map The added feature maps are processed in sequence to obtain the output feature map .

[0030] The beneficial effects of the above further scheme are as follows: the present invention designs the internal calculation process of the first extraction module and the third extraction module, so that the first extraction module has excellent feature mining and feature calibration capabilities, and the third extraction module has excellent feature guidance and feature integration capabilities. The first extraction module forms a coordination relationship with the corresponding third extraction module, and adopts the front-end-back-end modal matching strategy to realize a discriminative image feature construction mechanism, extracting useful feature information required for image reconstruction and eliminating interference information simultaneously, thereby improving the efficiency of image optimization model calculation and reducing the probability of artifacts in the output image.

[0031] Furthermore, the reconstruction module includes:

[0032] The thirteenth convolutional layer and the fourteenth activation function are used to output the feature map of the input Process them one by one to get the processed feature map ;

[0033] The fourteenth convolutional layer and the fifteenth activation function are used to sequentially map the feature maps. Process it to get the output image.

[0034] Furthermore, the present invention provides an image optimization device for power facility inspection, including a processor and a storage device, wherein the storage device stores a computer program, and is characterized in that the processor is used to execute the above-mentioned method by loading the computer program.

[0035] The beneficial effect of the above further scheme is: the present invention uses a storage device to store a computer program containing an image optimization method for power facility inspection, which is applied to images taken during the actual inspection process, thereby reducing the impact of the environment on the power facility inspection equipment, reducing the probability of artifacts in the output image, and achieving accurate noise elimination. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a flow chart of the method of the present invention.

[0037] Figure 2 2 is a diagram of the method architecture in this embodiment. DETAILED DESCRIPTION

[0038] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0039] Before describing the present invention, the following terms are explained:

[0040] ReLU: linear rectification function;

[0041] PReLU: parameter rectified linear unit;

[0042] SigMoid: S-shaped growth curve function.

[0043] Example

[0044] like Figure 1 As shown, the present invention provides an image optimization method for power facility inspection, and its implementation method is as follows:

[0045] S1. Obtain an image of the power facility inspection, and use the first convolutional layer to extract shallow feature information in the image to obtain a first feature map.

[0046] In this embodiment, all the convolutional layers mentioned are existing common convolutional operation layers, and the step sizes of the convolutional layers are all set to 1.

[0047] In this embodiment, assuming that the height, width and number of channels of the input image are a, b, and c respectively, the convolution kernel size of the first convolution layer is usually set to 3. 3. After the first convolutional layer operation, the output feature map size is a, b, d, where d is greater than c, and d can usually be 16, 24, 48, and 64.

[0048] S2, according to the feature information of the first feature map, the feature information is sequentially subjected to feature extraction by multiple first extraction modules, and a feature map is obtained after feature extraction by each first extraction module. And feature map , where the feature map The feature map is input to the third extraction module corresponding to the first extraction module through the jump connection. Input to the next first extraction module, and the last first extraction module converts the feature map input to the second extraction module, each of the first extraction modules is connected in sequence, and each of the third extraction modules is connected in sequence;

[0049] The first extraction module and the third extraction module are symmetrically arranged front and back, the first first extraction module is connected to the last third extraction module, and the last first extraction module is respectively connected to the second extraction module and the first third extraction module.

[0050] In this embodiment, the first first extraction module is connected to the last third extraction module, the second first extraction module is connected to the second-to-last third extraction module, the third first extraction module is connected to the third-to-last third extraction module, and so on, completing the front-to-back symmetrical setting of the first extraction module and the third extraction module.

[0051] The structures of the first extraction modules are the same, and include:

[0052] The second convolutional layer and the first activation function are used to process the feature map input to the first extraction module in sequence to obtain the feature map ;

[0053] The third convolutional layer and the second activation function are used to process the feature map input into the first extraction module in sequence to obtain the feature map ;

[0054] The fourth convolutional layer and the third activation function are used to process the feature map input into the first extraction module in sequence to obtain the feature map ;

[0055] The fifth convolutional layer and the fourth activation function are used to map the feature map Subtract feature map The feature maps after processing are processed in sequence, and the processed feature maps and feature maps are Do the element-by-element product to get the feature map , the feature map and feature map Add, and add the feature map and feature map Do the element-by-element product to get the indexed feature map;

[0056] The sixth convolutional layer and the fifth activation function are used to process the indexed feature maps in sequence to obtain the feature maps ;

[0057] The seventh convolutional layer and the sixth activation function are used to map the feature map and feature map The added feature maps are processed in sequence to obtain the feature maps .

[0058] In this embodiment, for the first first extraction module, the feature map input to the first extraction module is the first feature map; for each first extraction module except the first first extraction module, the feature map input to the first extraction module is the feature map output by the upstream first extraction module. .

[0059] In this embodiment, the convolution kernel size of the second convolutional layer is 3 3. The convolution kernel size of the third convolution layer is 7 7. The convolution kernel size of the fourth convolution layer is 1 1. The convolution kernel size of the fifth, sixth, and seventh convolutional layers is 5. 5; the eleventh activation function is the Logistic function, and the ninth activation function, the tenth activation function, the twelfth activation function, and the thirteenth activation function are all ReLU functions.

[0060] In this embodiment, the feature map , , , , as well as The dimensions are a, b, and d.

[0061] In this embodiment, the specific steps of sequentially processing the feature map by the convolution layer and the activation function are: first, use the convolution layer to perform a convolution operation on the feature map, and then use the activation function to activate the feature map that has been convolved by the convolution layer to obtain a convolution-activated feature map.

[0062] S3, based on the feature map of the last first extraction module , the feature map is obtained by processing with the second extraction module , and input to the first third extraction module;

[0063] The second extraction module comprises:

[0064] Deformable convolution layer and the seventh activation function are used to transform the input feature map Process them sequentially to obtain the conversion feature map;

[0065] Integrate the convolution layer and the eighth activation function to process the conversion feature map in sequence to obtain the feature map .

[0066] In this embodiment, the deformable convolution layer and the integrated convolution layer are both existing technologies, and the convolution kernel size is 3 3. The seventh activation function is the PReLU function, the eighth activation function is the SigMoid function, and the feature map The dimensions are a, b, d.

[0067] S4: The first third extraction module processes the input feature map And the feature map corresponding to the first extraction module Get the output feature map , and the output feature map The third extraction module is input to the next third extraction module, and the previous third extraction module of the current third extraction module except the first third extraction module is defined as the upstream third extraction module. Each third extraction module except the first third extraction module sequentially converts the feature map output by the upstream third extraction module into And the feature map corresponding to the first extraction module As input;

[0068] The structures of the third extraction modules are the same, and include:

[0069] The eighth convolutional layer and the ninth activation function are used to sequentially process the feature maps input from the upstream end to the third extraction module to obtain feature maps ;

[0070] The ninth convolutional layer and the tenth activation function are used to sequentially process the feature maps input from the upstream end to the third extraction module to obtain feature maps ; Input the feature map of the third extraction module Adding the feature map input from the upstream end to the third extraction module to obtain a cross-domain feature map;

[0071] The tenth convolutional layer and the eleventh activation function are used to process the cross-domain feature maps in sequence to obtain the feature maps ;

[0072] The eleventh convolutional layer is used to map the feature map Subtract feature map The convolution operation is performed on the feature map after the correction to obtain the correction feature map. The twelfth activation function is used to convolve the correction feature map and the feature map Activate the feature map after element-wise multiplication to get the feature map ;

[0073] The twelfth convolution layer and the thirteenth activation function are used to map the feature map and feature map The added feature maps are processed in sequence to obtain the output feature map .

[0074] In this embodiment, for the first third extraction module, the feature map input to the third extraction module from the upstream end is the feature map For each third extraction module except the first third extraction module, the feature map input to the third extraction module from the upstream end is the feature map output by the third extraction module at the upstream end. .

[0075] In this embodiment, the convolution kernel sizes of the eighth convolution layer, the eleventh convolution layer, and the twelfth convolution layer are all 3 3. The convolution kernel size of the ninth convolution layer is 5 5. The convolution kernel size of the tenth convolution layer is 1 1. The eleventh activation function is the Logistic function, and the ninth activation function, the tenth activation function, the twelfth activation function, and the thirteenth activation function are all ReLU functions.

[0076] In this embodiment, the feature map , , , as well as The dimensions are a, b, and d.

[0077] S5. Use the reconstruction module to process the output feature map of the last third extraction module , get the output image and complete the image optimization of power facility inspection.

[0078] The reconstruction module comprises:

[0079] The thirteenth convolutional layer and the fourteenth activation function are used to output the feature map of the input Process them one by one to get the processed feature map ;

[0080] The fourteenth convolutional layer and the fifteenth activation function are used to sequentially map the feature maps. Process it to get the output image.

[0081] In this embodiment, the sizes of the convolution kernels of the thirteenth convolutional layer and the fourteenth convolutional layer are both 3 3. The fourteenth activation function is the PReLU function, the fifteenth activation function is the ReLU function, the sizes of the processed feature map are a, b, d, and the sizes of the output image after adjusting the number of channels are a, b, 3.

[0082] like Figure 2As shown, in this embodiment, the input image is first convolutionally operated by the first convolution layer, and the first feature map is output. The first convolution layer is used to extract shallow feature information of the input image, and then the feature information is sequentially extracted by multiple first extraction modules. After the first extraction modules extract features, the feature map is obtained. And feature map , where the feature map The feature map is input to the third extraction module corresponding to the first extraction module through the jump connection. Input to the next first extraction module, and the last first extraction module converts the feature map The data are input to the second extraction module, the first extraction modules are connected in sequence, and the third extraction modules are connected in sequence.

[0083] In this embodiment, the second extraction module uses the last feature map of the first extraction module As input, it then outputs a feature map To the third extraction module. Multiple third extraction modules, the first third extraction module processes the input feature map And the feature map corresponding to the first extraction module Get the output feature map , and the output feature map The feature map output by the third extraction module at the upstream end is input to the next third extraction module. Each third extraction module except the first third extraction module sequentially converts the feature map output by the third extraction module at the upstream end into And the feature map corresponding to the first extraction module As input; the output feature map of the third extraction module of the last module After reconstruction by the reconstruction module, the output image is obtained.

[0084] In this embodiment, images taken during actual inspection operations are collected to construct a training set and a test set, and a comparative experiment is conducted between the method provided by the present invention and the existing advanced algorithm MPRNet (from the article Multi-Stage Progressive Image Restoration). The results are shown in Table 1. From the data in Table 1, it can be seen that the image quality output by this embodiment is much higher than that of MPRNet. Table 1 shows the comparative experimental results between this embodiment and MPRNet.

[0085] Table 1

[0086]

Claims

1. An image optimization method for power facility inspection, characterized in that: The following steps are involved: S1. Obtain an image of the power facility inspection, and use the first convolutional layer to extract shallow feature information in the image to obtain a first feature map; S2, according to the feature information of the first feature map, the feature information is sequentially subjected to feature extraction by multiple first extraction modules, and a feature map is obtained after feature extraction by each first extraction module. And feature map , where the feature map The feature map is input to the third extraction module corresponding to the first extraction module through the jump connection. Input to the next first extraction module, and the last first extraction module converts the feature map input to the second extraction module, each of the first extraction modules is connected in sequence, and each of the third extraction modules is connected in sequence; S3, based on the feature map of the last first extraction module , the feature map is obtained by processing with the second extraction module , and input to the first third extraction module; S4: The first third extraction module processes the input feature map And the feature map corresponding to the first extraction module Get the output feature map , and the output feature map The feature map output by the third extraction module at the upstream end is input to the next third extraction module. And the feature map corresponding to the first extraction module As input; S5. Use the reconstruction module to process the output feature map of the last third extraction module , get the output image, and complete the image optimization of power facility inspection; The method for extracting features from the first extraction module includes: using the second convolution layer and the first activation function to sequentially process the feature map input to the first extraction module to obtain a feature map ; The feature map input into the first extraction module is processed in sequence using the third convolution layer and the second activation function to obtain the feature map ; The feature map input to the first extraction module is processed in sequence using the fourth convolutional layer and the third activation function to obtain the feature map ; Use the fifth convolutional layer and the fourth activation function to transform the feature map Subtract feature map The feature maps after processing are processed in sequence, and the processed feature maps and feature maps are Do the element-by-element product to get the feature map , the feature map and feature map Add, and add the feature map and feature map Do the element-by-element product to get the indexed feature map; The indexed feature map is processed in sequence using the sixth convolutional layer and the fifth activation function to obtain the feature map ; Use the seventh convolution layer and the sixth activation function to transform the feature map and feature map The added feature maps are processed in sequence to obtain the feature maps ; The convolution kernel size of the second convolutional layer is 3 3. The convolution kernel size of the third convolution layer is 7 7. The convolution kernel size of the fourth convolution layer is 1 1; The method for processing the feature map by the third extraction module includes: using the eighth convolution layer and the ninth activation function to sequentially process the feature map input from the upstream end to the third extraction module to obtain the feature map ; The ninth convolutional layer and the tenth activation function are used to process the feature map input from the upstream end to the third extraction module in sequence to obtain the feature map , and input the feature map of the third extraction module Adding the feature map input from the upstream end to the third extraction module to obtain a cross-domain feature map; The cross-domain feature maps are processed in sequence using the tenth convolutional layer and the eleventh activation function to obtain the feature map ; Use the eleventh convolutional layer to map the feature map Subtract feature map The feature map after convolution operation is performed to obtain the correction feature map, and the correction feature map and feature map are convolved using the twelfth activation function. Activate the feature map after element-wise multiplication to get the feature map ; Use the twelfth convolution layer and the thirteenth activation function to transform the feature map and feature map The added feature maps are processed in sequence to obtain the output feature map ; The convolution kernel size of the eighth convolutional layer is 3 3. The convolution kernel size of the ninth convolution layer is 5 5.

2. The image optimization method for power facility inspection according to claim 1, characterized in that: The first extraction module and the third extraction module are symmetrically arranged front and back, the first first extraction module is connected to the last third extraction module, and the last first extraction module is respectively connected to the second extraction module and the first third extraction module.

3. The image optimization method for power facility inspection according to claim 2, characterized in that: The second extraction module comprises: Deformable convolution layer and the seventh activation function are used to transform the input feature map Process them sequentially to obtain the conversion feature map; Integrate the convolution layer and the eighth activation function to process the conversion feature map in sequence to obtain the feature map .

4. The image optimization method for power facility inspection according to claim 1, characterized in that: The reconstruction module comprises: The thirteenth convolutional layer and the fourteenth activation function are used to output the feature map of the input Process them one by one to get the processed feature map ; The fourteenth convolutional layer and the fifteenth activation function are used to sequentially map the feature maps. Process it to get the output image.

5. An image optimization device for power facility inspection, comprising a processor and a storage device, wherein the storage device stores a computer program, characterized in that: The processor is configured to execute the method according to any one of claims 1 to 4 by loading a computer program.