Data set image enhancement method applied to power transmission line guiding type robot

Abstract

The invention discloses a data set image enhancement method applied to a power transmission line guide type robot. The image enhancement method comprises the following steps: S1, multi-modal data acquisition and preprocessing; s2, de-noising and enhancing the image embedded by the self-adaptive attention mechanism; s3, super-resolution and detail enhancement; s4, multi-modal data fusion is carried out; and S5, performing real-time application and optimization. Through combination of the RCNN and the GAN, advantages of the RCNN and the GAN can be brought into full play, high-efficiency denoising and super-resolution processing can be realized, image detail information can be continuously optimized through multiple iterations and a feedback mechanism, it is ensured that the enhanced image has high quality and high resolution in various complex environments, and the image quality is improved. And the quality and detail information of the images of the power transmission line guiding type robot can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission lines, and in particular to a data set image enhancement method applied to a power transmission line guided robot. Background Art

[0002] With the development of power systems and the expansion of power grids, the maintenance and monitoring of transmission lines have become increasingly important. As a new type of intelligent maintenance tool, transmission line-guided robots can patrol and monitor lines by moving along them. However, since transmission lines are often located in complex environments, the quality of the captured images may be affected by multiple factors such as weather, light, and vibration, resulting in insufficient image resolution and detail information, which in turn affects the patrol effect.

[0003] In recent years, the rapid development of deep learning technology has provided new ideas and methods for image enhancement. In particular, convolutional neural networks (CNNs) have shown powerful capabilities in the field of image processing. Recursive convolutional neural networks (RCNNs) can achieve efficient image denoising and enhancement while retaining image details by introducing recursive structures. However, a single RCNN still has certain limitations when processing highly complex images. Generative adversarial networks (GANs), as another deep learning model, also have problems such as difficulty in stable training when used alone. Therefore, it is necessary to design an image enhancement method for a data set applied to transmission line guided robots to solve the above problems. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above problems existing in the existing data set image enhancement method applied to the power transmission line guided robot, the present invention is proposed.

[0006] Therefore, the purpose of the present invention is to provide a data set image enhancement method for a power transmission line guided robot, which is suitable for solving the problem that convolutional neural networks cannot be effectively combined with recursive convolutional neural networks, resulting in poor effects of existing image enhancement technologies and unclear images.

[0007] In order to solve the above technical problems, the present invention provides the following technical solution: a data set image enhancement method applied to a power transmission line guided robot, the image enhancement method comprising the following steps:

[0008] S1: Multimodal data acquisition and preprocessing;

[0009] S2: Image denoising and enhancement with adaptive attention mechanism embedding;

[0010] S3: Super-resolution and detail enhancement;

[0011] S4: Multimodal data fusion;

[0012] S5: Real-time application and optimization.

[0013] As a preferred solution of the data set image enhancement method for the power transmission line guided robot of the present invention, the input image is integrated in the spatial domain and the exponential decay function e is used. -λt Controlling the time effect, the image space domain integration formula is as follows:

[0014] I int (t) = ∫ Ω I input (x,y)e -λt dxdy

[0015] Among them, I input (x, y) is the original input image, Ω is the spatial domain of the image, indicating the coordinate range of the image, λ is the parameter that controls the exponential decay, I int (t) is the image integration result at time t.

[0016] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot of the present invention, in the output result of the image space domain integral formula:

[0017] If I int If the value of (t) is too high, it means that the highlight area in the image is too prominent and the image exposure needs to be lowered;

[0018] If I int If the value of (t) is too low, it means that the image area is too dark and the image exposure needs to be increased.

[0019] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot of the present invention, the output result of the image space domain integral formula is derived to obtain the change rate, and the change rate calculation formula is as follows:

[0020]

[0021] Among them, I int (t) is the output result of the image space domain integral formula, I rate (t) is the rate of change of the integral result.

[0022] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot of the present invention, in the output result of the change rate calculation formula:

[0023] If I rate If the value of (t) changes dramatically, it means that the details of the image change greatly and the image needs to be further smoothed;

[0024] If I rate If the value of (t) changes steadily or does not change, it means that the details of the image change little and there is no need to smooth the image.

[0025] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot of the present invention, the change rate result is input into the adaptive attention mechanism function to obtain the final enhanced image, and the image enhancement output formula is as follows:

[0026]

[0027] Among them, I input is the original input image, A(.) is the adaptive attention mechanism function, is the cumulative result of the rate of change, F(.) is the image enhancement function, I end is the enhanced image and is the final output result.

[0028] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot of the present invention, in the output result of the image enhancement output formula:

[0029] If I end The value range of cannot significantly improve the quality of image details or there is over-enhancement. It is necessary to adjust the parameters of the attention mechanism or the details of the enhancement function. Specifically, the final result of the image is optimized by adjusting the weight parameter A of the adaptive attention mechanism and the specific parameters in the image enhancement function F.

[0030] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot described in the present invention, the smoothness of the rate of change is controlled by adjusting the spatial domain Ω or the time parameter t during integration.

[0031] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot described in the present invention, wherein: in step S2, an adaptive attention mechanism is introduced into the image denoising and enhancement model to dynamically adjust the attention to different image areas.

[0032] As a preferred solution of the data set image enhancement method applied to the power transmission line guided robot described in the present invention, wherein: in step S3, RCNN and GAN are used to perform super-resolution processing on the denoised image to improve the image resolution and details.

[0033] The beneficial effects of the present invention are as follows: through the combination of RCNN and GAN, the advantages of both can be fully utilized to achieve efficient denoising and super-resolution processing; secondly, through multiple iterations and feedback mechanisms, the image detail information can be continuously optimized to ensure that the enhanced image has high quality and high resolution in various complex environments, which can effectively improve the quality and detail information of the image of the transmission line guidance robot, and provide efficient and reliable image enhancement technology for the inspection and maintenance of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0035] Figure 1 This is a schematic diagram of the steps of the data set image enhancement method applied to the power transmission line guidance robot proposed in the present invention. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0038] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0039] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0040] Embodiment 1

[0041] Reference Figure 1 , as an embodiment of the present invention, provides a data set image enhancement method applied to a power transmission line guided robot, the image enhancement method comprising the following steps:

[0042] S1: Multimodal data acquisition and preprocessing;

[0043] S2: Image denoising and enhancement with adaptive attention mechanism embedding;

[0044] Introducing an adaptive attention mechanism into the image denoising and enhancement model to dynamically adjust the attention paid to different image regions;

[0045] Use multimodal data and annotation information to train denoising and enhancement models, enabling them to adaptively improve image quality in key areas;

[0046] S3: Super-resolution and detail enhancement;

[0047] Use RCNN and GAN to perform super-resolution processing on the denoised image to improve image resolution and details;

[0048] Through multiple iterations and feedback, the detailed information in the image is further enhanced;

[0049] S4: Multimodal data fusion;

[0050] S5: Real-time application and optimization.

[0051] The input image is integrated in the spatial domain and decayed by an exponential function e -λt Controlling the time effect, the image space domain integration formula is as follows:

[0052] I int (t) = ∫ Ω I input (x,y)e -λt dxdy

[0053] Among them, I input (x, y) is the original input image, Ω is the spatial domain of the image, indicating the coordinate range of the image, λ is the parameter that controls the exponential decay, I int (t) is the image integration result at time t.

[0054] In the image space domain integral formula output result:

[0055] If I int If the value of (t) is too high, it means that the highlight area in the image is too prominent and the image exposure needs to be lowered;

[0056] If I int If the value of (t) is too low, it means that the image area is too dark and the image exposure needs to be increased.

[0057] The output result of the image space domain integral formula is derived to obtain the rate of change. The rate of change calculation formula is as follows:

[0058]

[0059] Among them, I int (t) is the output result of the image space domain integral formula, I rate (t) is the rate of change of the integral result.

[0060] In the output of the rate of change calculation formula:

[0061] If I rate If the value of (t) changes dramatically, it means that the details of the image change greatly and the image needs to be further smoothed;

[0062] If I rate If the value of (t) changes steadily or does not change, it means that the details of the image change little and there is no need to smooth the image;

[0063] By adjusting the spatial domain Ω or the time parameter t during integration, the smoothness of the rate of change can be controlled.

[0064] The change rate result is input into the adaptive attention mechanism function to obtain the final enhanced image. The image enhancement output formula is as follows:

[0065]

[0066] Among them, I input is the original input image, A(.) is the adaptive attention mechanism function, is the cumulative result of the rate of change, F(.) is the image enhancement function, I end is the enhanced image, which is the final output result;

[0067] In the output result of the image enhancement output formula:

[0068] If I endThe value range of cannot significantly improve the quality of image details or there is over-enhancement. It is necessary to adjust the parameters of the attention mechanism or the details of the enhancement function. Specifically, the final result of the image is optimized by adjusting the weight parameter A of the adaptive attention mechanism and the specific parameters in the image enhancement function F.

[0069] Embodiment 2

[0070] The difference from the first embodiment is that the recursive convolutional neural network (RCNN) is combined with the generative adversarial network (GAN) in the existing image enhancement technology for experiment.

[0071] Equipment used in the experiment:

[0072] Vision system: Sony Alpha 7R IV camera with FE 24-70mm F2.8 GM lens for high-resolution image acquisition;

[0073] LiDAR: Velodyne VLP-16, used to collect 3D point cloud data;

[0074] Image processing computing platform: NVIDIA Tesla V100 GPU, equipped with TensorFlow and PyTorch frameworks;

[0075] Specific steps of the experiment:

[0076] The Sony Alpha 7R IV camera and Velodyne VLP-16 LiDAR were used to collect high-resolution images and 3D point cloud data obtained by the transmission line guided robot during the inspection process. The collected images were standardized and preliminarily denoised using median and bilateral filtering.

[0077] Introducing an adaptive attention mechanism into the image denoising and enhancement model to dynamically adjust the attention paid to different image regions;

[0078] The denoised image is input into the recursive convolutional neural network (RCNN) and the generative adversarial network (GAN) for super-resolution processing to improve the resolution and details of the image. Through multiple iterations and feedback optimization, the detailed information in the image is further enhanced.

[0079] The processed high-resolution images are integrated with the 3D point cloud data to form a comprehensive data set, providing multi-angle and all-round image information for power transmission line inspection;

[0080] The enhanced images and fused data are input into the control system of the transmission line guidance robot for real-time inspection and optimization. Through continuous feedback and optimization, the recognition accuracy and inspection efficiency of the system are improved.

[0081] The experimental data are as follows:

[0082] Image enhancement effect comparison experimental data table

[0083]

[0084]

[0085] From the two important indicators of PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index), the method of the present invention shows higher image quality and detail enhancement effect in all test numbers;

[0086] In terms of PSNR, the average improvement is about 3.5dB, which shows that the present invention effectively retains and enhances the effective information in the image during the denoising and super-resolution processing, and significantly reduces noise interference;

[0087] From the perspective of SSIM index, the proposed method shows obvious advantages in enhancing image details, with an average improvement of about 0.15, indicating that the structure and details of the image are better preserved and enhanced through the adaptive attention mechanism and multiple iterative optimizations;

[0088] In summary, through comparative analysis of experimental data, it can be clearly seen that the dataset image enhancement method applied to the power transmission line guided robot of the present invention has significant advantages in image quality and detail enhancement. Compared with the prior art, the present invention not only improves the resolution and clarity of the image, but also can more effectively retain and enhance the detail information of key areas.

[0089] First, through the combination of RCNN and GAN, the advantages of both can be fully utilized to achieve efficient denoising and super-resolution processing; secondly, through multiple iterations and feedback mechanisms, the image detail information can be continuously optimized to ensure that the enhanced image has high quality and high resolution in various complex environments, which can effectively improve the quality and detail information of the transmission line guided robot image, and provide efficient and reliable image enhancement technology for the inspection and maintenance of transmission lines.

[0090] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A dataset image enhancement method for a power transmission line guided robot, characterized in that: The image enhancement method comprises the following steps: S1: Multimodal data acquisition and preprocessing; S2: Image denoising and enhancement with adaptive attention mechanism embedding; S3: Super-resolution and detail enhancement; S4: Multimodal data fusion; S5: Real-time application and optimization.

2. The method for enhancing the image data set for a power transmission line guided robot according to claim 1, characterized in that: The input image is integrated in the spatial domain and decayed by an exponential function e -λt Controlling the time effect, the image space domain integration formula is as follows: I int (t)=∫ Ω I input (x,y)e -λt dxdy Among them, I input (x, y) is the original input image, Ω is the spatial domain of the image, indicating the coordinate range of the image, λ is the parameter that controls the exponential decay, I int (t) is the image integration result at time t.

3. The method for enhancing the image data set for a power transmission line guided robot according to claim 2, characterized in that: In the image space domain integral formula output result: If I int If the value of (t) is too high, it means that the highlight area in the image is too prominent and the image exposure needs to be lowered; If I int If the value of (t) is too low, it means that the image area is too dark and the image exposure needs to be increased.

4. The method for enhancing the image data set for a power transmission line guided robot according to claim 2, characterized in that: The output result of the image space domain integral formula is derived to obtain the change rate, and the change rate calculation formula is as follows: Among them, I int (t) is the output result of the image space domain integral formula, I rate (t) is the rate of change of the integral result.

5. The method for enhancing the image data set for a power transmission line guided robot according to claim 4, characterized in that: In the output of the rate of change calculation formula: If I rate If the value of (t) changes dramatically, it means that the details of the image change greatly and the image needs to be further smoothed; If I rate If the value of (t) changes steadily or does not change, it means that the details of the image change little and there is no need to smooth the image.

6. The method for enhancing the image data set for a power transmission line guided robot according to claim 4, characterized in that: The change rate result is input into the adaptive attention mechanism function to obtain the final enhanced image. The image enhancement output formula is as follows: Among them, I input is the original input image, A(.) is the adaptive attention mechanism function, is the cumulative result of the rate of change, F(.) is the image enhancement function, I end is the enhanced image and is the final output result.

7. The method for enhancing the image data set for a power transmission line guided robot according to claim 6, characterized in that: In the output result of the image enhancement output formula: If I end The value range of cannot significantly improve the quality of image details or there is over-enhancement. It is necessary to adjust the parameters of the attention mechanism or the details of the enhancement function. Specifically, the final result of the image is optimized by adjusting the weight parameter A of the adaptive attention mechanism and the specific parameters in the image enhancement function F.

8. The method for enhancing the image data set for a power transmission line guided robot according to claim 5, characterized in that: By adjusting the spatial domain Ω or the time parameter t during integration, the smoothness of the rate of change can be controlled.

9. The method for enhancing the image data set for a power transmission line guided robot according to claim 1, characterized in that: In step S2, an adaptive attention mechanism is introduced into the image denoising and enhancement model to dynamically adjust the attention paid to different image regions.

10. The method for enhancing the image data set for a power transmission line guided robot according to claim 1, characterized in that: In step S3, RCNN and GAN are used to perform super-resolution processing on the denoised image to improve the image resolution and details.

Images