Butt joint method of real-time imaging robot and high-voltage line and related device

Abstract

The invention discloses a butt joint method of a real-time imaging robot and a high-voltage line and a related device, and the method comprises the steps: carrying out the imaging processing of the high-voltage line on a to-be-detected strain clamp based on a plurality of pulse visual signals collected by a butt joint controller, and obtaining a sequence image of the high-voltage line when the high-voltage line swings; performing definition compensation on the sequence image; performing depth-of-field reconstruction on the modified sequence image based on phase analysis to obtain depth-of-field reconstruction sequence image data; calculating swing frequency data of the high-voltage line based on the depth-of-field reconstruction sequence image data; based on the swing frequency data and the depth-of-field reconstruction sequence image data, utilizing a pulse neural network to predict the motion trail of the high-voltage line; and determining pose information of the unmanned aerial vehicle based on the motion track prediction information so as to control the real-time imaging robot to perform docking processing with the high-voltage line. The butt joint precision is greatly improved while the butt joint efficiency of the real-time imaging robot and the high-voltage line is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of power distribution networks, and in particular to a method for docking a real-time imaging robot with a high-voltage line and related devices. Background Art

[0002] The power system is an essential infrastructure in modern society, and high-voltage transmission lines carry the crucial task of transporting electricity from power stations to end users. In high-voltage transmission lines, tension clamps are key components connecting cables to power towers. Their quality directly impacts the safe and stable operation of the power system. Regular inspections are crucial to promptly identify any defects. Traditional manual inspections not only suffer from low efficiency and accuracy, but also pose risks of working at height and electric shock, which can easily lead to worker injuries or accidents. Therefore, drones and imaging robots have been introduced for inspection. During this inspection process, aligning the imaging robot with the high-voltage line is a crucial step. Currently, alignment of the imaging robot with the high-voltage line often results in unclear images of the line, making it difficult to accurately detect and locate the line. Furthermore, the high-voltage line is constantly oscillating, making pose analysis by the drone challenging. Current simulation analysis methods lack accuracy for pose analysis and are time-consuming, resulting in poor alignment between the imaging robot and the line. Summary of the Invention

[0003] The present invention provides a method and related device for docking a real-time imaging robot with a high-voltage line, which is used to solve the problem in the existing technology that when the high-voltage line is in a swinging state at any time, the position information of the drone cannot be accurately analyzed and obtained, and it takes a long time, resulting in poor docking effect between the imaging robot and the high-voltage line.

[0004] In view of this, a first aspect of the present invention provides a method for docking a real-time imaging robot with a high-voltage line, the method comprising:

[0005] Based on a plurality of pulse visual signals collected by the docking controller, the high-voltage wire on the tension clamp to be inspected is imaged and processed to obtain a sequence of images of the high-voltage wire when it is swinging;

[0006] Performing clarity compensation on the sequence of images to obtain a sequence of images after clarity compensation;

[0007] Performing depth of field reconstruction on the clarity compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data;

[0008] Calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data;

[0009] Based on the swing frequency data and the depth of field reconstruction sequence image data, a pulse neural network is used to predict the motion trajectory of the high-voltage line to obtain motion trajectory prediction information;

[0010] The position information of the UAV is determined based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the position information.

[0011] Optionally, the imaging processing of the high-voltage wire on the tension clamp to be inspected based on the plurality of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage wire when it is swinging includes:

[0012] Merging a plurality of pulse visual signals to obtain a merged signal sequence, and performing abnormality detection and correction on the merged signal sequence to generate a reconstructed image sequence;

[0013] Determining a motion trajectory of each pixel point based on the relative motion between the reconstructed image sequences, and generating an updated pixel point light intensity value based on the motion trajectory of each pixel point using an autoregressive model;

[0014] The high-voltage line is imaged based on the updated pixel point light intensity value in combination with the reconstructed image sequence to obtain a sequence of images of the high-voltage line when it is swinging.

[0015] Optionally, performing clarity compensation on the sequence of images to obtain clarity-compensated sequence of images includes:

[0016] Perform edge sharpening and histogram equalization processing on the sequence images to obtain equalized sequence images;

[0017] Determine the color adjustment parameters of the pixel points based on the clarity detail information in the equalized sequence images;

[0018] Performing pixel noise filtering on the equalized sequence images to obtain a noise-filtered sequence image, and determining transparency adjustment values of the pixels in the noise-filtered sequence image;

[0019] Based on the transparency adjustment value and the color adjustment parameter, a guided filtering algorithm is used to perform clarity compensation on the noise-filtered sequence images to obtain clarity-compensated sequence images.

[0020] Optionally, performing depth of field reconstruction on the clarity-compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data includes:

[0021] Calculating the phase distribution in the clarity-compensated sequence of images, and determining corresponding target three-dimensional mapping coefficients using a three-dimensional mapping coefficient table based on the phase distribution;

[0022] Based on the particle swarm optimization algorithm-back propagation neural network, the depth of field of the sequence images after clarity compensation is restored to obtain the sequence images after depth of field restoration;

[0023] Perform depth detection and foreground prediction on the sequence images after depth of field restoration to obtain a depth estimation map and a foreground probability map;

[0024] Depth of field reconstruction is performed based on the target three-dimensional mapping coefficient using the depth estimation map and the foreground probability map to obtain depth of field reconstruction sequence image data.

[0025] Optionally, the calculating of the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data includes:

[0026] Extract the contour of the high-voltage line based on the depth of field reconstruction sequence image data to obtain corresponding contour data;

[0027] Calculating displacement change data corresponding to contour data in depth of field reconstruction sequence image data;

[0028] The swing frequency data of the high-voltage line is calculated based on the displacement change data using bending curve analysis.

[0029] Optionally, the step of predicting the motion trajectory of the high-voltage line using a spiking neural network based on the oscillation frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information includes:

[0030] Positioning the high-voltage line in the sample sequence images of each preset time period to obtain position information of the high-voltage line in the corresponding sample sequence images;

[0031] Generating cluster trajectories using the position information based on a clustering algorithm, and smoothing the cluster trajectories based on a Kalman filter to obtain motion trajectory data of the high-voltage line in each preset time period;

[0032] Determining a training data set based on the motion trajectory data, and training a preset spiking neural network based on the training data set to obtain a trained spiking neural network;

[0033] The swing frequency data and depth of field reconstruction sequence image data are input into a trained pulse neural network to predict the motion trajectory of the high-voltage line, thereby obtaining motion trajectory prediction information.

[0034] Optionally, determining the position information of the UAV based on the motion trajectory prediction information, and the UAV controlling the real-time imaging robot to perform docking processing with the high-voltage line based on the position information, includes:

[0035] Estimating the future position of the high-voltage line based on the motion trajectory prediction information to obtain future position estimation information, and determining the position and posture information of the UAV based on the future position estimation information;

[0036] A swing control signal of the UAV is determined based on the posture information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the swing control signal.

[0037] A second aspect of the present invention provides a device for docking a real-time imaging robot with a high-voltage line, the device comprising:

[0038] Imaging processing module: used to perform imaging processing on the high-voltage line based on a number of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage line when it is swinging;

[0039] A clarity compensation module is used to perform clarity compensation on the sequence of images to obtain a sequence of images after clarity compensation;

[0040] Depth of field reconstruction module: used to reconstruct the depth of field of the sequence images after clarity compensation based on phase analysis to obtain depth of field reconstruction sequence image data;

[0041] Swing frequency calculation module: used for calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data;

[0042] A motion trajectory prediction module is configured to predict the motion trajectory of the high-voltage line using a pulse neural network based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information;

[0043] Docking module: used to determine the posture information of the UAV based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to dock with the high-voltage line based on the posture information.

[0044] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call the instructions in the memory so that the electronic device executes the method for docking a real-time imaging robot with a high-voltage line as described in the first aspect above.

[0045] The fourth aspect of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method for docking the real-time imaging robot and the high-voltage line described in the first aspect.

[0046] It can be seen from the above technical solutions that the present invention has the following advantages:

[0047] The present invention provides a method for docking a real-time imaging robot with a high-voltage power line. Based on a plurality of pulse visual signals collected by a docking controller, the method performs imaging processing on the high-voltage power line attached to the tension clamp to be inspected. This method can obtain an image with a high signal-to-noise ratio and no blur, achieving a significantly improved visual effect. Clarity compensation is performed on a sequence of images, and depth of field reconstruction is performed on the clarity-compensated sequence of images based on phase analysis to calculate the swing frequency data of the high-voltage power line. Clarity compensation can further improve the clarity of image details, and depth-of-field reconstruction of the sequence of images can more quickly and accurately obtain the swing frequency data of the high-voltage power line in the image. Based on the swing frequency data and depth-of-field reconstruction of the sequence of images, a spiking neural network is used to predict the motion trajectory of the high-voltage power line. This method can achieve rapid and accurate motion trajectory prediction. The motion trajectory prediction information is used to determine the position information of a drone to control the docking of the real-time imaging robot with the high-voltage power line. This method can improve the accuracy and efficiency of the drone's position analysis, greatly improving docking accuracy while ensuring the docking efficiency of the real-time imaging robot and the high-voltage power line, and enabling reliable deployment of the real-time imaging robot. This solves the problem in the existing technology that when the high-voltage line is in a swinging state at any time, it is impossible to accurately determine the swing control signal of the drone, and it takes a long time, resulting in poor docking effect between the imaging robot and the high-voltage line. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 1 is a flow chart of a method for docking a real-time imaging robot with a high-voltage line in an embodiment of the present invention;

[0050] Figure 2 is a flow chart of a method for docking a real-time imaging robot with a high-voltage line in another embodiment of the present invention;

[0051] Figure 3 Schematic diagram of the structure of the docking device between the real-time imaging robot and the high-voltage line in an embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0054] Example 1

[0055] See also Figure 1 , Figure 1 : is a flow chart of a method for docking a real-time imaging robot with a high-voltage line in an embodiment of the present invention, the method comprising:

[0056] S11: performing imaging processing on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage wire when it is swinging;

[0057] In the specific implementation process of the present invention, the imaging processing of the high-voltage line is performed based on a number of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage line when it is swinging, including: merging a number of pulse visual signals to obtain a merged signal sequence, and performing abnormality detection and correction on the merged signal sequence to generate a reconstructed image sequence; determining the motion trajectory of each pixel point based on the relative motion between the reconstructed image sequences, and using an autoregressive model to generate an updated pixel point light intensity value based on the motion trajectory of each pixel point; imaging processing of the high-voltage line is performed based on the updated pixel point light intensity value combined with the reconstructed image sequence to obtain a sequence of images of the high-voltage line when it is swinging.

[0058] It should be noted that the docking controller refers to a control device used to implement the docking function. The docking controller in this embodiment is installed on the real-time imaging robot and is used to control the real-time imaging robot to dock with the high-voltage line and to collect pulse visual signals from the high-voltage line.

[0059] It should be noted that the autoregressive model (AR model) is a type of model used in statistics and machine learning for time series analysis and forecasting. Its core idea is to predict future values based on the past values of the variable itself.

[0060] Specifically, the docking controller is a high dynamic bandwidth docking controller, which is equipped with a high-performance image processing unit that can achieve fast image acquisition and processing, and provide image data with extremely low latency. Through several pulse image sensors of the image processing unit in the docking controller, the high-voltage line in a swinging state on the tension clamp to be inspected is continuously sampled to obtain several corresponding pulse visual signals. Several pulse visual signals are merged to obtain a signal sequence longer than a single pulse visual signal, that is, a merged signal sequence, and the merged signal sequence is subjected to abnormality detection and correction. Since the sampling frequencies corresponding to the pulse image sensors are the same, the number of pulse signals in each sequence part in the merged signal sequence is the same. Abnormality detection can be performed on the number of pulse signals in each sequence part. For example, if the number of pulse signals in a certain sequence part is a, and the number of pulse signals in the remaining sequence parts is b, and a is less than b, then it can be considered that there is an abnormality in the sequence part. For the abnormal pulse signal sequence, the last pulse signal in the sequence part with only a pulse signal is copied ba times, and the copied ba pulse signals are added to the end of the sequence part to realize the correction of the signal sequence. The corrected merged signal sequence is used to generate a reconstructed image sequence using the pulse accumulation imaging method. The motion trajectory of each pixel is determined based on the relative motion between the reconstructed image sequences. The relative motion between the reconstructed image sequences is determined through the optical flow algorithm and pixel matching, thereby determining the motion trajectory of each pixel. Based on the motion trajectory of each pixel, an autoregressive model is used to generate updated pixel intensity values. An autoregressive model of each pixel in the time direction is established according to the motion trajectory of each pixel. The corresponding model parameters are determined through adaptive learning of the autoregressive model. The pixels in the reconstructed image sequence are filtered in the time dimension according to the model parameters, and the pixel intensity values are updated to obtain updated pixel intensity values. The high-voltage power line is imaged based on the updated pixel intensity values combined with the reconstructed image sequence. The adaptive imaging values in the reconstructed image sequence are determined based on the more similar pixel intensity values. This imaging process is used to obtain a sequence of images of the high-voltage power line when it is swinging.

[0061] S12: performing clarity compensation on the sequence images to obtain clarity-compensated sequence images;

[0062] In a specific implementation of the present invention, the clarity compensation of the sequence images to obtain the clarity-compensated sequence images includes: performing edge sharpening and histogram equalization on the sequence images to obtain the equalized sequence images; determining color adjustment parameters of the pixels based on the clarity detail information in the equalized sequence images; performing noise filtering on the pixels of the equalized sequence images to obtain the noise-filtered sequence images, and determining transparency adjustment values of the pixels in the noise-filtered sequence images; and performing clarity compensation on the noise-filtered sequence images using a guided filtering algorithm based on the transparency adjustment values and the color adjustment parameters to obtain the clarity-compensated sequence images.

[0063] Specifically, after obtaining a sequence of images of the high-voltage line when it is swinging, the image data is processed by a field programmable gate array or a dedicated image processing chip to quickly improve the image quality, perform edge sharpening and histogram equalization processing on the sequence images, use a fuzzy filter to blur the sequence images to obtain a fuzzy sequence image, calculate the difference image between the sequence image and the fuzzy sequence image, increase the sharpening intensity of the difference image to obtain a sharpened difference image, superimpose the sequence image and the sharpened difference image to obtain a sequence image with sharpened edges, perform grayscale processing on the sequence image with sharpened edges to obtain a grayscale sequence image, count the occurrence frequencies of different pixel values in the grayscale image to obtain a pixel distribution histogram, and calculate the image The pixel distribution histogram is cumulatively summed to obtain the cumulative distribution function. (In the field of image processing, the pixel distribution histogram shows the number of pixels corresponding to each grayscale level (or color channel value) in the image. Cumulative summation of the pixel distribution histogram to obtain the cumulative distribution function (CDF) is a common and important operation. Among them, the cumulative distribution function (CDF): in image processing, for grayscale images, the cumulative distribution function describes the cumulative probability of pixels in the image whose grayscale value is less than or equal to a certain specific value.) The mapped pixel value of each pixel value in the grayscale image is calculated by the cumulative distribution function, and the mapped pixel value is used to replace the pixel value in the grayscale image to achieve histogram equalization and obtain the equalized sequence image. The color adjustment parameters of the pixel points are determined based on the clarity detail information in the equalized sequence image. The clarity detail information includes the noise, texture detail information and transparency of each pixel point in the image. The color adjustment parameters of the pixel points are matched in the database through the clarity detail information. The color adjustment parameters include the adjustment values of the color and brightness parameters. The equalized sequence of images is subjected to pixel noise filtering, a noise filtering range is set, and pixels within the noise filtering range are filtered out, thereby obtaining a noise-filtered sequence of images. A transparency adjustment value is determined for the pixels in the noise-filtered sequence of images, and the transparency adjustment value of the corresponding pixels is matched based on clarity detail information in the noise-filtered sequence of images. A guided filtering algorithm is used to perform clarity compensation on the noise-filtered sequence of images based on the transparency adjustment value and color adjustment parameters. The transparency adjustment value is used to compensate for detail in the noise-filtered sequence of images. The color-corrected sequence of images is color-corrected using the color adjustment parameters. The color-corrected sequence of images is filtered according to the guided filtering algorithm, effectively removing noise and protecting edges, thereby obtaining a clarity-compensated sequence of images.

[0064] S13: reconstructing the depth of field of the sequence images after clarity compensation based on phase analysis to obtain depth of field reconstructed sequence image data;

[0065] In the specific implementation process of the present invention, the depth of field reconstruction of the sequence images after clarity compensation based on phase analysis to obtain depth of field reconstructed sequence image data includes: calculating the phase distribution in the sequence images after clarity compensation, and determining the corresponding target three-dimensional mapping coefficient based on the phase distribution using a three-dimensional mapping coefficient table; performing depth of field restoration on the sequence images after clarity compensation based on a particle swarm optimization algorithm-back propagation neural network to obtain a sequence image after depth of field restoration; performing depth detection and foreground prediction on the sequence images after depth of field restoration to obtain a depth estimation map and a foreground probability map; performing depth of field reconstruction using the depth estimation map and the foreground probability map based on the target three-dimensional mapping coefficient to obtain depth of field reconstructed sequence image data.

[0066] It should be noted that the "Particle Swarm Optimization Algorithm-Back Propagation Neural Network" technology combines the particle swarm optimization algorithm with the back propagation (BP) neural network. The back propagation neural network (BP neural network) is a multi-layer feedforward neural network trained using the error back propagation algorithm and is currently one of the most widely used neural network models. The BP neural network consists of an input layer, hidden layers, and an output layer, and its learning process consists of forward propagation and backward propagation. During the forward propagation process, input information is processed layer by layer from the input layer through the hidden layers and then transmitted to the output layer. If the actual output of the output layer does not match the expected output, the process switches to backward propagation, returning the error signal along the original connection path and modifying the weights of neurons in each layer to minimize the error signal. However, BP neural networks have drawbacks such as slow convergence and a tendency to fall into local optima. The particle swarm optimization algorithm (PSO) is an optimization algorithm based on swarm intelligence that simulates the behavior of flocks of birds or schools of fish to find the optimal solution. In the particle swarm algorithm, each particle represents a potential solution to a problem. The particle flies through the search space at a certain speed, which is dynamically adjusted based on its own flight experience and the flight experience of the swarm. Particles update their positions by tracking individual extreme values and global extreme values, iterating continuously and ultimately finding the optimal solution. This algorithm has the advantages of simple principle, fast convergence speed, and easy implementation. Combining the two: The particle swarm optimization algorithm is applied to the back propagation neural network, mainly using the particle swarm algorithm to optimize the weights and thresholds of the BP neural network. Since the BP neural network is prone to falling into local optimality, and the particle swarm algorithm has global search capabilities, the particle swarm algorithm is used to search for the optimal weights and threshold initial values on a global scale, and then the BP algorithm is used to perform local fine-tuning. This can improve the convergence speed and generalization ability of the BP neural network, avoid falling into local optimality, and thus improve the performance of the neural network, making it perform better in pattern recognition, prediction, control and other fields.

[0067] Specifically, the phase distribution of the clarity-compensated sequence of images can be calculated using a Fourier-assisted phase shift method. Based on the phase distribution, the corresponding target three-dimensional mapping coefficients are determined using a three-dimensional mapping coefficient table. The three-dimensional mapping coefficient table contains three-dimensional mapping coefficients corresponding to different phase distributions. Depth of field restoration is performed on the clarity-compensated sequence of images using a particle swarm optimization algorithm and a back-propagation neural network. The particle swarm optimization algorithm and the back-propagation neural network have complementary characteristics. The particle swarm optimization algorithm has strong local fine-tuning capabilities but is prone to falling into local optimal points, while the back-propagation neural network is highly robust but has low local search capabilities. Combining the two improves processing efficiency and accuracy. Depth of field restoration ensures the integrity of image information expression, and the depth-restored sequence of images is obtained. Depth detection and foreground prediction are performed on the depth-restored sequence of images. The depth-restored sequence of images is then downsampled, and depth estimation is performed based on the downsampled sequence of images to obtain a depth estimation map. The depth-restored sequence of images is then downsampled and foreground prediction is performed to obtain a foreground probability map. Based on the target three-dimensional mapping coefficient, depth of field reconstruction is performed using the depth estimation map and the foreground probability map. The three-dimensional point coordinates are determined according to the target three-dimensional mapping coefficient. Background blur parameters are generated according to the depth estimation map and the foreground probability map. Foreground enhancement parameters are generated according to the foreground probability map. Depth of field reconstruction is performed according to the three-dimensional mapping coefficients, the background blur parameters and the foreground enhancement parameters to obtain depth of field reconstructed sequence image data. Depth of field reconstruction can enhance the realism of the image, highlight the key parts of the image, and enhance the details of the image.

[0068] S14: calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data;

[0069] In the specific implementation process of the present invention, the calculation of the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data includes: extracting the contour of the high-voltage line based on the depth of field reconstruction sequence image data to obtain corresponding contour data; calculating the displacement change data corresponding to the contour data in the depth of field reconstruction sequence image data; and calculating the swing frequency data of the high-voltage line based on the displacement change data using bending curve analysis.

[0070] It should be noted that the bending curve analysis method calculates the high-voltage power line's oscillation frequency data by mathematically analyzing the displacement changes of the high-voltage power line in the depth-of-field reconstructed image sequence data to obtain its oscillation frequency. This method first plots the position curves of the high-voltage power line at various time points based on the high-voltage power line profile data in the depth-of-field reconstructed image sequence data. These position curves are then smoothed to reduce the effects of noise and errors, resulting in a more accurate bending curve. Next, mathematical methods are used to analyze the bending curve to calculate the high-voltage power line's oscillation period—the time interval between two adjacent peaks or troughs. Finally, the oscillation frequency data is obtained by counting the number of oscillation periods and dividing them by the total time. This method can accurately reflect the actual oscillation of the high-voltage power line and provide reliable data support for the docking of real-time imaging robots with the power line.

[0071] Specifically, the contour of the high-voltage line is extracted based on the depth-of-field reconstruction sequence image data, that is, the contour of the high-voltage line in each frame of the depth-of-field reconstruction sequence image data is extracted by a contour extraction algorithm to obtain corresponding contour data. The displacement change data corresponding to the contour data in the depth-of-field reconstruction sequence image data is calculated, and the displacement change calculation is performed based on the coordinate points of the contour data in the image to obtain the displacement change data, that is, the displacement of the high-voltage line contour in adjacent frame images is calculated. Based on the displacement change data, the swing frequency data of the high-voltage line is calculated using a bending curve analysis. The first swing frequency is calculated based on the displacement change data combined with the interval between each frame in the sequence image. The curvature change data of the bending curve of the high-voltage line contour in adjacent frame images is analyzed, and the second swing frequency is calculated based on the curvature change data. The final swing frequency data is calculated based on the first swing frequency and the second swing frequency. Average value calculation or weighted fusion calculation can be used.

[0072] S15: Predicting the motion trajectory of the high-voltage line using a spiking neural network based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information;

[0073] In the specific implementation process of the present invention, the motion trajectory of the high-voltage line is predicted by using a pulse neural network based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information, including: locating the high-voltage line in the sample sequence image of each preset time period to obtain the position information of the high-voltage line in the corresponding sample sequence image; generating a clustering trajectory based on the position information using a clustering algorithm, and smoothing the clustering trajectory based on a Kalman filter to obtain the motion trajectory data of the high-voltage line in each preset time period; determining a training data set based on the motion trajectory data, and training a preset pulse neural network based on the training data set to obtain a trained pulse neural network; inputting the swing frequency data and the depth of field reconstruction sequence image data into the trained pulse neural network to predict the motion trajectory of the high-voltage line to obtain motion trajectory prediction information.

[0074] It should be noted that the Spiking Neural Network (SNN) is a third-generation neural network that, based on artificial neural networks, further draws on the pulse emission mechanism of biological neurons. Compared with traditional artificial neural networks, spiking neural networks have higher efficiency and accuracy in processing time series data and dynamic changes. In the present invention, spiking neural networks are used to process the swing frequency data of high-voltage lines and depth of field reconstruction sequence image data, both of which contain information changes in the time dimension. Through the learning and optimization of spiking neural networks, the motion patterns of high-voltage lines can be captured more accurately, thereby achieving accurate prediction of the motion trajectory of high-voltage lines. This predictive ability is crucial for the docking of real-time imaging robots with high-voltage lines. It can help the robot make adjustments in advance to ensure the accuracy and safety of the docking process.

[0075] Specifically, the high-voltage wires are located in sample sequence images for each preset time period, i.e., the locations of the high-voltage wires in the sample sequence images for each preset time period are located, and positional information of the high-voltage wires in the corresponding sample sequence images is obtained. A clustering algorithm is used to generate cluster trajectories using this positional information. A density-based clustering algorithm is used to cluster the high-voltage wires between images, obtaining cluster trajectories for the high-voltage wires in each time period. The cluster trajectories are then smoothed using a Kalman filter. Because the resulting cluster trajectories may exhibit significant volatility, a Kalman filter is used to smooth the cluster trajectories, obtaining motion trajectory data for the high-voltage wires in each preset time period. A training dataset is determined based on the motion trajectory data, and a pre-set spiking neural network is trained based on the training dataset. The spiking neural network comprises a plurality of spiking neurons, each of which includes an input, an output, a threshold, and a pulse firing mechanism, to obtain a trained spiking neural network. The oscillation frequency data and depth-of-field reconstruction sequence image data are input into the trained spiking neural network to predict the motion trajectory of the high-voltage wires. Through the spiking neural network's ultra-high-speed motion target detection and tracking, the motion trajectory of the high-voltage wires can be quickly and accurately predicted, obtaining motion trajectory prediction information.

[0076] S16: Determine the posture information of the UAV based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the posture information.

[0077] In the specific implementation process of the present invention, the posture information of the drone is determined based on the motion trajectory prediction information, and the drone controls the real-time imaging robot to dock with the high-voltage line based on the posture information, including: estimating the future position of the high-voltage line based on the motion trajectory prediction information, obtaining future position estimation information, and determining the posture information of the drone based on the future position estimation information; determining the swing control signal of the drone based on the posture information, and the drone controls the real-time imaging robot to dock with the high-voltage line based on the swing control signal.

[0078] Specifically, the future position of the high-voltage line is estimated based on the motion trajectory prediction information. The coordinate information of the high-voltage line's future position is estimated based on the motion trajectory prediction, that is, the future position estimation information is obtained. The position and posture information of the drone is determined based on the future position estimation information, that is, the position and posture information corresponding to the drone is determined based on the future swinging position of the high-voltage line. The drone's swing control signal is determined based on the position and posture information. The drone controls the real-time imaging robot to dock with the high-voltage line based on the swing control signal. The drone's attitude controller deploys the real-time imaging robot based on the swing control signal, thereby controlling the real-time imaging robot to dock with the high-voltage line through the docking controller, so that the real-time imaging robot completes the docking with the high-voltage line.

[0079] In an embodiment of the present invention, imaging processing is performed on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by a docking controller, thereby obtaining an image with a high signal-to-noise ratio and no blur, thereby achieving a significantly improved visual effect. Clarity compensation is performed on the sequence of images, and depth of field reconstruction is performed on the clarity-compensated sequence of images based on phase analysis to calculate the swing frequency data of the high-voltage wire. Clarity compensation can further improve the clarity of image details, and depth of field reconstruction of the sequence of image data can more quickly and accurately obtain the swing frequency data of the high-voltage wire in the image. Based on the swing frequency data and depth of field reconstruction of the sequence of image data, a pulse neural network is used to predict the motion trajectory of the high-voltage wire, thereby achieving fast and accurate motion trajectory prediction. The motion trajectory prediction information is used to determine the position information of the drone to control the real-time imaging robot to dock with the high-voltage wire. This can improve the accuracy and efficiency of the drone's position analysis, greatly improve the docking accuracy while ensuring the docking efficiency of the real-time imaging robot and the high-voltage wire, and achieve reliable deployment of the real-time imaging robot.

[0080] Example 2

[0081] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of a method for docking a real-time imaging robot with a high-voltage line in another embodiment of the present invention, the method comprising:

[0082] S201: performing imaging processing on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage wire when it is swinging;

[0083] S202: performing clarity compensation on the sequence of images to obtain clarity-compensated sequence of images;

[0084] S203: reconstructing the depth of field of the clarity-compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data;

[0085] S204: Calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data;

[0086] S205: Positioning the high-voltage line in the sample sequence images of each preset time period to obtain position information of the high-voltage line in the corresponding sample sequence images;

[0087] S206: generating a cluster trajectory using the position information based on a clustering algorithm, and smoothing the cluster trajectory based on a Kalman filter to obtain motion trajectory data of the high-voltage line in each preset time period;

[0088] S207: Determine a training data set based on the motion trajectory data, and train a preset spiking neural network based on the training data set to obtain a trained spiking neural network;

[0089] S208: Inputting the swing frequency data and the depth of field reconstruction sequence image data into a trained spiking neural network to predict the motion trajectory of the high-voltage line, thereby obtaining motion trajectory prediction information;

[0090] S209: Determine the posture information of the UAV based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the posture information.

[0091] In an embodiment of the present invention, imaging processing is performed on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by a docking controller, thereby obtaining an image with a high signal-to-noise ratio and no blur, thereby achieving a significantly improved visual effect. Clarity compensation is performed on the sequence of images, and depth of field reconstruction is performed on the clarity-compensated sequence of images based on phase analysis to calculate the swing frequency data of the high-voltage wire. Clarity compensation can further improve the clarity of image details, and depth of field reconstruction of the sequence of image data can more quickly and accurately obtain the swing frequency data of the high-voltage wire in the image. Based on the swing frequency data and depth of field reconstruction of the sequence of image data, a pulse neural network is used to predict the motion trajectory of the high-voltage wire, thereby achieving fast and accurate motion trajectory prediction. The motion trajectory prediction information is used to determine the position information of the drone to control the real-time imaging robot to dock with the high-voltage wire. This can improve the accuracy and efficiency of the drone's position analysis, greatly improve the docking accuracy while ensuring the docking efficiency of the real-time imaging robot and the high-voltage wire, and achieve reliable deployment of the real-time imaging robot.

[0092] Example 3

[0093] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a docking device between a real-time imaging robot and a high-voltage line in an embodiment of the present invention, the device comprising:

[0094] Imaging processing module 31: used to perform imaging processing on the high-voltage line based on a plurality of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage line when it is swinging;

[0095] The clarity compensation module 32 is used to perform clarity compensation on the sequence of images to obtain the sequence of images after clarity compensation;

[0096] Depth of field reconstruction module 33: used for performing depth of field reconstruction on the clarity compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data;

[0097] Swing frequency calculation module 34: used for calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data;

[0098] Motion trajectory prediction module 35: used to predict the motion trajectory of the high-voltage line using a spiking neural network based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information;

[0099] The docking module 36 is used to determine the posture information of the UAV based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the posture information.

[0100] In the specific implementation process of the present invention, the specific implementation method of the device item can refer to the implementation method of the above-mentioned method item, which will not be repeated here.

[0101] In an embodiment of the present invention, imaging processing is performed on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by a docking controller, thereby obtaining an image with a high signal-to-noise ratio and no blur, thereby achieving a significantly improved visual effect. Clarity compensation is performed on the sequence of images, and depth of field reconstruction is performed on the clarity-compensated sequence of images based on phase analysis to calculate the swing frequency data of the high-voltage wire. Clarity compensation can further improve the clarity of image details, and depth of field reconstruction of the sequence of image data can more quickly and accurately obtain the swing frequency data of the high-voltage wire in the image. Based on the swing frequency data and depth of field reconstruction of the sequence of image data, a pulse neural network is used to predict the motion trajectory of the high-voltage wire, thereby achieving fast and accurate motion trajectory prediction. The motion trajectory prediction information is used to determine the position information of the drone to control the real-time imaging robot to dock with the high-voltage wire. This can improve the accuracy and efficiency of the drone's position analysis, greatly improve the docking accuracy while ensuring the docking efficiency of the real-time imaging robot and the high-voltage wire, and achieve reliable deployment of the real-time imaging robot.

[0102] An embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for docking a real-time imaging robot with a high-voltage power line according to any of the above-described embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, a storage device includes any medium that can store or transmit information in a readable form by a device (e.g., a computer or mobile phone), and can be a read-only memory, a disk, or an optical disk.

[0103] Example 4

[0104] See also Figure 4 , Figure 4 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0105] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. It will be understood by those skilled in the art that Figure 3The electronic devices shown do not constitute a limitation on all devices and may include more or fewer components than shown, or combinations of certain components. The memory 41 can be used to store the computer program 42 and various functional modules, and the processor 43 runs the computer program 42 stored in the memory 41, thereby executing various functional applications and data processing of the device. The memory can be internal memory or external memory, or include both internal memory and external memory. The internal memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. The external memory can include a hard disk, floppy disk, ZIP disk, USB flash drive, magnetic tape, etc. The processor 43 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, a single-chip microcomputer, or processor 43, or any conventional processor. The processor and memory disclosed in the present invention include but are not limited to these types of processors and memories. The processor and memory disclosed in the present invention are only examples and not limitations.

[0106] As an embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and are configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the docking method between the real-time imaging robot and the high-voltage line in any of the above-mentioned embodiments. For the specific implementation process, please refer to the above-mentioned embodiments and will not be repeated here.

[0107] In an embodiment of the present invention, imaging processing is performed on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by a docking controller, thereby obtaining an image with a high signal-to-noise ratio and no blur, thereby achieving a significantly improved visual effect. Clarity compensation is performed on the sequence of images, and depth of field reconstruction is performed on the clarity-compensated sequence of images based on phase analysis to calculate the swing frequency data of the high-voltage wire. Clarity compensation can further improve the clarity of image details, and depth of field reconstruction of the sequence of image data can more quickly and accurately obtain the swing frequency data of the high-voltage wire in the image. Based on the swing frequency data and depth of field reconstruction of the sequence of image data, a pulse neural network is used to predict the motion trajectory of the high-voltage wire, thereby achieving fast and accurate motion trajectory prediction. The motion trajectory prediction information is used to determine the position information of the drone to control the real-time imaging robot to dock with the high-voltage wire. This can improve the accuracy and efficiency of the drone's position analysis, greatly improve the docking accuracy while ensuring the docking efficiency of the real-time imaging robot and the high-voltage wire, and achieve reliable deployment of the real-time imaging robot.

[0108] In addition, the above is a detailed introduction to a method for docking a real-time imaging robot with a high-voltage line and related devices provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for docking a real-time imaging robot with a high-voltage line, characterized in that: The method comprises: Based on a plurality of pulse visual signals collected by the docking controller, the high-voltage wire on the tension clamp to be inspected is imaged and processed to obtain a sequence of images of the high-voltage wire when it is swinging; Performing clarity compensation on the sequence of images to obtain a sequence of images after clarity compensation; Performing depth of field reconstruction on the clarity compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data; Calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data; Based on the swing frequency data and the depth of field reconstruction sequence image data, a pulse neural network is used to predict the motion trajectory of the high-voltage line to obtain motion trajectory prediction information; The position information of the UAV is determined based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the position information.

2. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The method performs imaging processing on the high-voltage wire on the tension clamp to be inspected based on a plurality of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage wire when it is swinging, including: Merging a plurality of pulse visual signals to obtain a merged signal sequence, and performing abnormality detection and correction on the merged signal sequence to generate a reconstructed image sequence; Determining a motion trajectory of each pixel point based on the relative motion between the reconstructed image sequences, and generating an updated pixel point light intensity value based on the motion trajectory of each pixel point using an autoregressive model; The high-voltage line is imaged based on the updated pixel point light intensity value in combination with the reconstructed image sequence to obtain a sequence of images of the high-voltage line when it is swinging.

3. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The performing clarity compensation on the sequence of images to obtain the clarity-compensated sequence of images includes: Perform edge sharpening and histogram equalization processing on the sequence images to obtain equalized sequence images; Determine the color adjustment parameters of the pixel points based on the clarity detail information in the equalized sequence images; Performing pixel noise filtering on the equalized sequence images to obtain a noise-filtered sequence image, and determining transparency adjustment values of the pixels in the noise-filtered sequence image; Based on the transparency adjustment value and the color adjustment parameter, a guided filtering algorithm is used to perform clarity compensation on the noise-filtered sequence images to obtain clarity-compensated sequence images.

4. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The depth of field reconstruction of the clarity compensated sequence images based on phase analysis to obtain depth of field reconstructed sequence image data includes: Calculating the phase distribution in the clarity-compensated sequence of images, and determining corresponding target three-dimensional mapping coefficients using a three-dimensional mapping coefficient table based on the phase distribution; Based on the particle swarm optimization algorithm-back propagation neural network, the depth of field of the sequence images after clarity compensation is restored to obtain the sequence images after depth of field restoration; Perform depth detection and foreground prediction on the sequence images after depth of field restoration to obtain a depth estimation map and a foreground probability map; Depth of field reconstruction is performed based on the target three-dimensional mapping coefficient using the depth estimation map and the foreground probability map to obtain depth of field reconstruction sequence image data.

5. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The calculating of the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data comprises: Extract the contour of the high-voltage line based on the depth of field reconstruction sequence image data to obtain corresponding contour data; Calculating displacement change data corresponding to contour data in depth of field reconstruction sequence image data; The swing frequency data of the high-voltage line is calculated based on the displacement change data using bending curve analysis.

6. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The method of using a spiking neural network to predict the motion trajectory of the high-voltage line based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information includes: Positioning the high-voltage line in the sample sequence images of each preset time period to obtain position information of the high-voltage line in the corresponding sample sequence images; Generating cluster trajectories using the position information based on a clustering algorithm, and smoothing the cluster trajectories based on a Kalman filter to obtain motion trajectory data of the high-voltage line in each preset time period; Determining a training data set based on the motion trajectory data, and training a preset spiking neural network based on the training data set to obtain a trained spiking neural network; The swing frequency data and depth of field reconstruction sequence image data are input into a trained pulse neural network to predict the motion trajectory of the high-voltage line, thereby obtaining motion trajectory prediction information.

7. The method for docking a real-time imaging robot with a high-voltage line according to claim 1, characterized in that: The method of determining the posture information of the UAV based on the motion trajectory prediction information, and controlling the real-time imaging robot to perform docking processing with the high-voltage line based on the posture information by the UAV, includes: Estimating the future position of the high-voltage line based on the motion trajectory prediction information to obtain future position estimation information, and determining the position and posture information of the UAV based on the future position estimation information; A swing control signal of the UAV is determined based on the posture information, so that the UAV controls the real-time imaging robot to perform docking processing with the high-voltage line based on the swing control signal.

8. A docking device for a real-time imaging robot and a high-voltage line, characterized in that: The device comprises: Imaging processing module: used to perform imaging processing on the high-voltage line based on a number of pulse visual signals collected by the docking controller to obtain a sequence of images of the high-voltage line when it is swinging; A clarity compensation module is used to perform clarity compensation on the sequence of images to obtain a sequence of images after clarity compensation; Depth of field reconstruction module: used to reconstruct the depth of field of the sequence images after clarity compensation based on phase analysis to obtain depth of field reconstruction sequence image data; Swing frequency calculation module: used for calculating the swing frequency data of the high-voltage line based on the depth of field reconstruction sequence image data; A motion trajectory prediction module is configured to predict the motion trajectory of the high-voltage line using a pulse neural network based on the swing frequency data and the depth of field reconstruction sequence image data to obtain motion trajectory prediction information; Docking module: used to determine the posture information of the UAV based on the motion trajectory prediction information, so that the UAV controls the real-time imaging robot to dock with the high-voltage line based on the posture information.

9. An electronic device comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the method for docking a real-time imaging robot and a high-voltage line according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method for docking a real-time imaging robot with a high-voltage line according to any one of claims 1 to 7.

Images