Unmanned aerial vehicle for power transmission line inspection and target grabbing method thereof

Abstract

The invention relates to the technical field of target recognition and capture, in particular to an unmanned aerial vehicle for power transmission line inspection and a target capture method thereof, and the method comprises the steps: determining a second interference degree through analyzing the dispersion degree and energy distribution of the change trend of pixel values in pixel point neighborhoods in different regions in an inspection graph, so as to obtain a stable frame; on the basis of the difference between the discrete degrees of pixel values of all pixel points in neighborhoods of different edge points in each stable frame and the curvature mean value of all the edge points on each edge contour, and in combination with the difference between the optical flow vectors of all the edge points in each stable frame, determining a characteristic value so as to obtain a positioning map; and the foreign matters on the power transmission line are grabbed. The invention aims to solve the interference of the vibration of the unmanned aerial vehicle on the visual feature extraction of the foreign matters on the power transmission line, reduce the false detection and missing detection risks of vegetation tree shielding on the foreign matters on the power transmission line, and improve the positioning precision in the target grabbing process.

Description

Technical Field

[0001] The present application relates to the technical field of target recognition and capture, and specifically to a drone for power transmission line inspection and a target capture method thereof. Background Art

[0002] With the continuous development of power systems and the advancement of intelligent trends, the inspection and maintenance of transmission lines have become more complicated and dangerous. Transmission line inspections can be completed by drones, and power equipment can be accurately located through target capture technology. Compared with traditional transmission line inspection methods, it can quickly cover a larger area of transmission lines. On the basis of improving inspection efficiency, drone inspections can more accurately detect and locate transmission line faults, effectively ensuring the safety risks of power system inspection personnel and saving power operation and maintenance costs.

[0003] During autonomous drone inspections of power transmission lines, drones can capture vegetation, obstacles, foreign objects, wires, and other equipment within the transmission line corridor to conduct maintenance inspections. The key to target capture lies in the precise positioning of the target's visual features. Existing visual inspection algorithms for power transmission line inspections often target specific features, making them non-scalable. The drone's own flight vibrations can hinder the extraction of visual features. Furthermore, in mountainous areas or areas with dense high-voltage power lines, obstruction by trees and densely populated high-voltage lines can easily lead to false or missed detections of unusual targets, resulting in inaccurate positioning during target capture. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a UAV for power transmission line inspection and a target capture method thereof. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a target capture method for a drone for power transmission line inspection, the method comprising the following steps:

[0006] Real-time acquisition of inspection images of transmission lines during drone inspections;

[0007] Each inspection map is divided into multiple image blocks, and the degree of chaos in the gradient direction and the degree of discreteness of the gradient amplitude of all pixels in each image block are analyzed to determine the first interference degree of each image block. The second interference degree of each image block in each inspection map is determined by combining the difference between the mean energy of all high-frequency components and the mean energy of all low-frequency components in the frequency domain of each image block in each inspection map, as well as the similarity of all high-frequency component energies between each image block and all other image blocks. The second interference degree and the degree of chaos of all image blocks in each inspection map are combined to determine the comprehensive interference degree of each inspection map, so as to screen out stable frames from the inspection map.

[0008] Obtain the optical flow vector of each edge point on all edge contours in each stable frame, determine the edge distribution value of each stable frame based on the difference between the discrete degrees of pixel values of all pixels in the neighborhood of different edge points in each stable frame, and the mean curvature of all edge points on each edge contour, and determine the characteristic value of each stable frame based on the difference between the optical flow vectors of all edge points in each stable frame;

[0009] Based on the characteristic values, a positioning map is screened out from all stable frames to capture foreign objects on the transmission line.

[0010] Preferably, the first interference degree of each image block is a result of forward fusion of the discrete degree of the gradient amplitude of all pixels in each image block in the horizontal gradient direction and the Shannon entropy of the gradient direction of all pixels.

[0011] Preferably, the method for determining the second interference level of each image block in each inspection map is:

[0012] Calculate the similarity of all high-frequency component energies between each image block and the remaining image blocks in each inspection image, and record the sum of the similarities between each image block and all the remaining image blocks as the similarity sum value of each image block;

[0013] The second interference degree μ of image block j in inspection image i i,j The expression is: Where, γ i,j represents the difference between the energy mean of all high-frequency components and the energy mean of all low-frequency components in the frequency domain of image block j in inspection image i; Δb i,j Similarity sum value of image block j in inspection image i; a i,j represents the first interference degree of image block j in inspection image i; ε represents a preset constant greater than 0.

[0014] Preferably, the method for determining the comprehensive interference degree of each inspection diagram is:

[0015] The entropy weight of the second interference degree of each image block in each inspection pattern is calculated, the product of the entropy weight and the second interference degree of each image block is calculated, and the cumulative result of the products of all image blocks is used as the comprehensive interference degree of each inspection pattern.

[0016] Preferably, the step of selecting stable frames from the inspection graph includes:

[0017] During the UAV inspection process for transmission lines, the comprehensive interference degree of all inspection images within a preset time period before the current moment is used as the input of the threshold segmentation algorithm, and the segmentation threshold is output. The inspection images with a comprehensive interference degree less than the segmentation threshold are recorded as stable frames.

[0018] Preferably, the marginal distribution value of each stable frame is expressed as: Where θ u represents the marginal distribution value of the stable frame u; f u Represents the cumulative sum of the discrete degree differences of the pixel values of all pixels in the neighborhood between all edge points in the stable frame u; represents the cumulative sum of the mean values of the curvatures of all edge contours in the stable frame u.

[0019] Preferably, the expression of the characteristic value of each stable frame is: Where, ρ u represents the eigenvalue of the stable frame u; ω u Represents the cumulative sum of the differences between the optical flow vectors of all edge points in the stable frame u; norm() represents the normalization function; Indicates a preset constant greater than 0.

[0020] Preferably, the step of filtering out the positioning map from all stable frames includes:

[0021] Among the feature values of all stable frames within a preset time period before the current moment, the stable frames whose feature values are greater than a preset threshold are used as the positioning map at the current moment.

[0022] Preferably, the method of grabbing the target foreign object on the transmission line includes:

[0023] Collect a large number of foreign object images and classify and label them to establish a power transmission line foreign object database. The transmission line foreign object database stores images of various types of foreign objects and corresponding label values. The label values are used to indicate the types of foreign objects. Based on the transmission line foreign object database, a monitoring model is constructed using a neural network.

[0024] All the positioning maps at the current moment are used as the monitoring model to identify foreign objects in all the positioning maps at the current moment, and the position coordinates of each foreign object and the position of the drone and the position of the drone's mechanical arm at the current moment are obtained. The deviation between the position coordinates of each foreign object and the position of the drone, as well as the deviation between the position coordinates of the foreign object and the position of the drone's mechanical arm are calculated respectively. All the deviations of each foreign object are used as the input of the PID controller, and the control signal is input to capture the foreign object. In the second aspect, the embodiment of the present application also provides a drone for power transmission line inspection, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the target capture method of the drone for power transmission line inspection described in any one of the above items are implemented.

[0025] This application has at least the following beneficial effects:

[0026] The present application constructs a second interference degree by analyzing the discrete degree of the pixel value change trend in the neighborhood of pixel points in different areas of the inspection map, and combining the energy distribution of different areas in the frequency domain. The stable frames screened out from the inspection image based on the second interference degree can effectively prevent the visual features of the transmission line from being affected by the vibration interference of the drone itself, and greatly improve the quality of the visual feature extraction of the transmission line on the basis of reducing the computational complexity of the drone inspection map processing, thereby improving the accuracy of the subsequent positioning of foreign objects on the transmission line; further, the present application constructs a eigenvalue by analyzing the shape and texture information of the edge contour in the stable frame, and selects the stable frame from all the stable frames based on the eigenvalue. The positioning map for foreign object capture is further screened out in the fixed frame, which reduces the risk of subsequent drones misdetecting or missing abnormal targets on the transmission line due to obstruction by vegetation and trees or dense high-voltage lines during the drone transmission line inspection, and improves the accuracy of target capture and positioning operations of drones during the transmission line inspection process; this application screens out the positioning map by analyzing the distribution characteristics of pixel values of pixel points in the inspection map, effectively avoiding the interference of the drone's own vibration on the visual feature extraction of foreign objects on the transmission line, and reducing the risk of misdetection and missing detection of foreign objects on the transmission line due to obstruction by vegetation and trees, and improving the positioning accuracy during the target capture process. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 A flowchart of a method for capturing a target using a drone for power transmission line inspection and provided in accordance with an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the comprehensive interference extraction process provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] To further illustrate the technical means and effects adopted by this application to achieve the intended invention objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effects of the drone and target capture method for power transmission line inspection proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable form.

[0031] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0032] The specific scheme of the UAV for power transmission line inspection and its target capture method provided by this application is described in detail below with reference to the accompanying drawings.

[0033] The specific scheme of the UAV for power transmission line inspection and its target capture method provided by this application is described in detail below with reference to the accompanying drawings.

[0034] See also Figure 1 , which shows a flowchart of a target grabbing method for a UAV for power transmission line inspection provided by an embodiment of the present application, the method comprising the following steps:

[0035] An embodiment of the present application provides a target capture method for a UAV for power transmission line inspection. Specifically, the following UAV and target capture method for power transmission line inspection are provided. Figure 1 , the method comprises the following steps:

[0036] Step S1: Obtaining in real time the inspection map of the transmission line during the inspection process of the UAV.

[0037] During the drone inspection of transmission lines, the relevant technical indicators of the drone in this embodiment include: a robotic arm base is set on the drone body, the robotic arm has 6 degrees of freedom (DOF), the robotic arm span is ≥610mm, the drone has a rated load of ≥5Kg, the power supply is DC24V, the maximum speed of the robotic arm joint is 180° / s, the end repeatability accuracy is ±0.05mm, and a network camera is equipped to perform image acquisition tasks during transmission line training. The network camera image resolution is 640*480, and the maximum frame rate of the acquired image is 120 frames / s. In this embodiment, the network camera acquires the inspection image at a frame rate of 60 frames / s, and the network camera and robotic arm signals are transmitted via RS232 or WiFi.

[0038] The RTK positioning coordinates of the UAV and the position of the UAV's robotic arm are obtained, and the UAV is set to maintain a vertical height and be parallel to the transmission line during the transmission line inspection process, with a horizontal distance of ≥10m, and a flight speed of 4-8m / s. In complex areas, the speed is reduced to 2m / s to improve image clarity. The inspection map of the UAV facing the transmission line during the inspection process is obtained in real time through the above-mentioned network camera. In this embodiment, for the convenience of analysis, all the inspection maps obtained are converted into grayscale images, and the pixel values of the pixels involved in the following content are the grayscale values of the pixels. Furthermore, in order to prevent serious interference from external environmental noise, a filtering algorithm is used to denoise the grayscale image, and the UAV RTK positioning coordinates are synchronized with the image EXIF information to facilitate subsequent positioning. At this point, the pre-processed inspection map is obtained.

[0039] It should be noted that there are many commonly used filtering algorithms. In this embodiment, a mean filtering algorithm is used to perform noise reduction on grayscale images. In actual applications, as other implementation methods, implementers may also use other methods such as a Gaussian filtering algorithm or a median filtering algorithm. This embodiment does not impose any special restrictions on the selection of filtering algorithms.

[0040] The mean filter algorithm is a well-known technology, and its specific principle for image denoising is not described in detail here.

[0041] Step S2: Divide each inspection map into multiple image blocks, analyze the degree of chaos in the gradient direction of all pixels in each image block and the degree of discreteness of the gradient amplitude, determine the first interference degree of each image block, and combine the difference between the mean energy of all high-frequency components and the mean energy of all low-frequency components in the frequency domain of each image block in each inspection map, as well as the similarity of all high-frequency component energies between each image block and the remaining image blocks to determine the second interference degree of each image block in each inspection map; comprehensively analyze the second interference degrees and chaos degrees of all image blocks in each inspection map to determine the comprehensive interference degree of each inspection map, so as to screen out stable frames from all inspection maps.

[0042] During the transmission line inspection process, the UAV's own vibration and its flight trajectory parallel to the transmission line will cause difficulties in extracting visual features in the inspection map. That is, the UAV's vibration may cause moiré fringes to appear in the inspection map, reducing the accuracy of transmission line feature extraction and affecting the target positioning and capture during the UAV transmission line inspection process. At the same time, the UAV's own vibration during flight parallel to the transmission line may also aggravate the motion blur in the inspection map, causing the transmission line image features in the inspection map to show irregular distortion.

[0043] When the quality of the inspection map is poor due to the drone's own vibration during the transmission line inspection process, the difficulty of extracting visual features for user target capture increases, the moiré fringes in the inspection map become more obvious, the periodicity of the high-frequency components in the inspection map's corresponding spectrum becomes stronger, and the overall image contrast of the inspection map decreases due to motion blur, and the image gradient changes become blurred.

[0044] Therefore, based on the above analysis, this embodiment screens out stable frames from the inspection map by analyzing the distribution of pixel values in the inspection map, and uses them as reference images for subsequent target capture. The specific process is as follows:

[0045] (1) Each inspection map is divided into multiple image blocks. Specifically, in this embodiment, each inspection map is divided into N non-overlapping image blocks of the same area. Fast Fourier transform is used to obtain the representation of all image blocks in the inspection map in the frequency domain. The frequency components of the image blocks in the frequency domain are arranged in descending order. The first 5% of the frequency components in the arrangement result are used as the high-frequency components of each image block in the frequency domain, and the rest are low-frequency components. The mean energy of all low-frequency components of each image block in the frequency domain, i.e., the mean low-frequency component energy, is calculated.

[0046] (2) Further, the degree of disorder of the gradient directions of all pixels in each image block and the degree of discreteness of the gradient amplitude are analyzed to determine the first interference degree of each image block, specifically:

[0047] The result of the forward fusion of the discrete degree of the gradient amplitude of all pixels in each image block in the horizontal gradient direction and the Shannon entropy of the gradient direction of all pixels is used as the first interference degree of each image block.

[0048] It should be understood that forward fusion refers to combining two or more indicators through addition or multiplication to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations and can also include more complex statistical models and analysis methods. Implementers can choose according to their specific circumstances and this embodiment does not impose any special restrictions.

[0049] Preferably, in this embodiment, the multiplication result of the discrete degree of the gradient amplitude of all pixels in each image block in the horizontal gradient direction and the Shannon entropy of the gradient direction of all pixels is used as the first interference degree of each image block, which is used to characterize the obvious condition of the moiré fringes in each image block in the inspection image. The larger the first interference degree, the more obvious the moiré fringes and the worse the accuracy of locating foreign objects on the transmission cable.

[0050] It should be noted that there are many methods for measuring the degree of discreteness of a set of data. In this embodiment, the variance of the gradient amplitude of all pixels in each image block in the horizontal gradient direction is used as the degree of discreteness of the gradient amplitude of all pixels in each image block in the horizontal gradient direction. In actual application, as other implementation methods, implementers may also use other methods for measuring the degree of discreteness of data, such as standard deviation or dispersion coefficient. This embodiment does not impose any special restrictions on the selection of methods for measuring the degree of discreteness of data.

[0051] It is supplemented that, in this embodiment, variance is used to measure the degree of dispersion of data.

[0052] Among them, the calculation method of Shannon entropy is a well-known technology, and its specific calculation process is not repeated here. In this embodiment, the Sobel operator is used to obtain the gradient direction of each pixel point in each image block and the gradient amplitude of the pixel point in the horizontal gradient direction. The process of using the Sobel operator to obtain the gradient direction and gradient amplitude is a well-known technology, and its specific acquisition process is not repeated here.

[0053] (3) Further, based on the first interference degree of each image block, and in combination with the difference between the mean energy of all high-frequency components and the mean energy of all low-frequency components in the frequency domain of each image block in each inspection pattern, and the similarity of all high-frequency component energies between each image block and the remaining image blocks, the second interference degree of each image block in each inspection pattern is determined, specifically:

[0054] In this embodiment, the similarity of all high-frequency component energies between each image block and the remaining image blocks in each inspection image is calculated, and the sum of the similarities between each image block and all the remaining image blocks is recorded as the similarity sum value of each image block;

[0055] The second interference degree μ of image block j in inspection image i i,j The expression is: Where, γ i,j represents the difference between the energy mean of all high-frequency components and the energy mean of all low-frequency components in the frequency domain of image block j in inspection image i; Δb i,j Similarity sum value of image block j in inspection image i; a i,j represents the first interference degree of image block j in inspection image i; ε represents a preset constant greater than 0, which is used to prevent the denominator from being 0. The value of ε is set manually. In this embodiment, the value of ε is 0.01. Under the premise of ensuring that the denominator is not 0 and does not excessively affect the calculation results, the implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions.

[0056] It should be noted that there are many methods for measuring the differences between data. In this embodiment, the absolute value of the difference between the energy mean of all high-frequency components and the energy mean of all low-frequency components of image block j in the inspection image i in the frequency domain is taken as the difference between the energy mean of all high-frequency components and the energy mean of all low-frequency components of image block j in the inspection image i in the frequency domain. In actual application, the implementer may also use other methods to measure the difference between data, such as the square or ratio of the difference, based on the specific situation. This embodiment does not impose any special restrictions on the selection of methods for measuring the differences between data.

[0057] It is additionally noted that, in this embodiment, whenever differences between calculated data are involved, the method of taking the absolute value of the difference is adopted.

[0058] In addition, it should be understood that there are many methods for measuring the similarity between data groups. In this embodiment, the cosine similarity of all high-frequency component energies between each image block in each inspection image and the remaining image blocks is used as the similarity of all high-frequency component energies between each image block in each inspection image and the remaining image blocks. In actual application, as other implementation methods, the implementer may also use other methods for measuring the similarity between data groups, such as the reciprocal of the Euclidean distance. This embodiment does not impose any special restrictions on the selection of methods for measuring the similarity between data groups.

[0059] The calculation method of cosine similarity is a well-known technology, and its specific calculation process will not be described in detail.

[0060] Furthermore, according to the second interference degree of each image block in each inspection image, it can be understood that the second interference degree reflects the blurred image quality of the inspection image caused by the vibration of the drone itself during the transmission line inspection process, and the first interference degree reflects the obvious moiré fringes within the range of each image block in the inspection image. The larger the first interference degree, the more obvious the moiré fringes, and the worse the accuracy of locating foreign objects on the transmission cable. When the inspection image is more seriously affected by the vibration of the drone itself and the flight vibration parallel to the transmission line, the greater the difference in high-frequency energy between different image blocks in the inspection image, that is, the smaller the similarity sum value, and the greater the difference between the high-frequency frequency component and the low-frequency frequency component in the image block, the greater the final second interference degree, indicating that the drone's own vibration has a greater impact on the image clarity.

[0061] On the contrary, if the first interference degree is smaller, the moiré fringes are less obvious, and the accuracy of locating foreign objects on the transmission cable is better; when the inspection image is slightly affected by the vibration of the drone itself and the flight vibration parallel to the transmission line, the smaller the high-frequency energy difference between different image blocks in the inspection image, that is, the larger the similarity sum value, and the smaller the difference between the high-frequency component and the low-frequency component in the image block, the smaller the final second interference degree is, indicating that the vibration of the drone itself interferes less with the image clarity.

[0062] (4) Further, the second interference degree and the degree of confusion of all image blocks in each inspection map are integrated to determine the comprehensive interference degree of each inspection map, specifically:

[0063] In this embodiment, the entropy weight of the second interference degree of each image block in each inspection map is calculated, the product of the entropy weight and the second interference degree of each image block is calculated, and the cumulative result of the products of all image blocks is used as the comprehensive interference degree of each inspection map.

[0064] The calculation method of entropy weight is a well-known technology, and its specific calculation process will not be repeated here.

[0065] Preferably, the schematic diagram of the comprehensive interference extraction process provided in this embodiment is as follows: Figure 2 shown.

[0066] (5) Based on the comprehensive interference degree of each inspection map, stable frames are screened out from the inspection map. Specifically, during the inspection process of the UAV facing the transmission line, the comprehensive interference degree of all inspection maps within the preset time period before the current moment is used as the input of the threshold segmentation algorithm, and the segmentation threshold is output. The inspection map with a comprehensive interference degree less than the segmentation threshold is recorded as a stable frame, which is used to represent the inspection map that is less disturbed by the UAV vibration or is not disturbed by the UAV vibration.

[0067] It is additionally noted that the value of the preset time length is manually set. In this embodiment, the value of the preset time length is 1s. The implementer may also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0068] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the maximum inter-class variance algorithm is used to obtain the abnormal threshold of the comprehensive interference degree. In actual application, as other implementation methods, implementers can also adopt other methods. Regarding the selection of threshold segmentation algorithm, this embodiment does not impose any special restrictions.

[0069] Among them, the maximum inter-class variance algorithm is a well-known technology, and its specific principle is not repeated here.

[0070] Thus, by analyzing the distribution of pixels in the inspection map and the changing trend of pixel values, stable frames are selected from the inspection map within a preset time period before the current moment.

[0071] Step S3: Obtain the optical flow vector of each edge point on all edge contours in each stable frame, determine the edge distribution value of each stable frame based on the difference between the discrete degrees of pixel values of all pixels in the neighborhood of different edge points in each stable frame, and the mean curvature of all edge points on each edge contour, and determine the characteristic value of each stable frame in combination with the difference between the optical flow vectors of all edge points in each stable frame.

[0072] During the drone inspection of transmission lines, dense vegetation in mountainous areas may block the drone inspection map from certain angles, or even cover the visual features of the transmission lines, making it difficult to clearly capture the transmission line status based on the inspection map; and in areas with dense high-voltage lines, multiple transmission lines are intertwined, and it is difficult to accurately distinguish and identify each transmission line at certain drone flight angles. Accurate target positioning cannot be provided during the drone target capture process. In severe cases, inaccurate target capture and positioning may cause the drone to break through the safe distance between it and the transmission line, causing damage to the transmission line and the drone equipment.

[0073] When the shooting angle changes during the UAV transmission line inspection, the more serious the occlusion of the visual features of the transmission line by mountain vegetation or dense high-voltage lines, the more regular the edge contour of the transmission line becomes. The edge contour at the break point causes obvious jagged gradient changes due to occlusion by vegetation or dense high-voltage lines; the feature loss or optical flow vector confusion caused by occlusion between stable frames within the time segment is aggravated.

[0074] Based on the above analysis, this embodiment determines the feature value of the stabilized frame by analyzing the shape and texture features of the edge contour in the stabilized frame. The specific process is as follows:

[0075] (1) Obtain the optical flow vector of each edge point on all edge contours in each stable frame, specifically:

[0076] The stable frame is used as the input of the edge detection algorithm, all edge contours in the stable frame are extracted, and all pixel points on the edge contours are recorded as edge points.

[0077] It should be noted that there are many commonly used edge detection algorithms. In this embodiment, the Canny edge detection algorithm is used to obtain the edge contour in the stable frame. In actual application, the implementer may also use other methods such as the Sobel operator. Regarding the selection of the edge detection algorithm, this embodiment does not impose any special restrictions.

[0078] Furthermore, an optical flow tracking algorithm is used to obtain the optical flow vector of each edge point in the stable frame. There are many commonly used optical flow tracking algorithms. In this embodiment, the Farneback dense optical flow field algorithm is used to obtain the optical flow vector of the edge point. In actual application, as other implementation methods, the implementer may also use other optical flow tracking algorithms such as the Lucas-Kanade algorithm. Regarding the selection of the optical flow tracking algorithm, this embodiment does not impose any special restrictions.

[0079] Among them, the Canny edge detection algorithm and the Farneback dense optical flow algorithm are both well-known technologies, and their specific principles are not described in detail here.

[0080] (2) Further, based on the difference between the discrete degrees of pixel values of all pixels in the neighborhood of different edge points in each stable frame and the mean curvature of all edge points on each edge contour, the edge distribution value of each stable frame is determined, specifically:

[0081] The marginal distribution value θ of the stable frame u u The expression is: Where, f u Represents the cumulative sum of the discrete degree differences of the pixel values of all pixels in the neighborhood between all edge points in the stable frame u; represents the cumulative sum of the mean values of the curvatures of all edge contours in the stable frame u.

[0082] It should be noted that the neighborhood of the edge point in this embodiment is: an M×M window constructed with the edge point as the center, where the value of M is manually set. In this embodiment, the value of M is 5. The implementer can also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0083] According to the edge distribution value of each stable frame, it can be understood that the edge distribution value reflects the regularity of the transmission line contour in the stable frame and the sawtooth gradient change caused by occlusion. The larger the edge distribution value, the greater the possibility and degree of interference of the transmission line by vegetation, the greater the degree of change of the edge contour grayscale value, the greater the curvature mean of the edge contour in the corresponding stable frame, and the greater the distribution difference of the pixel values of the pixels in the neighborhood corresponding to the edge point, that is, the greater the difference in the discrete degree of the pixel values of all pixels in the neighborhood between the edge points;

[0084] On the contrary, the smaller the edge distribution value, the smoother and more regular the outline of the transmission line is, without obvious interference or occlusion. The corresponding curvature mean is also smaller, indicating that the degree of curvature of the edge outline is lower, the corresponding outline is straighter and smoother, and the grayscale value of the corresponding edge outline does not change dramatically, indicating that the possibility of vegetation occlusion is smaller. The transmission line outline in the UAV inspection image is clearer and regular, with less external interference and higher image quality, which is beneficial for subsequent image analysis and processing.

[0085] (3) Furthermore, the edge distribution value of each stable frame is combined with the difference between the optical flow vectors of all edge points in each stable frame to determine the characteristic value of each stable frame, specifically:

[0086] As an implementation method, in this embodiment, the characteristic value ρ of the stable frame u u The expression is: Where, ω u Represents the cumulative sum of the differences between the optical flow vectors of all edge points in the stable frame u; norm() represents the normalization function; Indicates a constant greater than 0 to prevent the denominator from being 0. The value of is set artificially. The value of is 0.01. Under the premise of ensuring that the denominator is not 0 and does not excessively affect the calculation results, the implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions.

[0087] According to the eigenvalues of each stable frame, it can be understood that the eigenvalue reflects the occlusion of the drone's stable frame by mountain vegetation or dense high-voltage power lines due to the shooting angle. It can characterize the accuracy of visual feature extraction when the stable frame is used as the drone's target capture and positioning frame. The larger the eigenvalue, the less likely the power line in the stable frame is to be blocked, the higher the accuracy of foreign object positioning, the smaller the difference between the optical flow vectors of the corresponding edge points, and the less likely and the less likely the power line is to be interfered with by vegetation, that is, the smaller the edge distribution value.

[0088] On the contrary, if the edge distribution value is larger, it means that the transmission line is more likely to be blocked by vegetation. At the same time, if the difference in optical flow vectors between different edge points is larger, it means that the optical flow vector chaos is more obvious, the accuracy of foreign object positioning is lower, and the final eigenvalue is smaller, which means that the transmission line is more likely to be interfered with by vegetation.

[0089] At this point, by analyzing the differences in the discrete degree of pixel values between local areas of different edge points and combining them with the curvature at the edge points, the characteristic values are determined to determine the possibility and degree of obstruction of the transmission line.

[0090] Step S4: Based on the characteristic values, a positioning map is screened out from all stable frames to capture foreign objects on the transmission line.

[0091] Based on the feature values obtained in step S3, a positioning map is selected from all stable frames for positioning. Specifically:

[0092] Among the feature values of all stable frames within a preset time period before the current moment, the stable frame with a feature value greater than the preset threshold is used as the positioning map at the current moment. It is believed that the image quality of the drone inspection is good and can effectively extract the visual features of the transmission line, which can provide precise positioning for subsequent drones to identify abnormal conditions of the transmission line and perform target capture operations.

[0093] It should be noted that the value of the preset threshold is set manually. In this embodiment, the value of the preset threshold is 0.6. The implementer can also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0094] Furthermore, a large number of foreign object images are collected and classified and labeled to establish a power transmission line foreign object database. The power transmission line foreign object database stores images of various types of foreign objects and corresponding label values, and the label values are used to indicate the types of foreign objects. Based on the power transmission line foreign object database, a monitoring model is constructed using a neural network.

[0095] It should be noted that there are many commonly used neural networks. In this embodiment, a fast convolutional neural network (Faster R-CNN) is used. In actual application, as other implementation methods, the implementer may also use a YOLO series neural network. Regarding the selection of the neural network, this embodiment does not impose any special restrictions.

[0096] Among them, the training principle of the fast convolutional neural network is a well-known technology, and its specific training process will not be repeated here.

[0097] Furthermore, all the positioning maps at the current moment are used as the monitoring model to identify foreign objects in all the positioning maps at the current moment, and the position coordinates of each foreign object are obtained. The deviation between the position coordinates of each foreign object and the position of the drone obtained in step S1, as well as the deviation between the position coordinates of the foreign object and the position of the drone's mechanical arm obtained in step S1, are calculated. All the deviations of each foreign object are used as inputs to a PID controller, and a control signal is input to grasp the foreign object. The working principle of the PID controller is well known, and its specific operation process is not repeated here.

[0098] At this point, this embodiment screens out a positioning map by analyzing the distribution characteristics of the pixel values of the pixel points in the inspection map, effectively avoiding the interference of the drone's own vibration on the visual feature extraction of foreign objects on the transmission line, and reducing the risk of false detection and missed detection of foreign objects on the transmission line due to obstruction by vegetation and trees, thereby improving the positioning accuracy during the target capture process.

[0099] Based on the same inventive concept as the above method, an embodiment of the present application also provides a drone for transmission line inspection, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned target grasping methods for drones for transmission line inspection are implemented.

[0100] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0102] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them. Modifications to the technical solutions described in the aforementioned embodiments, or equivalent replacements of some of the technical features therein, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A target grabbing method for UAVs used in power transmission line inspection, characterized in that: The method comprises the following steps: Real-time acquisition of inspection images of transmission lines during drone inspections; Each inspection map is divided into multiple image blocks, and the degree of chaos in the gradient direction and the degree of discreteness of the gradient amplitude of all pixels in each image block are analyzed to determine the first interference degree of each image block. The second interference degree of each image block in each inspection map is determined by combining the difference between the mean energy of all high-frequency components and the mean energy of all low-frequency components in the frequency domain of each image block in each inspection map, as well as the similarity of all high-frequency component energies between each image block and all other image blocks. The second interference degree and the degree of chaos of all image blocks in each inspection map are combined to determine the comprehensive interference degree of each inspection map, so as to screen out stable frames from the inspection map. Obtain the optical flow vector of each edge point on all edge contours in each stable frame, determine the edge distribution value of each stable frame based on the difference between the discrete degrees of pixel values of all pixels in the neighborhood of different edge points in each stable frame, and the mean curvature of all edge points on each edge contour, and determine the characteristic value of each stable frame based on the difference between the optical flow vectors of all edge points in each stable frame; Based on the characteristic values, a positioning map is screened out from all stable frames to capture foreign objects on the transmission line.

2. The target capture method for a UAV for power transmission line inspection according to claim 1, characterized in that: The first interference degree of each image block is a result of forward fusion of the discrete degree of the gradient amplitude of all pixels in each image block in the horizontal gradient direction and the Shannon entropy of the gradient direction of all pixels.

3. The target capture method for a UAV for power transmission line inspection according to claim 1, characterized in that: The method for determining the second interference degree of each image block in each inspection map is as follows: Calculate the similarity of all high-frequency component energies between each image block and the remaining image blocks in each inspection image, and record the sum of the similarities between each image block and all the remaining image blocks as the similarity sum value of each image block; The second interference degree μ of image block j in inspection image i i,j The expression is: Where, γ i,j represents the difference between the energy mean of all high-frequency components and the energy mean of all low-frequency components in the frequency domain of image block j in inspection image i; Δb i,j Similarity sum value of image block j in inspection image i; a i,j represents the first interference degree of image block j in inspection image i; ε represents a preset constant greater than 0.

4. The target capture method for a UAV for power transmission line inspection according to claim 1, characterized in that: The method for determining the comprehensive interference degree of each inspection diagram is as follows: The entropy weight of the second interference degree of each image block in each inspection pattern is calculated, the product of the entropy weight and the second interference degree of each image block is calculated, and the cumulative result of the products of all image blocks is used as the comprehensive interference degree of each inspection pattern.

5. The target capture method for a UAV for power transmission line inspection according to claim 1, characterized in that: The step of selecting stable frames from the inspection graph includes: During the UAV inspection process for transmission lines, the comprehensive interference degree of all inspection images within a preset time period before the current moment is used as the input of the threshold segmentation algorithm, and the segmentation threshold is output. The inspection images with a comprehensive interference degree less than the segmentation threshold are recorded as stable frames.

6. The target capture method for a UAV for power transmission line inspection according to claim 1, characterized in that: The expression of the edge distribution value of each stable frame is: Where θ u represents the marginal distribution value of the stable frame u; f u Represents the cumulative sum of the discrete degree differences of the pixel values of all pixels in the neighborhood between all edge points in the stable frame u; represents the cumulative sum of the mean values of the curvatures of all edge contours in the stable frame u.

7. The target capture method for a UAV for power transmission line inspection according to claim 6, characterized in that: The expression of the characteristic value of each stable frame is: Where, ρ u represents the eigenvalue of the stable frame u; ω u Represents the cumulative sum of the differences between the optical flow vectors of all edge points in the stable frame u; norm() represents the normalization function; Indicates a preset constant greater than 0.

8. The target capture method for a UAV for power transmission line inspection according to claim 5, characterized in that: The positioning map is filtered out from all stable frames, including: Among the feature values of all stable frames within a preset time period before the current moment, the stable frames whose feature values are greater than a preset threshold are used as the positioning map at the current moment.

9. The target capture method for a UAV for power transmission line inspection according to claim 8, characterized in that: The method of grabbing a target foreign object on a transmission line includes: Collect a large number of foreign object images and classify and label them to establish a power transmission line foreign object database. The transmission line foreign object database stores images of various types of foreign objects and corresponding label values. The label values are used to indicate the types of foreign objects. Based on the transmission line foreign object database, a monitoring model is constructed using a neural network. All the positioning maps at the current moment are used as the monitoring model to identify foreign objects in all the positioning maps at the current moment, and the position coordinates of each foreign object and the position of the drone and the position of the drone's robotic arm at the current moment are obtained. The deviation between the position coordinates of each foreign object and the position of the drone, as well as the deviation between the position coordinates of the foreign object and the position of the drone's robotic arm are calculated respectively. All the deviations of each foreign object are used as the input of the PID controller, and the control signal is input to grasp the foreign object.

10. A UAV for power transmission line inspection, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the target capture method for a drone for power transmission line inspection as described in any one of claims 1 to 9 are implemented.

Images