A method and system for processing inspection image data based on deep learning

Abstract

The present invention discloses a patrol image data processing method and system based on deep learning, which relates to the field of image recognition technology and comprises the following steps: acquiring a patrol image, the patrol image including a transmission line, segmenting the patrol image, and extracting a portion of the image including the transmission line; identifying edge connection features of the transmission line, identifying connection feature components based on connection feature patterns, distinguishing proximal features in the connection features, and generating a proximal feature image set; extracting features in the proximal feature image set that are in contact with the transmission line, identifying the proximal features and the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features; the present invention identifies the same obstacle feature in multiple images to determine the position of the obstacle, judge whether the obstacle will affect the transmission line, and determine the position information of the obstacle through the obstacle feature and the comparison feature, and judge whether the obstacle is within the range that affects the safety of the transmission line.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for processing inspection image data based on deep learning. Background Art

[0002] Drone inspections play an important role in many fields. Here are some of the main application scenarios:

[0003] Power industry: Drones are used for inspection of transmission lines and substations, quickly identifying equipment defects and line faults, and improving inspection efficiency and safety.

[0004] Oil and gas: Drones are used for pipeline inspections to detect leaks, corrosion, and other issues, reducing the risks of manual inspections.

[0005] Transportation: Drones are used for railway and highway inspections to detect road damage, bridge structural problems, etc., to ensure traffic safety.

[0006] Agriculture: Drones are used to inspect farmland and crops, monitor crop health and pests and diseases, and optimize agricultural management.

[0007] Buildings and Infrastructure: Drones are used to inspect buildings and infrastructure, detect structural damage and safety hazards, and ensure building safety.

[0008] The advantages of drone inspections include high efficiency, accuracy, and wide coverage, making them particularly suitable for inspection tasks in areas with complex terrain or dangerous areas.

[0009] Publication No. CN116758081A discloses a method for processing images used in drone road and bridge inspections. The method comprises the following steps: obtaining an image set based on inspection images; obtaining a target region and background region of each image, as well as several connected domains within the target region, and calculating the weight of each pixel in the target region; calculating the degree of difference between the background region and the target region of the image; obtaining an optimal segmentation template based on the degree of difference; obtaining the target region in the inspection image based on the optimal segmentation template; and performing quality inspections and safety problem investigations on roads and bridges based on the target region in the inspection image. This method ensures the accuracy of road and bridge inspection results, allowing for the timely identification of existing bridge problems and the elimination of potential safety hazards.

[0010] When drones conduct power inspections, they acquire images of transmission lines and monitor them. Image processing of transmission lines usually includes the following parts: image dehazing, which removes the impact of weather conditions on image quality; image segmentation, which extracts important parts of the image; image enhancement, which optimizes image quality and improves image readability; and image restoration, which processes images that have been degraded due to distortion, blur, or noise, and restores the original image as much as possible.

[0011] Because the drone is above or beside the transmission line during the transmission line data collection process, and in areas with lush branches and leaves in summer, it blocks the transmission line, resulting in incomplete inspection of the transmission line. Traditional image processing methods cannot determine the position of branches and leaves, and cannot determine whether the branches and leaves will affect the transmission line. Summary of the Invention

[0012] One of the objectives of the present invention is to provide a method and system for processing inspection image data based on deep learning, to identify drone inspection images, determine the impact between the location of the obstructed transmission line and the transmission line, judge whether the obstacle will affect the transmission line, and identify whether the obstacle in the image affects the inspection of the transmission line.

[0013] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for processing inspection image data based on deep learning, comprising the following steps:

[0014] Acquire an inspection image, the inspection image containing the transmission line, segment the inspection image, and extract the portion of the image containing the transmission line;

[0015] Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set;

[0016] Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features;

[0017] The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set;

[0018] Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest clarity of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and acquired at different angles from the analysis image as auxiliary images;

[0019] Calculate the distance between the obstacle feature and the transmission line based on the positional relationship between the obstacle feature and the transmission line at different angles in the analysis image and the auxiliary image;

[0020] Obtain the distance between towers, the height of the image acquisition equipment, and the distance between the obstacle feature position and the towers, establish a transmission line swing model, calculate the transmission line sag value, and calculate the transmission line swing range based on the transmission line sag value.

[0021] In one or more embodiments of the present invention, the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected is determined, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is obtained, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is determined as the analysis angle, and the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image is determined based on the analysis angle.

[0022] In one or more embodiments of the present invention, the steps for calculating the distance between the obstacle feature and the transmission line based on the analysis image and the auxiliary image are as follows:

[0023] Constructing a triangle with the image acquisition point, the obstacle feature point and the transmission line comparison point as vertices in the analysis image and the auxiliary image;

[0024] Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle I :

[0025] I =∣ I 1 -I 2|

[0026] in, I 1 and I 2 are the grayscale values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively;

[0027] Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point in the two angles D :

[0028] ;

[0029] in, B is the baseline distance between the two angles, f is the camera focal length;

[0030] Analyze the distance difference to determine the actual distance between the obstacle and the transmission line.

[0031] In one or more embodiments of the present invention, the actual distance between the obstacle and the transmission line L The calculation method is as follows:

[0032] Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Dth :

[0033] Δθ=θ 2 -θ 1;

[0034] in, i 1 and i 2 is the acquisition angle of the acquisition image points in the analysis image and auxiliary image;

[0035] Through the angle difference Dth and distance difference D Calculate actual distance L :

[0036] .

[0037] In one or more embodiments of the present invention, the steps of establishing a transmission line swing model are as follows:

[0038] Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension;

[0039] A transmission line swing model is established based on the tower spacing, transmission line weight, and initial transmission line tension to calculate the transmission line sag value;

[0040] Simulate external forces to determine the impact of wind load, ice weight, and short-circuit power on transmission lines. Use the finite element method to simulate the motion trajectory and tension changes of transmission lines under external forces.

[0041] Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

[0042] In one or more embodiments of the present invention, the transmission line sag calculation formula is as follows:

[0043] ;

[0044] in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line;

[0045] In the absence of horizontal tension in the transmission line, the verticality of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle. The horizontal tension is then determined based on the verticality of the transmission line:

[0046] .

[0047] In one or more embodiments of the present invention, the steps for analyzing the galloping characteristics of a transmission line are as follows:

[0048] A transmission line galloping model is established, and the galloping model equation is:

[0049] my+cy+ky=F(t) ;

[0050] in, m is the transmission line quality, c is the damping coefficient, k is the stiffness coefficient, y is the transmission line displacement, F(t) For external forces;

[0051] Numerical simulations are performed using the finite element method to obtain the motion trajectory and tension changes of transmission lines under different conditions;

[0052] Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations using the measured data;

[0053] Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

[0054] In one or more embodiments of the present invention, the distance between the obstacle feature and the tower closest to the obstacle feature in the analysis image is obtained, the swing amount of the transmission line at the obstacle feature position is determined, the safety distance of the transmission line installation position is obtained, and the safety range of the transmission line is determined by the swing amount and the safety distance.

[0055] In one or more embodiments of the present invention, obstacle features are compared with big data to identify obstacles represented by the obstacle features, and it is analyzed whether the obstacles will be affected by wind changes. After determining that the obstacles are affected by wind, the image acquisition device is adjusted at an angle to interfere with the obstacles, thereby changing the position of the obstacles and completing the acquisition of the transmission lines at the locations blocked by the obstacles.

[0056] The present invention also provides an inspection image data processing system, comprising:

[0057] An image processor, used for processing inspection images;

[0058] Image segmentation module, which recognizes inspection images and marks transmission lines, and segments transmission line connection features and obstacle features;

[0059] The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag and horizontal tension of the transmission line;

[0060] The dancing model calculates the motion trajectory and tension changes of transmission lines under different conditions.

[0061] Through the above technical solution, the present invention has the following beneficial effects:

[0062] 1. The present invention identifies the same obstacle features in multiple images to determine the location of the obstacle and judge whether the obstacle will affect the transmission line. It also determines the location information of the obstacle through the obstacle features and the comparison features, judges whether the obstacle is within the range that affects the safety of the transmission line, and determines whether the obstacle will affect the integrity of the transmission line inspection.

[0063] 2. Acquire multiple images containing the same obstacle, determine the analysis image and auxiliary image based on the obstacle characteristics of the obstacle, combine the analysis image and auxiliary image to determine the obstacle characteristics, and determine the difference in obstacle characteristics between the analysis image and the auxiliary image, judge the positional relationship between the obstacle and the transmission line, and determine whether the obstacle is within the safe range of the transmission line based on the positional relationship.

[0064] 3. Obtain obstacle images and identify obstacles, determine the specific status of the obstacles, and whether the status of the obstacles can be adjusted by drones. This will determine whether transmission line inspections can be carried out after the obstacle status is adjusted by drones. During the drone inspection process, the impact of obstacles on the transmission lines can be reduced, and a more complete transmission line image can be obtained.

[0065] 4. By identifying the sag changes of the transmission line in the image, the swing amount of the transmission line in different environments is determined. The swing range of the transmission line at different locations is determined based on the swing amount. The safe distance of the obstacle is obtained based on the swing range of the transmission line. The closer to the tower, the smaller the swing range of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of the processing method of the present invention;

[0067] Figure 2 is a schematic diagram of a processing system of the present invention;

[0068] Figure 3 Schematic diagram of image acquisition points for analysis images and auxiliary images of the present invention. DETAILED DESCRIPTION

[0069] The following drawings illustrate various embodiments of the present invention. For clarity, many practical details are included in the following description. However, it should be understood that these practical details are not intended to limit the present invention. In other words, in some embodiments of the present invention, these practical details are optional. Furthermore, features from different embodiments may be interchangeably applicable, where practically possible.

[0070] Unless otherwise defined, all terms used herein (including technical and scientific terms) have their ordinary meanings as understood by those skilled in the art. Furthermore, the definitions of the aforementioned terms in commonly used dictionaries should be interpreted in the context of this specification as consistent with the meanings in the art relevant to the present invention. Unless otherwise explicitly defined, these terms should not be interpreted as having idealized or overly formal meanings.

[0071] See also Figure 1-Figure 3 The present invention provides a deep learning-based inspection image data processing method for identifying UAV inspection images and determining the occlusion of transmission lines by obstacles in UAV images.

[0072] In one embodiment, the inspection image data processing method includes the following steps:

[0073] Acquire an inspection image, the inspection image containing the transmission line, segment the inspection image, and extract the portion of the image containing the transmission line;

[0074] Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set;

[0075] Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features;

[0076] The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set;

[0077] Optionally, the Brenner gradient function is calculated as follows:

[0078] ;

[0079] In the gradient function formula, I (i, j) Representing an image (i, j) Gray value at position; I (i+2, j) Representing an image （i+ 2，j） The grayscale value at position , that is, the position two pixels apart in the horizontal direction; (I(i,j)-I(i+2,j)) 2 Indicates the square of the grayscale difference between two pixels in the horizontal direction;

[0080] Indicates the image (i, j) The sum of all pixel positions is calculated. The larger the grayscale difference, the clearer the edges and details of the image, and the larger the function value.

[0081] By using grayscale differences to determine the approximate distance between the transmission line and the obstacle, the image can be further screened and re-segmented during image recognition, retaining obstacle features within the transmission line obstacle range. This allows for more accurate obstacle feature analysis in the image, reducing the impact of proximal features on obstacle feature analysis and improving image analysis speed.

[0082] Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest clarity of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and acquired at different angles from the analysis image as auxiliary images;

[0083] Calculate the distance between the obstacle feature and the transmission line based on the positional relationship between the obstacle feature and the transmission line at different angles in the analysis image and the auxiliary image;

[0084] Obtain the distance between towers, the height of the image acquisition equipment, and the distance between the obstacle feature position and the towers, establish a transmission line swing model, calculate the transmission line sag value, and calculate the transmission line swing range based on the transmission line sag value.

[0085] In this embodiment, obstacles blocking the transmission line during the inspection process prevent the drone from capturing the complete transmission line, resulting in incomplete identification of the transmission line. Obstacles include but are not limited to branches, leaves, walls and other objects, and the transmission line is at an unsafe distance from the obstacle, which can easily lead to line failure and property loss.

[0086] By using drones to automatically identify images and determine the distance between the obstacle features in the image and the transmission lines, it is possible to analyze whether the existence of the obstacle features will have an impact on the transmission lines.

[0087] Among them, since the transmission line will experience sag after being erected, it will swing under the interference of external forces. The farther the transmission line is from the tower, the larger the range of swing it can theoretically achieve. Therefore, the range of swing of the transmission line increases with the distance between the transmission line and the tower, and the safety range is calculated based on the position of the transmission line.

[0088] In one embodiment, the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected is determined, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is obtained, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is determined as the analysis angle, and the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image is determined based on the analysis angle.

[0089] In this embodiment, the image acquisition points, obstacle feature points and transmission line comparison points of the image acquisition device are inspection images acquired at different angles, and the difference in viewing angles when determining the obstacle feature points and the transmission line comparison points causes a difference in the distance between the obstacle feature and the transmission line, making it impossible to accurately calculate the distance. Therefore, when determining the obstacle feature points and the transmission line comparison points in the analysis image, the obstacle feature points and the transmission line comparison points are marked, and the obstacle feature points and the transmission line comparison points are determined in the auxiliary image, and their positions are consistent with those of the obstacle feature points and the transmission line comparison points in the analysis image.

[0090] When marking obstacle feature points and transmission line comparison points in the auxiliary image, determine and analyze the positioning features near the obstacle feature points and transmission line comparison points in the image, and determine the positions of the positioning features and the obstacle feature points and transmission line comparison points based on the acquisition angle of the image acquisition point.

[0091] In one embodiment, the steps for calculating the distance between the obstacle feature and the transmission line based on the analysis image and the auxiliary image are as follows:

[0092] Constructing a triangle with the image acquisition point, the obstacle feature point and the transmission line comparison point as vertices in the analysis image and the auxiliary image;

[0093] Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle I :

[0094] I =∣ I 1 -I 2|

[0095] in, I 1 and I 2 are the grayscale values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively;

[0096] Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point in the two angles D :

[0097] ;

[0098] in, B is the baseline distance between the two angles, f is the camera focal length;

[0099] Analyze the distance difference to determine the actual distance between the obstacle and the transmission line.

[0100] In this embodiment, the actual distance between the obstacle and the transmission line is determined by determining the distance difference at two angles and analyzing the angular difference between the image and multiple auxiliary images, thereby determining the precise distance. The grayscale difference in the multiple auxiliary images can calibrate the distance between the obstacle and the transmission line to ensure the accuracy of the distance.

[0101] Optionally, when multiple auxiliary images do not correspond to each other during distance calibration, the grayscale difference between the obstacle feature points and the transmission line comparison points in the analysis image and the multiple auxiliary images is calculated, and the total average value of the distance differences is calculated after the distance differences are calculated.

[0102] In one embodiment, the actual distance between the obstacle and the transmission line L The calculation method is as follows:

[0103] Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Dth :

[0104] Δθ=θ 2 -θ 1;

[0105] in, i 1 and i 2 is the acquisition angle of the acquisition image points in the analysis image and auxiliary image;

[0106] Through the angle difference Dth and distance difference D Calculate actual distance L :

[0107] .

[0108] In this embodiment, based on the analysis images and auxiliary images obtained from different angles, the obstacle feature points and transmission line comparison points in the image are identified, the actual distance is calculated, and the precise distance between the obstacle and the transmission line is obtained, further ensuring the accurate calculation of the distance.

[0109] Among them, since the actual distance cannot be determined through the image when the transmission line, obstacle and image acquisition equipment are in the same straight line, calculating the actual distance through multiple images at different angles can determine whether the obstacle is within the safe range of the transmission line.

[0110] In one embodiment, the steps of establishing a transmission line swing model are as follows:

[0111] Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension;

[0112] A transmission line swing model is established based on the tower spacing, transmission line weight, and initial transmission line tension to calculate the transmission line sag value;

[0113] Simulate external forces to determine the impact of wind load, ice weight, and short-circuit power on transmission lines. Use the finite element method to simulate the motion trajectory and tension changes of transmission lines under external forces.

[0114] Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

[0115] In this embodiment, by establishing a transmission line swing model and analyzing the swing range of the transmission line at different locations, the safe range of the transmission line swing at different locations can be determined. During the drone inspection process, the safe range of the transmission line at the inspected location can be determined based on the distance from the tower. Based on the identification and calculation of the distance between the obstacle and the transmission line, it is determined whether the obstacle is within the safe range of the transmission line.

[0116] In one embodiment, the transmission line sag calculation formula is as follows:

[0117] ;

[0118] in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line;

[0119] In the absence of horizontal tension in the transmission line, the verticality of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle. The horizontal tension is then determined based on the verticality of the transmission line:

[0120] .

[0121] In this embodiment, by obtaining the sag and horizontal tension of the transmission line, the swingable range of the transmission line at different positions is obtained, and then the safety range at different positions can be calculated.

[0122] In one embodiment, the steps for analyzing the galloping characteristics of a transmission line are as follows:

[0123] A transmission line galloping model is established, and the galloping model equation is:

[0124] my+cy+ky=F(t) ;

[0125] in, m is the transmission line quality, c is the damping coefficient, k is the stiffness coefficient,y is the transmission line displacement, F(t) For external forces;

[0126] Numerical simulations are performed using the finite element method to obtain the motion trajectory and tension changes of transmission lines under different conditions;

[0127] Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations using the measured data;

[0128] Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

[0129] For example, a model is established: the mass of the transmission line is m=1 kg / m, the damping coefficient is c=0.1 Ns / m, and the stiffness coefficient is k=100 N / m.

[0130] Numerical simulation: The geometric characteristics and material properties of the transmission line are input into the finite element software, and a wind load F(t) = 10 / sin(2 / pit)N is applied.

[0131] Experimental verification: Small-span tests were conducted in the laboratory to measure the vibration characteristics of the transmission line under wind load.

[0132] Analysis results: Through numerical simulation and measurement data, it is found that the swing amplitude of the transmission line is 0.5m and the frequency is 1Hz.

[0133] In this embodiment, the dancing characteristics are analyzed to determine the swing amount of the transmission line in different environments, and then the swing range of the transmission line can be determined according to the geographical location and environment of the transmission line.

[0134] In one embodiment, the distance between the obstacle feature and the tower closest to the obstacle feature in the analysis image is obtained, the swing amount of the transmission line at the obstacle feature position is determined, the safety distance of the transmission line installation position is obtained, and the safety range of the transmission line is determined based on the swing amount and the safety distance.

[0135] In this embodiment, the safety range of the transmission line is calculated, and whether there are obstacles within the safety range of the transmission line in the inspection image is identified and calculated. The obstacles within the safety range are marked to facilitate obstacle processing.

[0136] In one embodiment, obstacle features are compared with big data to identify the obstacles represented by the obstacle features, and it is analyzed whether the obstacles will be affected by wind changes. After determining that the obstacles are affected by wind, the image acquisition device is adjusted at an angle to interfere with the obstacles, thereby changing the position of the obstacles and completing the acquisition of the transmission lines at the locations blocked by the obstacles.

[0137] In this embodiment, at the location where the transmission line is installed, the obstacles that affect the transmission line are mostly leaves. During the inspection process, the drone will interfere with the leaves through wind at the characteristic position, changing the position of the leaves, so that the leaves can swing, so as to obtain a complete transmission line inspection image.

[0138] An embodiment of the present invention further provides an inspection image data processing system, comprising:

[0139] An image processor, used for processing inspection images;

[0140] Image segmentation module, which recognizes inspection images and marks transmission lines, and segments transmission line connection features and obstacle features;

[0141] The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag and horizontal tension of the transmission line;

[0142] The dancing model calculates the motion trajectory and tension changes of transmission lines under different conditions.

[0143] In summary, the technical solutions disclosed in the above embodiments of the present invention have at least the following advantages:

[0144] 1. The present invention identifies the same obstacle features in multiple images to determine the location of the obstacle and judge whether the obstacle will affect the transmission line. It also determines the location information of the obstacle through the obstacle features and the comparison features, judges whether the obstacle is within the range that affects the safety of the transmission line, and determines whether the obstacle will affect the integrity of the transmission line inspection.

[0145] 2. Acquire multiple images containing the same obstacle, determine the analysis image and auxiliary image based on the obstacle characteristics of the obstacle, combine the analysis image and auxiliary image to determine the obstacle characteristics, and determine the difference in obstacle characteristics between the analysis image and the auxiliary image, judge the positional relationship between the obstacle and the transmission line, and determine whether the obstacle is within the safe range of the transmission line based on the positional relationship.

[0146] 3. Obtain obstacle images and identify obstacles, determine the specific status of the obstacles, and whether the status of the obstacles can be adjusted by drones. This will determine whether transmission line inspections can be carried out after the obstacle status is adjusted by drones. During the drone inspection process, the impact of obstacles on the transmission lines can be reduced, and a more complete transmission line image can be obtained.

[0147] 4. By identifying the sag changes of the transmission line in the image, the swing amount of the transmission line in different environments is determined. The swing range of the transmission line at different locations is determined based on the swing amount. The safe distance of the obstacle is obtained based on the swing range of the transmission line. The closer to the tower, the smaller the swing range of the transmission line.

[0148] Although the present invention is disclosed in conjunction with the above embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the attached claims.

Claims

1. A method for processing inspection image data based on deep learning, characterized in that: The following steps are involved: Acquire an inspection image, the inspection image containing the transmission line, segment the inspection image, and extract the portion of the image containing the transmission line; Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set; Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features; The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set; Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest clarity of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and acquired at different angles from the analysis image as auxiliary images; Calculate the distance between the obstacle feature and the transmission line based on the positional relationship between the obstacle feature and the transmission line at different angles in the analysis image and the auxiliary image; Obtain the distance between towers, the height of the image acquisition device, and the distance between the obstacle feature location and the towers, establish a transmission line swing model, calculate the transmission line sag value, and calculate the transmission line swing range based on the transmission line sag value; The steps for calculating the distance between obstacle features and transmission lines based on the analysis image and auxiliary image are as follows: Constructing triangles with image acquisition points, obstacle feature points, and transmission line comparison points as vertices in the analysis image and the auxiliary image; Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle ΔI : ΔI =∣ I 1 -I 2∣； in, I 1 and I 2 are the grayscale values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively; Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point in the two angles D : ； in, B is the baseline distance between the two angles, f is the camera focal length; Analyze the distance difference to determine the actual distance between the obstacle and the transmission line; The actual distance between the obstacle and the transmission line L The calculation method is as follows: Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Δθ : Δθ=θ 2 -θ 1； in, θ 1 and θ 2 is the acquisition angle of the acquisition image points in the analysis image and auxiliary image; Through the angle difference Δθ and distance difference D Calculate actual distance L : 。 2. The inspection image data processing method based on deep learning according to claim 1, characterized in that: Determine the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected, obtain the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image, determine the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image as the analysis angle, and determine the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image based on the analysis angle.

3. The inspection image data processing method based on deep learning according to claim 2, characterized in that: The steps to establish a transmission line swing model are as follows: Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension; A transmission line swing model is established based on the tower spacing, transmission line weight, and initial transmission line tension to calculate the transmission line sag value; Simulate external forces to determine the impact of wind load, ice weight, and short-circuit power on transmission lines. Use the finite element method to simulate the motion trajectory and tension changes of transmission lines under external forces. Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

4. The inspection image data processing method based on deep learning according to claim 3 is characterized in that: The calculation formula for the sag value of transmission lines is as follows: ； in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line; In the absence of horizontal tension in the transmission line, the sag value of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle. The horizontal tension is then determined based on the sag value of the transmission line: 。 5. The inspection image data processing method based on deep learning according to claim 4 is characterized in that: The steps to analyze the galloping characteristics of transmission lines are as follows: A transmission line galloping model is established, and the galloping model equation is: my+cy+ky=F(t) ； in, m is the transmission line quality, c is the damping coefficient, k is the stiffness coefficient, y is the transmission line displacement, F(t) For external forces; Numerical simulations are performed using the finite element method to obtain the motion trajectory and tension changes of transmission lines under different conditions; Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations using the measured data; Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

6. The inspection image data processing method based on deep learning according to claim 5, characterized in that: Obtain the distance between the obstacle feature and the tower closest to the obstacle feature in the analysis image, determine the swing amount of the transmission line at the obstacle feature location, obtain the safe distance of the transmission line installation location, and determine the safety range of the transmission line based on the swing amount and the safe distance.

7. The inspection image data processing method based on deep learning according to claim 6, characterized in that: By comparing obstacle features with big data, we can identify the obstacles represented by the obstacle features and analyze whether the obstacles will be affected by wind changes. Once it is determined that the obstacles are affected by wind, we can adjust the angle of the image acquisition equipment to interfere with the obstacles, change the position of the obstacles, and complete the acquisition of the transmission lines at the locations blocked by the obstacles.

8. A patrol inspection image data processing system, used in the patrol inspection image data processing method according to any one of claims 1 to 7, characterized in that: include: An image processor, used for processing inspection images; Image segmentation module, which recognizes inspection images and marks transmission lines, and segments transmission line connection features and obstacle features; The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag value and horizontal tension of the transmission line; The dancing model calculates the motion trajectory and tension changes of transmission lines under different conditions.

Images