Patrol image data processing method and system based on deep learning

Abstract

The invention discloses an inspection image data processing method and system based on deep learning, and relates to the technical field of image recognition, and the method comprises the following steps: obtaining an inspection image including a power transmission line, segmenting the inspection image, and extracting a part of the image including the power transmission line; identifying edge connection features of the power transmission line, identifying connection feature components based on connection feature lines, distinguishing near-end features in the connection features, and generating a near-end feature image set; extracting features in contact with the power transmission line in the near-end feature image set, and identifying the definition of the near-end features and the power transmission line connected with the edge position of the near-end features; according to the method, the same obstacle feature in the multiple images is identified, the position of the obstacle is determined, whether the obstacle affects the power transmission line or not is judged, the position information of the obstacle is determined through the obstacle feature and the comparison feature, and whether the obstacle is within the range which affects the safety of the power transmission line or not is judged.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and system for processing inspection image data based on deep learning. Background Art

[0002] Drone inspection plays an important role in many fields. Here are some of the main application scenarios: Power industry: Drones are used for inspection of transmission lines and substations. They can quickly identify equipment defects and line faults, improving inspection efficiency and safety.

[0003] Oil and gas: Drones are used to inspect pipelines, detect leaks, corrosion and other problems, and reduce the risks of manual inspections.

[0004] Transportation: Drones are used for inspections of railways and roads to detect road damage, bridge structure problems, etc., to ensure traffic safety.

[0005] Agriculture: Drones are used to inspect farmland and crops, monitor crop health and pests and diseases, and optimize agricultural management.

[0006] Buildings and Infrastructure: Drones are used to inspect buildings and infrastructure, detect structural damage and safety hazards, and ensure building safety.

[0007] The advantages of drone inspections include high efficiency, accuracy, and wide coverage, and they are particularly suitable for inspection tasks in areas with complex terrain or dangerous areas.

[0008] Announcement No. CN116758081A discloses a method for processing images of road and bridge inspection by unmanned aerial vehicles, including: obtaining an image set based on inspection images; obtaining the target area and background area of ​​each image and several connected domains in the target area, and calculating the weight of each pixel in the target area; calculating the degree of difference between the background area and the target area of ​​the image; obtaining an optimal segmentation template based on the degree of difference, obtaining the target area in the inspection image based on the optimal segmentation template, and performing quality inspection and safety problem investigation on roads and bridges based on the target area in the inspection image. The present invention ensures the accuracy of inspection results of roads and bridges, timely discovers problems existing in bridges, and eliminates safety hazards.

[0009] When drones are conducting power inspections, images of transmission lines are acquired through drones to monitor the transmission lines. Image processing of transmission lines usually includes the following parts: image defogging to remove the impact of weather conditions on image quality; image segmentation to extract important parts of the image; image enhancement to optimize image quality and improve image readability; image restoration to process images that have been degraded due to distortion, blur, or noise, and to restore the source image as much as possible.

[0010] Because the drone is above or beside the transmission line during the transmission line data collection process, and in areas with lush branches and leaves in summer, the transmission line is blocked, resulting in incomplete inspection of the transmission line. Traditional image processing methods cannot determine the position of the branches and leaves, and cannot determine whether the branches and leaves will affect the transmission line. Summary of the invention

[0011] One of the purposes of the present invention is to provide a method and system for processing inspection image data based on deep learning, to identify drone inspection images, to determine the influence between the position of the obstructed transmission line and the transmission line, to judge whether the obstacle will have an impact on the transmission line, and to identify whether the obstacle in the image affects the inspection of the transmission line.

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for processing inspection image data based on deep learning, comprising the following steps: Acquire an inspection image, the inspection image includes a transmission line, segment the inspection image, and extract a portion of the image including the transmission line; Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set; Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features; The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set; Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest definition of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and at different acquisition angles from the analysis image as auxiliary images; The distance between the obstacle feature and the transmission line is calculated based on the positional relationship between the obstacle feature at different angles and the transmission line in the analysis image and the auxiliary image; The distance between towers, the height of image acquisition equipment, and the distance between the obstacle feature position and the tower are obtained, a transmission line swing model is established, the sag value of the transmission line is calculated, and the swing range of the transmission line is calculated based on the sag value of the transmission line.

[0013] In one or more embodiments of the present invention, the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected is determined, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is obtained, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is determined as the analysis angle, and the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image is determined based on the analysis angle.

[0014] In one or more embodiments of the present invention, the steps of calculating obstacle features and transmission line distance based on the analysis image and the auxiliary image are as follows: Constructing a triangle with image acquisition points, obstacle feature points and transmission line comparison points as vertices in the analysis image and the auxiliary image; Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle I : I =∣ I 1 -I 2∣; in, I 1 and I 2 are the gray values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively; Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point at two angles D : ; in, B is the baseline distance between the two angles, f is the camera focal length; Analyze the distance difference to determine the actual distance between the obstacle and the transmission line.

[0015] In one or more embodiments of the present invention, the actual distance between the obstacle and the transmission line L The calculation method is as follows: Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Dth : Δθ=θ 2 -θ 1; in, i 1 and i 2 is the acquisition angle of the acquisition image points in the analysis image and the auxiliary image; Through the angle difference Dth And the distance difference D Calculate actual distance L : .

[0016] In one or more embodiments of the present invention, the steps of establishing a transmission line swing model are as follows: Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension; A transmission line swing model is established based on the tower spacing, transmission line weight and initial tension of the transmission line to calculate the transmission line sag value; Simulate external forces to obtain the impact of wind load, ice weight and short-circuit power on the transmission line. Use the finite element method to simulate the movement trajectory and tension changes of the transmission line under the action of external forces. Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

[0017] In one or more embodiments of the present invention, the transmission line sag calculation formula is as follows: ; in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line; In the absence of horizontal tension in the transmission line, the verticality of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle, and the horizontal tension is determined based on the verticality of the transmission line: .

[0018] In one or more embodiments of the present invention, the steps of analyzing the galloping characteristics of a transmission line are as follows: The transmission line galloping model is established, and the galloping model equation is: my+cy+ky=F(t) ; in, m is the quality of the transmission line, c is the damping coefficient, k is the stiffness coefficient, y is the transmission line displacement, F(t) For external forces; Numerical simulation is performed using the finite element method to obtain the movement trajectory and tension changes of the transmission line under different conditions; Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations through measurement data; Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

[0019] In one or more embodiments of the present invention, the distance between the obstacle feature in the analysis image and the distance between the tower pole closest to the obstacle feature are obtained, the swing amount of the transmission line at the obstacle feature position is determined, the safety distance of the transmission line installation position is obtained, and the safety range of the transmission line is determined by the swing amount and the safety distance.

[0020] In one or more embodiments of the present invention, obstacle features are compared with big data to identify obstacles represented by the obstacle features, and it is analyzed whether the obstacles will be affected by wind changes. After determining that the obstacles are affected by wind, the image acquisition device is used to adjust the angle to interfere with the obstacles, change the position of the obstacles, and complete the acquisition of the transmission lines at the locations blocked by the obstacles.

[0021] The present invention also provides an inspection image data processing system, comprising: An image processor, used for processing inspection images; Image segmentation module, which identifies the inspection image and marks the transmission line, and segments the connection features and obstacle features of the transmission line; The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag and horizontal tension of the transmission line; The dancing model calculates the motion trajectory and tension changes of the transmission line under different conditions.

[0022] Through the above technical solution, the present invention has the following beneficial effects: 1. The present invention identifies the same obstacle features in multiple images to determine the location of the obstacle, judges whether the obstacle will affect the transmission line, and determines the location information of the obstacle through the obstacle features and comparison features, judges whether the obstacle is within the range that affects the safety of the transmission line, and determines whether the obstacle will affect the integrity of the transmission line inspection.

[0023] 2. Acquire multiple images with the same obstacle, determine the analysis image and the auxiliary image according to the obstacle characteristics of the obstacle, combine the analysis image and the auxiliary image to determine the obstacle characteristics, and determine the difference in obstacle characteristics between the analysis image and the auxiliary image, judge the positional relationship between the obstacle and the transmission line, and determine whether the obstacle is within the safety range of the transmission line based on the positional relationship.

[0024] 3. Obtain obstacle images and identify obstacles, determine the specific state of the obstacles, and whether the state of the obstacles can be adjusted by drones. This will determine whether the transmission line inspection can be carried out after the obstacle state is adjusted by drones. During the drone inspection process, the impact of obstacles on the transmission line can be reduced, and a more complete transmission line image can be obtained.

[0025] 4. By identifying the sag changes of the transmission line in the image, the swing amount of the transmission line in different environments is determined, and the swing range of the transmission line at different positions is determined according to the swing amount. The safe distance of the obstacle is obtained according to the swing range of the transmission line. The closer to the tower, the smaller the swing range of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the processing method of the present invention; Figure 2 is a schematic diagram of a processing system of the present invention; Figure 3 It is a schematic diagram of image acquisition points of analysis images and auxiliary images of the present invention. DETAILED DESCRIPTION

[0027] The following will disclose multiple embodiments of the present invention with the accompanying drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. In other words, in some embodiments of the present invention, these practical details are not necessary. And if it is possible to implement, the features of different embodiments can be applied interchangeably.

[0028] Unless otherwise defined, all words (including technical and scientific terms) used herein have their usual meanings, which can be understood by those familiar with this field. Furthermore, the definitions of the above words in commonly used dictionaries should be interpreted in the content of this specification as consistent with the meanings of the relevant fields of the present invention. Unless otherwise clearly defined, these words will not be interpreted as idealized or overly formal meanings.

[0029] See also Figure 1-Figure 3 The present invention provides a deep learning-based inspection image data processing method for identifying UAV inspection images and determining the occlusion of transmission lines by obstacles in UAV images.

[0030] In one embodiment, the inspection image data processing method includes the following steps: Acquire an inspection image, the inspection image includes a transmission line, segment the inspection image, and extract a portion of the image including the transmission line; Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set; Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features; The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set; Optionally, the Brenner gradient function calculation formula is as follows: ; In the gradient function formula, I (i, j) Representing images (i, j) Gray value at position; I (i+2, j) Representing images （i+ 2，j） The grayscale value at the position, that is, the position two pixels apart in the horizontal direction; (I(i,j)-I(i+2,j)) 2 It represents the square of the grayscale difference between two pixels in the horizontal direction; Indicates the image (i, j) The sum of all pixel positions is calculated. The larger the grayscale difference, the clearer the edges and details of the image, and the larger the function value.

[0031] The approximate distance between the transmission line and the obstacle is determined by the grayscale difference. When performing image recognition, the image can be further screened and segmented again to retain the obstacle features within the transmission line obstacle range. When performing obstacle feature analysis in the image, it can be more accurate and reduce the impact of proximal features on obstacle feature analysis. The speed of image analysis is improved.

[0032] Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest definition of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and at different acquisition angles from the analysis image as auxiliary images; The distance between the obstacle feature and the transmission line is calculated based on the positional relationship between the obstacle feature at different angles and the transmission line in the analysis image and the auxiliary image; The distance between towers, the height of image acquisition equipment, and the distance between the obstacle feature position and the tower are obtained, a transmission line swing model is established, the sag value of the transmission line is calculated, and the swing range of the transmission line is calculated based on the sag value of the transmission line.

[0033] In this embodiment, obstacles block the transmission lines during the inspection process, resulting in the drone being unable to capture the complete transmission lines, resulting in incomplete identification of the transmission lines. Obstacles include but are not limited to branches, walls and other objects, and the transmission lines are at an unsafe distance from the obstacles, which can easily lead to line failures and cause property losses.

[0034] By using drones to automatically identify images and determine the distance between the obstacle features in the image and the transmission lines, it is possible to analyze whether the existence of the obstacle features will have an impact on the transmission lines.

[0035] Among them, since the transmission line will experience sag after being erected, the transmission line will swing under the interference of external forces. The farther the transmission line is from the tower, the greater the theoretical swing range. Therefore, the swing range of the transmission line increases with the distance between the transmission line and the tower, and the safety range is calculated according to the position of the transmission line.

[0036] In one embodiment, the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected is determined, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is obtained, the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image is determined as the analysis angle, and the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image is determined based on the analysis angle.

[0037] In this embodiment, the image acquisition points, obstacle feature points and transmission line comparison points of the image acquisition device are inspection images acquired at different angles, and the visual angle difference in determining the obstacle feature points and the transmission line comparison points causes a difference in the distance between the obstacle feature and the transmission line, making it impossible to accurately calculate the distance. Therefore, when determining the obstacle feature points and the transmission line comparison points in the analysis image, the obstacle feature points and the transmission line comparison points are marked, and the obstacle feature points and the transmission line comparison points are determined in the auxiliary image, which are consistent with the positions of the obstacle feature points and the transmission line comparison points in the analysis image.

[0038] When marking obstacle feature points and transmission line comparison points in the auxiliary image, determine and analyze the positioning features near the obstacle feature points and transmission line comparison points in the image, and determine the positions of the positioning features and the obstacle feature points and transmission line comparison points based on the acquisition angle of the image acquisition point.

[0039] In one embodiment, the steps of calculating obstacle features and transmission line distance based on the analysis image and the auxiliary image are as follows: Constructing a triangle with image acquisition points, obstacle feature points and transmission line comparison points as vertices in the analysis image and the auxiliary image; Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle I : I =∣ I 1 -I 2∣; in, I 1 and I 2 are the gray values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively; Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point at two angles D : ; in, B is the baseline distance between the two angles, f is the camera focal length; Analyze the distance difference to determine the actual distance between the obstacle and the transmission line.

[0040] In this embodiment, the actual distance between the obstacle and the transmission line is determined by determining the distance difference at two angles and analyzing the angle difference between the image and multiple auxiliary images, thereby determining the precise distance. The grayscale difference in multiple auxiliary images can calibrate the distance between the obstacle and the transmission line to ensure the accuracy of the distance.

[0041] Optionally, when multiple auxiliary images do not correspond to each other during distance calibration, the grayscale difference between the obstacle feature points and the transmission line comparison points in the analysis image and multiple auxiliary images is calculated, and after the distance difference is calculated, the total average value of the distance difference is calculated.

[0042] In one embodiment, the actual distance between the obstacle and the transmission line L The calculation method is as follows: Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Dth : Δθ=θ 2 -θ 1; in, i 1 and i 2 is the acquisition angle of the acquisition image points in the analysis image and the auxiliary image; Through the angle difference Dth And the distance difference D Calculate actual distance L : .

[0043] In this embodiment, based on the analysis images and auxiliary images obtained at different angles, the obstacle feature points and the transmission line comparison points in the image are identified, the actual distance is calculated, and the precise distance between the obstacle and the transmission line is obtained, further ensuring the accurate calculation of the distance.

[0044] Among them, since the transmission line, obstacle and image acquisition equipment are in the same straight line, the actual distance cannot be determined through the image. Therefore, calculating the actual distance through multiple images at different angles can determine whether the obstacle is within the safe range of the transmission line.

[0045] In one embodiment, the steps of establishing a transmission line swing model are as follows: Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension; A transmission line swing model is established based on the tower spacing, transmission line weight and initial tension of the transmission line to calculate the transmission line sag value; Simulate external forces to obtain the impact of wind load, ice weight and short-circuit power on the transmission line. Use the finite element method to simulate the movement trajectory and tension changes of the transmission line under the action of external forces. Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

[0046] In this embodiment, by establishing a transmission line swing model and analyzing the swing range of the transmission line at different locations, the safe range of the transmission line swing at different locations can be determined. During the drone inspection, the safe range of the transmission line at the inspected location can be determined based on the distance from the tower. Based on the identification and calculation of the distance between the obstacle and the transmission line, it is determined whether the obstacle is within the safe range of the transmission line.

[0047] In one embodiment, the transmission line sag calculation formula is as follows: ; in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line; In the absence of horizontal tension in the transmission line, the verticality of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle, and the horizontal tension is determined based on the verticality of the transmission line: .

[0048] In this embodiment, by obtaining the verticality and horizontal tension of the transmission line, the swingable range of the transmission line at different positions is obtained, and then the safety range at different positions can be calculated.

[0049] In one embodiment, the steps of analyzing the galloping characteristics of a transmission line are as follows: The transmission line galloping model is established, and the galloping model equation is: my+cy+ky=F(t) ; in, m is the quality of the transmission line, c is the damping coefficient, k is the stiffness coefficient, y is the transmission line displacement, F(t) For external forces; Numerical simulation is performed using the finite element method to obtain the movement trajectory and tension changes of the transmission line under different conditions; Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations through measurement data; Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

[0050] Exemplarily, a model is established: the mass of the transmission line is m=1kg / m, the damping coefficient is c=0.1Ns / m, and the stiffness coefficient is k=100N / m.

[0051] Numerical simulation: The geometric characteristics and material properties of the transmission line are input into the finite element software, and the wind load F(t)=10 / sin(2 / pit)N is applied.

[0052] Experimental verification: Small-span tests are conducted in the laboratory to measure the vibration characteristics of the transmission line under wind load.

[0053] Analysis results: Through numerical simulation and measurement data, it is found that the swing amplitude of the transmission line is 0.5m and the frequency is 1Hz.

[0054] In this embodiment, the dancing characteristics are analyzed to determine the swing amount of the power transmission line in different environments, and then the swing range of the power transmission line can be determined according to the geographical location and environment of the power transmission line.

[0055] In one embodiment, the distance between the obstacle feature in the analysis image and the tower pole closest to the obstacle feature is obtained, the swing amount of the transmission line at the obstacle feature position is determined, the safety distance of the transmission line installation position is obtained, and the safety range of the transmission line is determined by the swing amount and the safety distance.

[0056] In this embodiment, the safety range of the power transmission line is calculated, whether there are obstacles within the safety range of the power transmission line in the inspection image is identified and calculated, and the obstacles within the safety range are marked to facilitate handling of the obstacles.

[0057] In one embodiment, obstacle features are compared with big data to identify obstacles represented by the obstacle features, and it is analyzed whether the obstacles will be affected by wind changes. After determining that the obstacles are affected by wind, the image acquisition device is used to adjust the angle to interfere with the obstacles, change the position of the obstacles, and complete the acquisition of the transmission lines at the locations blocked by the obstacles.

[0058] In this embodiment, at the location where the transmission line is erected, the obstacles that affect the transmission line are mostly leaves. During the inspection process, the drone will interfere with the leaves through wind at the characteristic position, changing the position of the leaves, so that the leaves can swing, so as to obtain a complete transmission line inspection image.

[0059] The embodiment of the present invention further provides an inspection image data processing system, comprising: An image processor, used for processing inspection images; Image segmentation module, which identifies the inspection image and marks the transmission line, and segments the connection features and obstacle features of the transmission line; The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag and horizontal tension of the transmission line; The dancing model calculates the motion trajectory and tension changes of the transmission line under different conditions.

[0060] In summary, the technical solutions disclosed in the above embodiments of the present invention have at least the following advantages: 1. The present invention identifies the same obstacle features in multiple images to determine the location of the obstacle, judges whether the obstacle will affect the transmission line, and determines the location information of the obstacle through the obstacle features and comparison features, judges whether the obstacle is within the range that affects the safety of the transmission line, and determines whether the obstacle will affect the integrity of the transmission line inspection.

[0061] 2. Acquire multiple images with the same obstacle, determine the analysis image and the auxiliary image according to the obstacle characteristics of the obstacle, combine the analysis image and the auxiliary image to determine the obstacle characteristics, and determine the difference in obstacle characteristics between the analysis image and the auxiliary image, judge the positional relationship between the obstacle and the transmission line, and determine whether the obstacle is within the safety range of the transmission line based on the positional relationship.

[0062] 3. Obtain obstacle images and identify obstacles, determine the specific state of the obstacles, and whether the state of the obstacles can be adjusted by drones. This will determine whether the transmission line inspection can be carried out after the obstacle state is adjusted by drones. During the drone inspection process, the impact of obstacles on the transmission line can be reduced, and a more complete transmission line image can be obtained.

[0063] 4. By identifying the sag changes of the transmission line in the image, the swing amount of the transmission line in different environments is determined, and the swing range of the transmission line at different positions is determined according to the swing amount. The safe distance of the obstacle is obtained according to the swing range of the transmission line. The closer to the tower, the smaller the swing range of the transmission line.

[0064] Although the present invention is disclosed in conjunction with the above embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the attached claims.

Claims

1. A method for processing inspection image data based on deep learning, characterized in that: The following steps are involved: Acquire an inspection image, the inspection image includes a transmission line, segment the inspection image, and extract a portion of the image including the transmission line; Identify the edge connection features of the transmission line, identify the connection feature components based on the connection feature texture, distinguish the proximal features in the connection features, and generate a proximal feature image set; Extract the features in contact with the transmission line in the proximal feature image set, and identify the clarity of the transmission line connected to the proximal features and the edge positions of the proximal features; The Brenner gradient function calculates the square of the grayscale difference between the proximal features and the transmission line pixels, determines the obstacle features of the transmission line within the obstacle range through the grayscale difference, and segments the obstacle features to generate an obstacle feature image set; Acquire multiple inspection images containing the same obstacle feature, determine the inspection image with the highest definition of the obstacle feature among the multiple inspection images as the analysis image, and determine other inspection images containing the obstacle feature and at different acquisition angles from the analysis image as auxiliary images; The distance between the obstacle feature and the transmission line is calculated based on the positional relationship between the obstacle feature at different angles and the transmission line in the analysis image and the auxiliary image; The distance between towers, the height of image acquisition equipment, and the distance between the obstacle feature position and the tower are obtained, a transmission line swing model is established, the sag value of the transmission line is calculated, and the swing range of the transmission line is calculated based on the sag value of the transmission line.

2. The inspection image data processing method based on deep learning according to claim 1 is characterized in that: Determine the grayscale difference between the obstacle feature in the analysis image and the transmission line to which it is connected, obtain the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image, determine the angular relationship between the image acquisition point, the obstacle feature point and the transmission line comparison point in the analysis image as the analysis angle, and determine the angular change of the image acquisition point, the obstacle feature point and the transmission line comparison point in the auxiliary image based on the analysis angle.

3. The inspection image data processing method based on deep learning according to claim 2 is characterized in that: The steps for calculating obstacle features and transmission line distance based on the analysis image and auxiliary image are as follows: Constructing a triangle with image acquisition points, obstacle feature points and transmission line comparison points as vertices in the analysis image and the auxiliary image; Determine the grayscale difference between the obstacle feature point and the transmission line comparison point in the analysis image, as well as the grayscale difference between the obstacle feature point and the transmission line comparison point in the auxiliary image with the acquisition angle closest to the analysis image acquisition angle ΔI : ΔI =∣ I 1 -I 2∣； in, I 1 and I 2 are the gray values ​​of obstacle feature points or transmission line comparison points in the analysis image and auxiliary image respectively; Calculate the distance difference based on the grayscale difference between the obstacle feature point and the transmission line comparison point at two angles D : ； in, B is the baseline distance between the two angles, f is the camera focal length; The distance difference is analyzed to determine the actual distance between the obstacle and the transmission line.

4. The inspection image data processing method based on deep learning according to claim 3 is characterized in that: Actual distance between the obstacle and the transmission line L The calculation method is as follows: Calculate the angle difference between the image point collected in the analysis image and the image point collected in the auxiliary image Δθ : Δθ=θ 2 -θ 1； in, θ 1 and θ 2 is the acquisition angle of the acquisition image points in the analysis image and the auxiliary image; Through the angle difference Δθ And the distance difference D Calculate actual distance L : 。 5. The inspection image data processing method based on deep learning according to claim 4 is characterized in that: The steps to establish the transmission line swing model are as follows: Obtain basic parameters of the transmission line, including tower spacing, transmission line weight, and initial transmission line tension; A transmission line swing model is established based on the tower spacing, transmission line weight and initial tension of the transmission line to calculate the transmission line sag value; Simulate external forces to obtain the impact of wind load, ice weight and short-circuit power on the transmission line. Use the finite element method to simulate the movement trajectory and tension changes of the transmission line under the action of external forces. Analyze the dancing of transmission lines under different conditions and determine the swing range of transmission lines.

6. The inspection image data processing method based on deep learning according to claim 5 is characterized in that: The calculation formula for the sag of a transmission line is as follows: ； in, q is the weight per unit length of the transmission line, R is the tower spacing, H is the horizontal tension at the midpoint of the transmission line; In the absence of horizontal tension in the transmission line, the verticality of the transmission line is determined by the acquisition height of the image acquisition device and the distance between the transmission line and the image acquisition point at the same acquisition angle, and the horizontal tension is determined based on the verticality of the transmission line: 。 7. The inspection image data processing method based on deep learning according to claim 6 is characterized in that: The steps to analyze the galloping characteristics of transmission lines are as follows: The transmission line galloping model is established, and the galloping model equation is: my+cy+ky=F(t) ； in, m is the quality of the transmission line, c is the damping coefficient, k is the stiffness coefficient, y is the transmission line displacement, F(t) For external forces; Numerical simulation is performed using the finite element method to obtain the movement trajectory and tension changes of the transmission line under different conditions; Measure the vibration characteristics of transmission lines under wind loads and verify the accuracy of numerical simulations through measurement data; Analysis of transmission line galloping characteristics based on numerical simulation and measurement data.

8. The inspection image data processing method based on deep learning according to claim 7 is characterized in that: Obtain the distance between the obstacle feature and the tower closest to the obstacle feature in the analysis image, determine the swing amount of the transmission line at the obstacle feature position, obtain the safe distance of the transmission line installation position, and determine the safety range of the transmission line based on the swing amount and the safe distance.

9. The inspection image data processing method based on deep learning according to claim 8, characterized in that: By comparing obstacle features with big data, we can identify the obstacles represented by the obstacle features and analyze whether the obstacles will be affected by wind changes. Once it is determined that the obstacles are affected by wind, we can adjust the angle of the image acquisition device to interfere with the obstacles, change the position of the obstacles, and complete the acquisition of the transmission lines at the locations blocked by the obstacles.

10. A patrol inspection image data processing system, used in the patrol inspection image data processing method according to any one of claims 1 to 9, characterized in that: include: An image processor, used for processing inspection images; Image segmentation module, which identifies the inspection image and marks the transmission line, and segments the connection features and obstacle features of the transmission line; The transmission line sway model is established based on the tower spacing and transmission line parameters to calculate the sag and horizontal tension of the transmission line; The dancing model calculates the motion trajectory and tension changes of the transmission line under different conditions.

Images