A method for identifying and locating thermal insulation faults in pipelines inspected by drones

Abstract

A method for identifying and locating pipeline insulation faults during drone inspections includes: preprocessing pipeline insulation infrared thermal imaging images to construct a standard fault data set file; constructing a neural network for extracting features from pipeline insulation infrared thermal imaging images taken by drones and training them to obtain image feature maps; constructing a neural network for identifying fault targets and locating fault areas in pipeline insulation infrared thermal imaging images, using the obtained image feature maps to train a model and embed it into a drone detection platform; formulating a maintenance strategy based on the drone detection results to achieve preventive maintenance of pipeline insulation. The present invention enables drones to automatically identify pipeline insulation faults, making pipeline insulation detection more comprehensive and detailed during the operation phase, saving a lot of manpower and material resources, and reducing the cost of safe operation and maintenance.

Description

Technical Field

[0001] The present invention relates to a pipeline insulation fault identification and positioning method, and more particularly to a pipeline insulation fault identification and positioning method for unmanned aerial vehicle inspection. Background Art

[0002] There are a large number of process pipelines between and inside the various devices of refining and chemical enterprises. The heat dissipation loss of the pipelines accounts for a large proportion of the energy consumption of the devices. For example, in the steam pipe network system, 5% to 20% of the heat is lost in the process of steam being produced from the steam source and transported to the user through the pipe network. In particular, the heat loss is higher in rainy and snowy weather, which directly causes the inlet temperature of the terminal unit to drop by more than 10°C, affecting normal production. The results of the Sinopec pipeline insulation survey and test show that about 40% of the process pipelines exceed the standard heat dissipation due to insulation damage or insulation performance attenuation. However, many pipelines in the factory are in overhead pipe corridors, and the damage of pipeline insulation often occurs on the upper surface. The existing detection technology is difficult to measure the insulation condition of the upper surface and the damage condition of the pipeline insulation, which will cause over-repair and under-repair of insulation renovation, directly resulting in a waste of funds for insulation renovation projects.

[0003] At present, the inspection of pipeline insulation status is traditionally done by maintenance personnel using handheld infrared thermal imagers to roughly scan the pipeline on the ground. After finding the fault point, they mark it on the pipeline single-line diagram and then treat it. Since the occurrence of pipeline insulation damage is a random process, this manual method is prone to miss scans and consumes a lot of manpower and material resources. Therefore, it is particularly necessary to develop a method for identifying and locating pipeline insulation fault targets during drone inspection based on computer vision. So far, there is no complete system and method for identifying and locating pipeline insulation fault targets in large petrochemical plants. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for identifying and locating thermal insulation faults in pipelines during unmanned aerial vehicle inspection, which can realize long-term operation of thermal insulation of pipelines.

[0005] The technical solution adopted by the present invention is: a method for identifying and locating thermal insulation faults in pipelines inspected by drones, comprising the following steps:

[0006] 1) Preprocess the pipeline insulation infrared thermal imaging images taken by drones, build a pipeline insulation fault attribute information library, obtain the pipeline insulation infrared thermal imaging image fault data set, and establish a standard fault data set file;

[0007] 2) Construct a neural network for feature extraction of pipeline insulation infrared thermal imaging images taken by drones;

[0008] 3) Combined with the constructed pipeline insulation infrared thermal imaging image feature extraction neural network, a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection neural network is constructed;

[0009] 4) using the training set in the standard data set file obtained in step 1) to train the constructed pipeline insulation red infrared thermal imaging image feature extraction neural network to obtain an image feature map, and then using the image feature map to train the pipeline insulation red infrared thermal imaging image fault target recognition and fault area location detection neural network to obtain a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection model;

[0010] 5) Place the pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection model into the drone detection platform, and use the drone detection platform to detect the pipeline insulation;

[0011] 6) Formulate maintenance strategies based on drone inspection results to achieve preventive maintenance of pipeline insulation.

[0012] The invention discloses a method for identifying and locating thermal insulation faults in pipelines inspected by drones, which has the following advantages:

[0013] 1. The present invention realizes the automatic identification of pipeline insulation faults by drones, making the detection of pipeline insulation in the operation stage more comprehensive and detailed, and also saving a lot of manpower and material resources.

[0014] 2. When the image data set is limited, the present invention can utilize the labeled samples of the target image to assist in more effective recognition of the target image data.

[0015] 3. Before the major overhaul of the equipment, the present invention determines the overhaul plan according to the insulation damage fault condition of a certain pipeline in the pipeline insulation fault distribution location diagram at different periods, reduces the safe operation and maintenance cost, and realizes the long-term operation of the pipeline insulation. DETAILED DESCRIPTION

[0016] The vertical wave buoy detection device and method based on a lead screw and a linear guide rail of the present invention are described in detail below in conjunction with the embodiments.

[0017] The invention discloses a method for identifying and locating thermal insulation faults of pipelines inspected by unmanned aerial vehicles. The method comprises the following steps: preprocessing the thermal insulation infrared imaging images of pipelines taken by unmanned aerial vehicles, constructing a pipeline insulation fault attribute information library, and obtaining a pipeline insulation infrared thermal imaging image fault data set; constructing a feature extraction neural network for the thermal insulation infrared imaging images of pipelines taken by unmanned aerial vehicles, which is used to extract the feature map of the thermal insulation infrared imaging images of pipelines taken by unmanned aerial vehicles; constructing a neural network for fault target identification and fault area positioning detection of the thermal insulation infrared imaging images of pipelines taken by unmanned aerial vehicles; training the neural network for fault target identification and fault area positioning detection with the thermal insulation infrared imaging image data set, and obtaining a thermal insulation infrared thermal imaging image fault target identification and fault area positioning detection model; placing the thermal insulation infrared thermal imaging image fault target identification and fault area positioning detection model into an unmanned aerial vehicle detection platform; and using the unmanned aerial vehicle detection platform to detect the thermal insulation of pipelines, so as to realize automatic identification of thermal insulation faults of pipelines during the movement of the unmanned aerial vehicle, and automatically forming a pipe bundle fault distribution positioning map according to the detection results, and determining a maintenance plan according to the thermal insulation damage fault condition of a certain pipeline in the thermal insulation fault distribution positioning map of different periods.

[0018] A method for identifying and locating thermal insulation faults in pipelines inspected by an unmanned aerial vehicle of the present invention is characterized by comprising the following steps:

[0019] 1) Preprocess the pipeline insulation infrared thermal imaging images taken by drones, build a pipeline insulation fault attribute information library, obtain the pipeline insulation infrared thermal imaging image fault data set, and establish a standard fault data set file; including:

[0020] (1) The infrared thermal imaging images are numbered and named in the format of 1.jpg to n.jpg to form a pipeline insulation infrared thermal imaging image group;

[0021] (2) Using the image feature judgment method in the pipeline insulation infrared detection method, the pipeline insulation infrared thermal imaging image group is judged for damage faults: when the pipeline insulation is damaged, the corresponding part in the image is displayed as a red area, and when the pipeline insulation is operating normally, the pipe bundle in the image is a yellow area;

[0022] (3) Establish a pipeline insulation infrared thermal imaging image fault data set, and mark the infrared anomalies that appear into three categories according to the form of the anomaly: the maximum length of the abnormal area is less than or equal to 0.5 meters as mild damage, the maximum length of the abnormal area is greater than 0.5 meters and less than or equal to 1.5 meters as moderate damage, and the maximum length of the abnormal area is greater than 1.5 meters as severe damage;

[0023] (4) Use the labelling software tool to frame the red area in each image in the pipeline insulation infrared thermal imaging image group with a rectangular annotation box, and generate an XML file corresponding to the image in the pipeline insulation infrared thermal imaging image group for saving. The XML file contains the image file name, size, rectangular annotation box position, and annotation target category information. Finally, the labeled data files are classified into training set, validation set, and test set in a ratio of 70:15:15 to establish a standard fault data set file.

[0024] 2) Construct a neural network for feature extraction of pipeline insulation infrared thermal imaging images taken by drones;

[0025] It uses the residual convolutional neural network ResNet50 architecture, which includes 1 image preprocessing block and 5 stacking blocks from top to bottom. The first stacking block is composed of a standard convolution layer and a maximum pooling layer, the second stacking block is a residual module containing 3 convolution blocks, the third stacking block is a residual module containing 4 convolution blocks, the fourth stacking block is a residual module containing 6 convolution blocks, and the fifth stacking block is a residual module containing 3 convolution blocks, 1 average pooling and 1 fully connected layer. Among them,

[0026] (1) The image preprocessing block is used to crop and edge-expand the image taken by the received drone to convert the original image into a 227×227×3 image;

[0027] (2) The first stacking block includes 64 convolution kernels and 64 maximum poolers;

[0028] (3) The second stacking block is divided into two paths, one path is the expanded dimension convolution kernel, and the other path includes three convolution blocks. Each convolution block has three convolution layers from top to bottom. The first layer is 64 1×1 convolution kernels, the second layer is 64 3×3 convolution kernels, and the third layer is 256 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the expanded dimension convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the expanded dimension convolution kernel and used as the input of the third convolution block. The output result of the third convolution block enters the third stacking block;

[0029] (4) The third stacking block includes: a dilated convolution kernel and four convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer has 128 1×1 convolution kernels, the second convolution layer has 128 3×3 convolution kernels, and the third convolution layer has 512 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the dilated convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the dilated convolution kernel and used as the input of the third convolution block. The output result of the third convolution block is added to the output result of the dilated convolution kernel and used as the input of the fourth convolution block. The input of the fourth convolution block enters the fourth stacking block.

[0030] (5) The fourth stacking block includes: a dilated convolution kernel and 6 convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer has 256 1×1 convolution kernels, the second convolution layer has 256 3×3 convolution kernels, and the third convolution layer has 1024 1×1 convolution kernels. The output result of the first convolution block among the 6 convolution blocks is added to the output result of the dilated convolution kernel as the input of the next convolution block. The output result of the 6th convolution block enters the fifth stacking block.

[0031] (6) The fifth stack includes: a dilated convolution kernel and three convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer is 512 1×1 convolution kernels, the second convolution layer is 512 3×3 convolution kernels, and the third convolution layer is 2048 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the dilated convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the dilated convolution kernel and used as the input of the third convolution block. The output result of the third convolution block is the extracted feature.

[0032] Each convolution layer in each of the above stacks uses batch normalization operation. The output of the first convolution layer and the output of the second convolution layer of each convolution block in each stack are activated using the Relu function. The output of each convolution block is activated using the Relu function. The batch normalization operation formula is as follows:

[0033]

[0034]

[0035]

[0036]

[0037] where x ai is the input of a batch of n elements of the convolutional network layer a, xai ={x a1 ,x a2 ,…,x an},μ a is the mean of a batch of elements, is the variance of a batch element, is the result of normalizing all elements in the same batch, ζ is to ensure that the denominator is non-zero, and y ai is the final output, γ ai is the scale scaling, β ai Offset parameters;

[0038] The activation function Relu is expressed as:

[0039]

[0040] Among them, f(y ai ) is the ReLU activation function value of the output of the a-th layer of the convolutional network.

[0041] 3) Combined with the constructed pipeline insulation infrared thermal imaging image feature extraction neural network, a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection neural network is constructed;

[0042] The neural network for fault target recognition and fault area location detection of pipeline thermal insulation infrared thermal imaging images is a neural network based on the FasterR-CNN structure, including a neural network for feature extraction of pipeline thermal insulation infrared thermal imaging images, a regional production module RPN, a regional proposal pooling module, and a regional method module RCNN based on convolutional neural network features. Among them, the following:

[0043] (1) The regional production module RPN includes the following operation processes:

[0044] (1.1) Use a 3×3×2048 sliding window to slide on the feature map extracted by the pipeline insulation infrared thermal imaging image feature extraction neural network, find the pixel position corresponding to the center of each three-dimensional sliding window in the original image, and then generate k anchor frames on the original image. The centers of these k anchor frames coincide with the center corresponding to the three-dimensional sliding window. At each coordinate point in the feature map, calculate the offset between the center of the anchor frame and the coordinate. Through the relative offset between the anchor frame and the generated one, calculate the position of all anchor frames in each feature map;

[0045] (1.2) All anchor box positions in each feature map are input into a 1×1 convolution layer. The 1×1 convolution layer outputs 2k×20×16 anchor box image features. Then, the reshape function is used to map the 2k×20×16 anchor box image features into 2×20k×16 to obtain the foreground and background scores of the anchor boxes. The results are sent to the softmax function for probability calculation. The obtained features are then transformed into 2k×20×16 anchor box image features, and finally the probability of each anchor box belonging to the foreground and background is output; all anchor box positions in each feature map are input into another 1×1 convolution layer. The 1×1 convolution layer outputs 4k×20×16 features. 4k contains the predictions of k anchor boxes. Each anchor box has 4 data, which represent the horizontal and vertical coordinates of the center point of each anchor box and the width and height of each anchor box. These 4 quantities are the offsets relative to the rectangular annotation box.

[0046] (1.3) The calculation process of the rectangular annotation box for RPN module classification and regression offset is as follows: Calculate the intersection over union (IOU) ratio (A i , B j ), and the cross ratio IOU(A i , B j ) is expressed as:

[0047]

[0048] Among them, IOU(A i , B j ) represents the union-intersection ratio of the area of ​​the i-th rectangular annotation box annotated in the image to the j-th anchor box; A i represents the area of ​​the i-th rectangular annotation box in the image; B j Represents the area of ​​the jth anchor box generated;

[0049] (1.4) According to the intersection over union ratio, each anchor box is determined to be a positive sample or a negative sample. The criteria for judging positive and negative samples are as follows: The intersection over union ratio (IOU) of all rectangular annotation boxes (A i , B j ) are all less than 0.3 as negative samples; and the intersection over union ratio (IOU) of any rectangular annotation box (A i , B j ) Anchor boxes with a value greater than or equal to 0.7 are positive samples; anchor boxes that are neither positive samples nor negative samples are ignored;

[0050] (1.5) Calculate the overall loss function of the RPN module and optimize the parameters of each layer of the network based on the loss function;

[0051] The loss function of the RPN module includes two parts: classification loss and regression loss. The formula is as follows:

[0052]

[0053]

[0054]

[0055] in represents the classification cross entropy loss of the filtered anchor box; P k is the true value of the category of the nth anchor box; is the predicted category of the nth anchor box, when the nth anchor box is a positive sample The value is 1, which is a negative sample. The value is negative 1; N cls is the classification loss weight value, N reg is the weight value of regression loss, represents the regression loss, t k ={t x , t y , t w , t h} is the distance between the center coordinates and width and height of the anchor box and the center coordinates and width and height of the rectangular annotation box. is the predicted value of the distance between the center coordinates and width and height of the anchor box and the center coordinates and width and height of the rectangular annotation box;

[0056] (1.6) The parameters of the RPN module are trained according to the rectangular annotation boxes calibrated in the pipeline insulation infrared thermal imaging image training set. The parameters of the RPN module are optimized alternately by the stochastic gradient descent method. When optimizing the classification parameters, the regression parameters are fixed unchanged. When optimizing the regression parameters, the classification parameters are fixed unchanged.

[0057] (2) The operation steps of the region proposal pooling module are as follows:

[0058] Among the anchor frames constituting positive samples and negative samples, 256 anchor frames are taken out and mapped to the feature map extracted by the pipeline insulation infrared thermal imaging image feature extraction neural network, and the feature area mapped by each anchor frame is evenly divided into 7×7 squares, and 4 sampling points are selected in each small square. Then, the 4 points in each small square are bilinearly interpolated by bilinear interpolation to obtain 4 values, and then maximum pooling is performed to obtain the value of each small square, and the feature map after maximum pooling is obtained;

[0059] (3) The operation process of the regional method module RCNN based on convolutional neural network features:

[0060] The feature map after maximum pooling is fully connected to predict the classification of each of the 256 anchor boxes, and the offset between the anchor box and the corresponding rectangular annotation box is predicted. The offset calculation method is the same as the rectangular annotation box calculation method of the RPN module classification and regression offset (1.3).

[0061] 4) using the training set in the standard data set file obtained in step 1) to train the constructed pipeline insulation red infrared thermal imaging image feature extraction neural network to obtain an image feature map, and then using the image feature map to train the pipeline insulation red infrared thermal imaging image fault target recognition and fault area location detection neural network to obtain a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection model;

[0062] 5) The pipeline insulation infrared thermal imaging image fault target recognition and fault area positioning detection model is placed into the drone detection platform, and the pipeline insulation is tested using the drone detection platform. The drone follows the pipeline direction and above the pipeline, uses the carried infrared thermal imaging camera to shoot the pipeline insulation, and uses the pipeline insulation infrared thermal imaging image fault target recognition and fault area positioning detection model to analyze and process the image, mark the insulation fault location, and mark the location information of the faulty pipeline, and upload it to the host computer;

[0063] 6) Formulate maintenance strategies based on the drone inspection results to achieve preventive maintenance of pipeline insulation; the maintenance strategies include:

[0064] According to the test results, a distribution map of pipeline insulation failures is formed. According to the degree of damage to the identified pipeline insulation, the staff is reminded to take corresponding measures in a timely manner, specifically:

[0065] According to the design specifications of industrial equipment and pipeline insulation engineering, determine the medium temperature T in the pipeline L The maximum allowable heat dissipation loss per unit area q e , as the limit value of heat dissipation loss per unit area of ​​pipeline insulation;

[0066] When testing, the medium temperature in the tested pipeline is T L When the pipeline has a length of 10 meters, the number of minor damages is greater than 10, or the number of moderate damages is greater than 5, or the number of severe damages is greater than 3, and the calculated heat dissipation loss per unit area of ​​the total damaged pipeline insulation is greater than the maximum allowable heat dissipation loss per unit area q e When the problem occurs, mark the area and repair it at a later date;

[0067] When testing, the medium temperature in the tested pipeline is T LWhen the number of damaged points of the pipeline exceeds 50, and the calculated heat dissipation loss per unit area of ​​the total damaged pipeline insulation is greater than the maximum allowable heat dissipation loss per unit area q e , replace the pipeline insulation as a whole at an appropriate time.

[0068] The medium temperature in the tested pipeline is T L The heat dissipation loss per unit area of ​​the pipeline insulation is calculated according to the outer surface temperature of the pipeline insulation layer, the ambient temperature and the surface heat transfer coefficient as follows:

[0069] q=α×(T W -T F )

[0070] Where: q is the heat dissipation loss per unit area of ​​pipeline insulation, unit is W / m 2 ; α is the surface heat transfer coefficient, the unit is W / (m 2 ·K); T W is the outer surface temperature of the pipe insulation layer, in K; T F is the ambient temperature, in K;

[0071] Where: The surface heat transfer coefficient α is obtained by the following formula:

[0072]

[0073] Where: ω is wind speed, unit is m / s;

[0074] Medium temperature in the pipeline T L The maximum allowable heat dissipation loss per unit area q e as follows:

[0075] Medium temperature in the pipeline T L When the temperature is 373K, 423K, 473K, 523K, 573K, 623K, 673K, 723K, 773K, 823K, the corresponding maximum allowable heat loss per unit area q e 84W / m 2 、104W / m 2 、126W / m 2 、147W / m 2 、167W / m 2 、188W / m 2 、204W / m 2 , 220W / m 2 、236W / m 2 、251W / m 2 , the medium temperature in other pipelines is T L The corresponding maximum allowable heat dissipation loss per unit area q e Determine by the following formula:

[0076] q e =0.4×T L -59.71

[0077] Among them, q e is the medium temperature in the pipeline T L The corresponding maximum allowable heat dissipation loss per unit area, T L is the medium temperature in the pipeline, in K.

Claims

1. A method for identifying and locating thermal insulation faults in pipelines inspected by drones, characterized in that: The steps include: 1) Preprocess the pipeline insulation infrared thermal imaging images taken by drones, build a pipeline insulation fault attribute information library, obtain the pipeline insulation infrared thermal imaging image fault data set, and establish a standard fault data set file; 2) Construct a neural network for feature extraction of pipeline insulation infrared thermal imaging images taken by drones; 3) Combined with the constructed pipeline insulation infrared thermal imaging image feature extraction neural network, a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection neural network is constructed; 4) using the training set in the standard data set file obtained in step 1) to train the constructed pipeline insulation red infrared thermal imaging image feature extraction neural network to obtain an image feature map, and then using the image feature map to train the pipeline insulation red infrared thermal imaging image fault target recognition and fault area location detection neural network to obtain a pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection model; 5) Place the pipeline insulation infrared thermal imaging image fault target recognition and fault area location detection model into the drone detection platform, and use the drone detection platform to detect the pipeline insulation; 6) Formulate maintenance strategies based on drone inspection results to achieve preventive maintenance of pipeline insulation; The maintenance strategy formulation includes: According to the test results, a distribution map of pipeline insulation failures is formed. According to the degree of damage to the identified pipeline insulation, the staff is reminded to take corresponding measures in a timely manner, specifically: According to the design specifications of industrial equipment and pipeline insulation engineering, determine the medium temperature T in the pipeline L The maximum allowable heat dissipation loss per unit area q e , as the limit value of heat dissipation loss per unit area of ​​pipeline insulation; When testing, the medium temperature in the tested pipeline is T L When the pipeline has a length of 10 meters, the number of minor damages is greater than 10, or the number of moderate damages is greater than 5, or the number of severe damages is greater than 3, and the calculated heat dissipation loss per unit area of ​​the total damaged pipeline insulation is greater than the maximum allowable heat dissipation loss per unit area q e When the problem occurs, mark the area and repair it at a later date; When testing, the medium temperature in the tested pipeline is T L When the number of damaged points of the pipeline exceeds 50, and the calculated heat dissipation loss per unit area of ​​the total damaged pipeline insulation is greater than the maximum allowable heat dissipation loss per unit area q e , replace the pipe insulation as a whole at an appropriate time; The medium temperature in the tested pipeline is T L The heat dissipation loss per unit area of ​​the pipeline insulation is calculated according to the outer surface temperature of the pipeline insulation layer, the ambient temperature and the surface heat transfer coefficient as follows: q=α×(T W -T F ) Where: q is the heat dissipation loss per unit area of ​​pipeline insulation, unit is W / m 2 ; α is the surface heat transfer coefficient, unit is W / (m 2 ·K); T W is the outer surface temperature of the pipe insulation layer, in K; T F is the ambient temperature, in K; Where: The surface heat transfer coefficient α is obtained by the following formula: Where: ω is wind speed, unit is m / s; Medium temperature in the pipeline T L The maximum allowable heat dissipation loss per unit area q e as follows: Medium temperature in the pipeline T L When the temperature is 373K, 423K, 473K, 523K, 573K, 623K, 673K, 723K, 773K, 823K, the corresponding maximum allowable heat loss per unit area q e 84W / m 2 、104W / m 2 、126W / m 2 、147W / m 2 、167W / m 2 、188W / m 2 、204W / m 2 , 220W / m 2 、236W / m 2 、251W / m 2 , the medium temperature in other pipelines is T L The corresponding maximum allowable heat dissipation loss per unit area q e Determine as follows: q e =0.4×T W -59.71 Among them, q e is the medium temperature in the pipeline T L The corresponding maximum allowable heat dissipation loss per unit area, T L is the medium temperature in the pipeline, in K.

2. The method for identifying and locating thermal insulation faults of pipelines inspected by drones according to claim 1 is characterized in that: Step 1) includes: (1) The infrared thermal imaging images are numbered and named in the format of 1.jpg to n.jpg to form a pipeline insulation infrared thermal imaging image group; (2) Using the image feature judgment method in the pipeline insulation infrared detection method, the pipeline insulation infrared thermal imaging image group is judged for damage faults: when the pipeline insulation is damaged, the corresponding part in the image is displayed as a red area, and when the pipeline insulation is operating normally, the pipe bundle in the image is a yellow area; (3) Establish a pipeline insulation infrared thermal imaging image fault data set, and mark the infrared anomalies that appear into three categories according to the form of the anomaly: the maximum length of the abnormal area is less than or equal to 0.5 meters as mild damage, the maximum length of the abnormal area is greater than 0.5 meters and less than or equal to 1.5 meters as moderate damage, and the maximum length of the abnormal area is greater than 1.5 meters as severe damage; (4) Use the labelling software tool to frame the red area in each image in the pipeline insulation infrared thermal imaging image group with a rectangular annotation box, and generate an XML file corresponding to the image in the pipeline insulation infrared thermal imaging image group for saving. The XML file contains the image file name, size, rectangular annotation box position, and annotation target category information. Finally, the labeled data files are classified into training set, validation set, and test set in a ratio of 70:15:15 to establish a standard fault data set file.

3. The method for identifying and locating thermal insulation faults of pipelines inspected by drones according to claim 1 is characterized in that: The feature extraction neural network of the pipeline insulation infrared thermal imaging image taken by the drone in step 2) adopts the residual convolutional neural network ResNet50 architecture, which includes 1 image preprocessing block and 5 stacking blocks from top to bottom. The first stacking block is composed of a standard convolution layer and a maximum pooling layer, the second stacking block is a residual module containing 3 convolution blocks, the third stacking block is a residual module containing 4 convolution blocks, the fourth stacking block is a residual module containing 6 convolution blocks, and the fifth stacking block is a residual module containing 3 convolution blocks, 1 average pooling and 1 fully connected layer.

4. The method for identifying and locating thermal insulation faults of pipelines inspected by drones according to claim 3 is characterized by: (1) The image preprocessing block is used to crop and edge-expand the image taken by the received drone to convert the original image into a 227×227×3 image; (2) The first stacking block includes 64 convolution kernels and 64 maximum poolers; (3) The second stacking block is divided into two paths, one path is the expanded dimension convolution kernel, and the other path includes three convolution blocks. Each convolution block has three convolution layers from top to bottom. The first layer is 64 1×1 convolution kernels, the second layer is 64 3×3 convolution kernels, and the third layer is 256 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the expanded dimension convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the expanded dimension convolution kernel and used as the input of the third convolution block. The output result of the third convolution block enters the third stacking block; (4) The third stacking block includes: a dilated convolution kernel and four convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer has 128 1×1 convolution kernels, the second convolution layer has 128 3×3 convolution kernels, and the third convolution layer has 512 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the dilated convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the dilated convolution kernel and used as the input of the third convolution block. The output result of the third convolution block is added to the output result of the dilated convolution kernel and used as the input of the fourth convolution block. The input of the fourth convolution block enters the fourth stacking block. (5) The fourth stacking block includes: a dilated convolution kernel and 6 convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer has 256 1×1 convolution kernels, the second convolution layer has 256 3×3 convolution kernels, and the third convolution layer has 1024 1×1 convolution kernels. The output result of the first convolution block among the 6 convolution blocks is added to the output result of the dilated convolution kernel as the input of the next convolution block. The output result of the 6th convolution block enters the fifth stacking block. (6) The fifth stack includes: a dilated convolution kernel and three convolution blocks from top to bottom. Each convolution block has three convolution layers from top to bottom. The first convolution layer is 512 1×1 convolution kernels, the second convolution layer is 512 3×3 convolution kernels, and the third convolution layer is 2048 1×1 convolution kernels. The output result of the first convolution block is added to the output result of the dilated convolution kernel and used as the input of the second convolution block. The output result of the second convolution block is added to the output result of the dilated convolution kernel and used as the input of the third convolution block. The output result of the third convolution block is the extracted feature.

5. The method for identifying and locating thermal insulation faults of pipelines inspected by unmanned aerial vehicles according to claim 4 is characterized in that: Each convolutional layer in each stack uses batch normalization operation. The output of the first convolutional layer and the output of the second convolutional layer of each convolutional block in each stack are activated using the Relu function. The output of each convolutional block is activated using the Relu function. The batch normalization operation formula is as follows: where x ai is the input of a batch of n elements of the convolutional network layer a, x ai ={x a1 ,x a2 ,…,x an },μ a is the mean of a batch of elements, is the variance of a batch element, is the result of normalizing all elements in the same batch, ζ is to ensure that the denominator is non-zero, and y ai is the final output, γ ai is the scale scaling, β ai Offset parameters; The activation function Relu is expressed as: Among them, f(y ai ) is the ReLU activation function value of the output of the a-th layer of the convolutional network.

6. The method for identifying and locating thermal insulation faults of pipelines inspected by drones according to claim 1 is characterized in that: Step 3) The neural network for constructing fault target recognition and fault area positioning detection in pipeline insulation infrared thermal imaging images is a neural network based on the FasterR-CNN structure, including a pipeline insulation infrared thermal imaging image feature extraction neural network, a regional production module RPN, a regional proposal pooling module, and a regional method module RCNN based on convolutional neural network features.

7. The method for identifying and locating thermal insulation faults of pipelines inspected by drones according to claim 6 is characterized in that: Said: (1) The regional production module RPN includes the following operation processes: (1.1) Use a 3×3×2048 sliding window to slide on the feature map extracted by the pipeline insulation infrared thermal imaging image feature extraction neural network, find the pixel position corresponding to the center of each three-dimensional sliding window in the original image, and then generate k anchor frames on the original image. The centers of these k anchor frames coincide with the center corresponding to the three-dimensional sliding window. At each coordinate point in the feature map, calculate the offset between the center of the anchor frame and the coordinate. Through the relative offset between the anchor frame and the generated one, calculate the position of all anchor frames in each feature map; (1.2) All anchor box positions in each feature map are input into a 1×1 convolution layer. The 1×1 convolution layer outputs 2k×20×16 anchor box image features. Then, the reshape function is used to map the 2k×20×16 anchor box image features into 2×20k×16 to obtain the foreground and background scores of the anchor boxes. The results are sent to the softmax function for probability calculation. The obtained features are then transformed into 2k×20×16 anchor box image features, and finally the probability of each anchor box belonging to the foreground and background is output; all anchor box positions in each feature map are input into another 1×1 convolution layer. The 1×1 convolution layer outputs 4k×20×16 features. 4k contains the predictions of k anchor boxes. Each anchor box has 4 data, which represent the horizontal and vertical coordinates of the center point of each anchor box and the width and height of each anchor box. These 4 quantities are the offsets relative to the rectangular annotation box. (1.3) The calculation process of the rectangular annotation box for RPN module classification and regression offset is as follows: Calculate the intersection over union (IOU) ratio (A i , B j ), and the cross ratio IOU(A i , B j ) is expressed as: Among them, IOU(A i , B j ) represents the union-intersection ratio of the area of ​​the i-th rectangular annotation box annotated in the image to the j-th anchor box; A i represents the area of ​​the i-th rectangular annotation box in the image; B j Represents the area of ​​the jth anchor box generated; (1.4) According to the intersection over union ratio, each anchor box is determined to be a positive sample or a negative sample. The criteria for judging positive and negative samples are as follows: The intersection over union ratio (IOU) of all rectangular annotation boxes (A i , B j ) are all less than 0.3 as negative samples; and the intersection over union ratio (IOU) of any rectangular annotation box (A i , B j ) Anchor boxes with a value greater than or equal to 0.7 are positive samples; anchor boxes that are neither positive samples nor negative samples are ignored; (1.5) Calculate the overall loss function of the RPN module and optimize the parameters of each layer of the network based on the loss function; The loss function of the RPN module includes two parts: classification loss and regression loss. The formula is as follows: in represents the classification cross entropy loss of the filtered anchor box; P k is the true value of the category of the nth anchor box; is the predicted category of the nth anchor box, when the nth anchor box is a positive sample The value is 1, which is a negative sample. The value is negative 1; N cls is the classification loss weight value, N reg is the weight value of regression loss, represents the regression loss, t k ={t x , t y , t w , t h } is the distance between the center coordinates and width and height of the anchor box and the center coordinates and width and height of the rectangular annotation box. is the predicted value of the distance between the center coordinates and width and height of the anchor box and the center coordinates and width and height of the rectangular annotation box; (1.6) The parameters of the RPN module are trained according to the rectangular annotation boxes calibrated in the pipeline insulation infrared thermal imaging image training set. The parameters of the RPN module are optimized alternately by the stochastic gradient descent method. When optimizing the classification parameters, the regression parameters are fixed unchanged. When optimizing the regression parameters, the classification parameters are fixed unchanged. (2) The operation steps of the region proposal pooling module are as follows: Among the anchor frames constituting positive samples and negative samples, 256 anchor frames are taken out and mapped to the feature map extracted by the pipeline insulation infrared thermal imaging image feature extraction neural network, and the feature area mapped by each anchor frame is evenly divided into 7×7 squares, and 4 sampling points are selected in each small square. Then, the 4 points in each small square are bilinearly interpolated by bilinear interpolation to obtain 4 values, and then maximum pooling is performed to obtain the value of each small square, and the feature map after maximum pooling is obtained; (3) The operation process of the regional method module RCNN based on convolutional neural network features: The feature map after maximum pooling is fully connected to predict the classification of each of the 256 anchor boxes, and the offset between the anchor box and the corresponding rectangular annotation box is predicted. The offset calculation method is the same as the rectangular annotation box calculation method of the RPN module classification and regression offset (1.3).

Images