An unmanned aerial vehicle inspection image recognition method

Abstract

The present application relates to the technical field of unmanned aerial vehicle power inspection and image recognition, and particularly relates to an unmanned aerial vehicle inspection image recognition method, comprising: collecting current infrared thermal images, unique numbers, load currents and ambient temperatures of power equipment; generating an adversarial network according to the number to call historical data set training conditions, taking working condition parameters as input and output normal baseline infrared thermal images; inputting the current current and temperature into the adversarial network to generate a baseline thermal image under the current working condition; after registering the current thermal image and the baseline thermal image, a residual thermal image is obtained by pixel-by-pixel difference; an adaptive threshold segmentation is used to extract the connected region exceeding the threshold of the residual error, and a thermal defect abnormal area is obtained after post-processing; the area change trend of multiple inspections is recorded, and the inspection cycle is shortened and a warning is given when the preset condition is met. The present application realizes personalized dynamic baseline and predictive maintenance, solves the problem of insufficient data of new equipment through a cold start mechanism, and improves the early thermal defect recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of UAV power line inspection and image recognition technology, and in particular to a UAV inspection image recognition method. Background Technology

[0002] With the maturity of drone technology, drones equipped with infrared thermal imagers have been widely used in the field of power equipment inspection, identifying overheating defects by collecting infrared thermal images of the equipment surface. However, the actual temperature of power equipment is affected by multiple dynamic factors such as load current, ambient temperature, wind speed, and sunlight, and the normal temperature difference under different operating conditions can reach tens of degrees Celsius. Currently, the mainstream identification methods mainly rely on fixed temperature thresholds (such as alarms when the temperature exceeds 80°C) or the lateral comparison method based on three-phase equipment in the same circuit (i.e., the "three-phase comparison method"), attempting to determine whether there are thermal defects in the equipment through absolute or relative values.

[0003] The core technical problem with the aforementioned existing technologies lies in the inability to establish a personalized normal thermal benchmark for each specific device that can dynamically adapt to the current operating conditions (especially load current and ambient temperature). This leads to the fixed threshold method easily misjudging normal high temperatures as defects under full load in summer, while failing to detect early minute temperature rises under light load in winter. Furthermore, the three-phase comparison method is completely ineffective for single-phase devices, shared-enclosure devices, or devices with only one unit operating. Therefore, how to generate an infrared thermal image benchmark that "should be normal under the current operating conditions" for each device based on its own operating conditions, thereby achieving high-precision pixel-level anomaly detection, is a technical challenge that needs to be solved in this field. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides an image recognition method for UAV inspection, which aims to improve the problem that the lack of a personalized dynamic thermal benchmark that adapts to working conditions in existing UAV infrared inspections leads to the inability to accurately detect early thermal defects.

[0005] This invention provides the following technical solution: a method for image recognition during unmanned aerial vehicle (UAV) inspections, comprising:

[0006] S1. Control the infrared thermal imager carried by the drone to collect the current infrared thermal image of the power equipment to be tested, and simultaneously collect the unique number of the equipment, the current load current and the current ambient temperature;

[0007] S2. Retrieve the historical inspection dataset of the device according to the unique number. The dataset contains paired historical infrared thermal images and operating parameters. Use the dataset to train a conditional generative adversarial network. The network takes the operating parameters as input and outputs the corresponding normal baseline infrared thermal image.

[0008] S3. Input the current load current and current ambient temperature into the trained conditional generative adversarial network to generate a normal reference infrared thermal image under the current operating conditions.

[0009] S4. Perform image spatial registration between the current infrared thermal image and the normal reference infrared thermal image to establish a spatial mapping relationship between the two.

[0010] S5. Perform pixel-by-pixel difference calculation on the registered current infrared thermal image and the normal reference infrared thermal image to obtain the residual thermal image;

[0011] S6. Based on the residual values ​​of each pixel in the residual heatmap, an adaptive threshold segmentation method is used to extract connected regions with residual values ​​greater than the dynamic threshold, and the thermal defect abnormal region is obtained after morphological post-processing.

[0012] S7. Record the area change trend of the thermal defect abnormal area during multiple inspections. When the trend meets the preset conditions, shorten the inspection cycle of the equipment and output an active warning.

[0013] The present invention has the following beneficial effects:

[0014] 1. In this invention, by training an adversarial network for each device individually, a baseline infrared thermal image of the device under the current operating conditions can be generated based on the currently collected load current and ambient temperature. This eliminates the reliance on fixed thresholds or horizontal comparisons with similar devices, and improves the accuracy and reliability of early thermal defect identification in complex operating environments.

[0015] 2. In this invention, by performing pixel-by-pixel differential calculations on the registered measured thermal image and the generated reference thermal image, a residual thermal image is obtained, which can locate the temperature deviation at any position on the surface of the device. Compared with the traditional method based on the global maximum temperature or regional average temperature, this solution can discover small, localized early heating hazards and move the defect identification window forward.

[0016] 3. In this invention, the segmentation threshold is dynamically calculated based on the mean and standard deviation of the historical residual distribution of the equipment itself. This can automatically filter out the inherent measurement noise and individual thermal differences of each equipment, ensuring the statistical reliability of anomaly judgment and effectively avoiding false alarms caused by fixed thresholds. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an image recognition method for unmanned aerial vehicle (UAV) inspection proposed in this invention.

[0018] Figure 2 This is a schematic diagram of the conditional generative adversarial network training and cold start process for an image recognition method for UAV inspection proposed in this invention.

[0019] Figure 3This is a schematic diagram of the abnormal region extraction and predictive maintenance closed-loop process of the UAV inspection image recognition method proposed in this invention. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides a method for image recognition during unmanned aerial vehicle (UAV) inspections, such as... Figures 1-3 As shown, it includes the following steps:

[0022] S1. Control the infrared thermal imager carried by the drone to collect the current infrared thermal image of the power equipment to be tested, and simultaneously collect the unique number of the equipment, the current load current and the current ambient temperature.

[0023] Specifically, firstly, the UAV, equipped with an infrared thermal imager and a visible light camera, autonomously flies along a preset route to a predetermined hovering position above the power equipment to be inspected. In this embodiment, for disconnect switches within a substation, the hovering distance is set to 3 to 5 meters to ensure sufficient spatial resolution in the infrared thermal image. After the UAV hovers stably, the following data acquisition operations are performed: Acquire the current infrared thermal image: Take a picture of the current infrared thermal image of the equipment using the infrared thermal imager, and record it as... The infrared thermal image is a two-dimensional matrix, where the value of each pixel represents the temperature value at the corresponding location on the device surface. In this embodiment, the infrared thermal imager has a resolution of 1024×768 and a thermal sensitivity better than 0.02℃. Acquiring the unique device identifier: An image of the nameplate or QR code on the device is captured by a visible light camera. Using a deep learning-based text detection and recognition model, such as a convolutional recurrent neural network (CRNN), the unique identifier of the device is automatically extracted from the identifier image and recorded as its ID. This ID is globally unique, for example, "#110kV-Isolating Switch-021-A Phase," and is used to associate all historical inspection data throughout the device's entire lifecycle. Acquiring the current ambient temperature: The ambient temperature around the device is collected in real time at the time of inspection using an airborne meteorological sensor mounted on the UAV and recorded as... The unit is degrees Celsius. Current load current is collected: The current load current value of the circuit containing the device is obtained in real time from a power monitoring system, such as a SCADA system, via a data interface, and recorded as... The unit is amperes. This interface uses a dedicated power safety isolation device for unidirectional data acquisition; it only reads data and does not issue control commands.

[0024] Finally, the acquired current infrared thermal image Device unique ID, current ambient temperature Current load current The current inspection timestamp is associated with the package and stored in the inspection database as the original record of this inspection.

[0025] Through the above operations, step S1 achieves precise binding between device identity and current operating condition, providing a data foundation for subsequent personalized benchmark generation and anomaly detection.

[0026] S2. Retrieve the historical inspection dataset of the device based on the unique number. The dataset contains paired historical infrared thermal images and operating parameters. Use this dataset to train a conditional generative adversarial network. The network takes the operating parameters as input and outputs the corresponding normal baseline infrared thermal image.

[0027] Furthermore, in S2, the specific steps for outputting the corresponding normal reference infrared thermal image include:

[0028] All historical inspection records of the device are retrieved from the historical inspection database based on the unique number. Each record contains a historical infrared thermal image and the corresponding historical load current and historical ambient temperature, forming a paired dataset.

[0029] The number of samples in the paired dataset is counted. If the number of samples is lower than a preset threshold, a cold start strategy is executed to expand the training samples.

[0030] Using operating condition parameters as input and corresponding historical infrared thermal images as output ground truth, a conditional generative adversarial network is trained. The network contains a generator and a discriminator. Through alternating optimization, the generator can output the corresponding normal baseline infrared thermal image based on the input operating condition parameters.

[0031] The trained conditional generative adversarial network is evaluated using a reserved test dataset. When the structural similarity index and peak signal-to-noise ratio between the generated heatmap and the real heatmap both meet the preset quality standards, the model is confirmed to be ready for use.

[0032] Furthermore, the specific steps of implementing a cold start strategy to expand the training samples include:

[0033] Obtain the three-dimensional geometric model and material thermal properties of the device, and establish an electrothermal coupled finite element simulation model.

[0034] Different combinations of load current and ambient temperature are set as boundary conditions in the simulation model. The surface temperature field distribution of the equipment under each combination is generated by finite element calculation and rendered as a synthetic infrared thermal image.

[0035] The synthesized infrared thermal images were paired with the corresponding load current and ambient temperature, and added to the paired dataset as supplementary training samples.

[0036] Furthermore, the steps of implementing a cold start strategy to expand the training samples also include:

[0037] Obtain pre-trained source conditional generative adversarial network models for devices of the same model;

[0038] Using a small amount of existing real historical inspection data from the device as fine-tuning samples, the generator and discriminator of the source model are incrementally trained in a limited number of rounds to adapt the model to the individual thermal characteristics of the current device.

[0039] The fine-tuned model is used as the conditional generative adversarial network for the current device, and subsequent real inspection data is collected to gradually replace the simulation samples or supplement the fine-tuned samples.

[0040] Specifically, the core of this step is to train a dedicated conditional generative adversarial network for the current device, enabling it to generate a baseline infrared thermal image that the device should function normally under given load current and ambient temperature. Based on the device's unique ID obtained in step S1, all historical inspection records for the device are retrieved from the historical inspection database. Each record contains a historical infrared thermal image and its corresponding historical load current and historical ambient temperature. The retrieval results are then used to form a paired dataset. Where M is the total number of historical records, For the i-th historical infrared thermal image, This corresponds to the load current. This corresponds to the ambient temperature.

[0041] Count the number of samples M in the paired dataset D. If M is lower than a preset threshold... If so, a cold start strategy is executed to expand the training samples. In this embodiment, =20. The cold start strategy provides the following two optional paths. Path 1: Generation of synthetic data based on finite element simulation:

[0042] First, the three-dimensional geometric model and material thermophysical parameters of the device, including density, specific heat capacity, thermal conductivity, and resistivity, are obtained. An electrothermal coupling simulation model is then established using finite element analysis software. This model converts the current load into a Joule heat source and solves the steady-state heat conduction equation.

[0043] ;

[0044] in Where is the thermal conductivity, T is the temperature, and q is the volumetric heat source density, calculated from the current density and resistivity.

[0045] Then, different load currents are set in the simulation model. With ambient temperature Combinations are used as boundary conditions. In this embodiment, the load current is incremented in 50A increments according to the actual operating range, and the ambient temperature is incremented in 5°C increments, generating approximately 100 combinations. For example, for a disconnector with a rated current of 600A, the load current is taken at 7 levels: 200A, 300A, 400A, 500A, 600A, 700A, and 800A, and the ambient temperature is taken at 6 levels: -10°C, 0°C, 10°C, 20°C, 30°C, and 40°C, generating 42 boundary conditions. The increments can be appropriately increased based on equipment parameters. Finite element analysis is performed on each boundary condition to obtain the steady-state temperature field distribution on the equipment surface, which is then rendered as a synthetic infrared thermal image. ,Will It is added to dataset D as a supplementary training sample.

[0046] Path Two: Model Fine-tuning Based on Transfer Learning

[0047] First, obtain the pre-trained source conditional generative adversarial network model of the same model of equipment. The source model has been pre-trained on a large amount of historical data from other devices of the same model. Then, using a small amount of real historical inspection data already available on the current device (3 to 5 sets of samples in this embodiment), the generator G and discriminator D of the source model are incrementally trained for a limited number of rounds. A small learning rate is used during fine-tuning, for example, one-tenth of the original learning rate, and the training rounds do not exceed 50 rounds. After fine-tuning, the resulting model is used as the conditional generative adversarial network for the current device. Subsequently, as real inspection data continues to accumulate, the simulated samples are gradually replaced with real data or supplemented with fine-tuned samples to achieve continuous optimization of the model.

[0048] Regardless of the path chosen, once the sample size of dataset D reaches a preset threshold, the conditional generative adversarial network (GAN) is trained as follows. This network comprises two modules: a generator G and a discriminator D. The generator employs an encoder-decoder structure, specifically a U-Net architecture with skip connections. The encoder consists of multiple convolutional and pooling layers for progressively extracting image features; the decoder consists of multiple upsampling and convolutional layers for progressively restoring the image's spatial resolution. Operating parameters... At each level of the encoder, features are fused with image features through feature concatenation. The generator takes a random noise vector z and operating conditions c as input and outputs the generated infrared thermal image. The discriminator D uses a PatchGAN structure, which divides the input image into multiple N×N blocks, outputs a true / false probability for each block, and finally averages the results as the overall discrimination result. The input to the discriminator is the infrared heatmap I and the operating condition c, and the output is the probability that the image is a real sample under the given operating condition. The training process uses an alternating optimization strategy. In each iteration, the generator G is fixed first, and the discriminator D is updated. The loss function of the discriminator is:

[0049] ;

[0050] in To obtain the true infrared thermal images sampled from dataset D, For the true data distribution, For noise distribution, this embodiment uses a Gaussian distribution.

[0051] Then, with the discriminator D fixed, the generator G is updated. The generator's loss function includes adversarial loss and perceptual loss:

[0052] ;

[0053] in To perceive the loss, high-level features of the generated image and the real image are extracted through a pre-trained VGG network, and the mean square error between the feature maps is calculated. As a balance coefficient, this embodiment takes... =10. The perceptual loss ensures that the generated image is semantically consistent with the real normal heatmap, rather than merely a mechanical approximation of pixel values. The training process continues until both the discriminator loss and the generator loss converge. In this embodiment, the Adam optimizer is used with an initial learning rate of 0.0002, a batch size of 4, and 200 training epochs.

[0054] The trained conditional generative adversarial network was evaluated using a reserved test dataset (approximately 15% to 20% of the total historical data, and not used in training). Two quantitative metrics were used for evaluation: the structural similarity index (SSIM) and the peak signal-to-noise ratio (PSNR). For each sample in the test set, a heatmap was generated. Compared to real heatmaps Between SSIM and PSNR:

[0055] ;

[0056] in , are the means of the x and y values ​​of the image, respectively. , , where is the variance. For covariance, , It is a stability constant. PSNR is defined as:

[0057] ;

[0058] Where MAX is the maximum pixel value in the image, and MSE is the mean square error.

[0059] The model is considered usable when the average SSIM is not lower than 0.85 and the average PSNR is not lower than 28dB on the test set; otherwise, retrain the model by increasing the amount of training data or adjusting the network hyperparameters. After acceptance, the model is stored as a dedicated model for the current device and used for benchmark heatmap generation in subsequent steps.

[0060] S3. Input the current load current and current ambient temperature into the trained conditional generative adversarial network to generate a normal reference infrared thermal image under the current operating conditions.

[0061] Furthermore, in S3, the steps for generating a normal reference infrared thermal image under the current operating conditions specifically include:

[0062] Combine the current load current and the current ambient temperature into an input operating condition feature vector;

[0063] The input working condition feature vector is fed into the generator of the conditional generative adversarial network. The generator adopts an encoder-decoder structure and fuses the working condition feature vector with the image features step by step during the encoding process.

[0064] The generator outputs a normal reference infrared thermal image with the same pixel size as the current infrared thermal image, where each pixel value represents the predicted normal temperature value of the corresponding location on the device surface under this operating condition.

[0065] Specifically, firstly, the current load current obtained in step S1 is... With current ambient temperature Combined into input working condition feature vector In this embodiment, the load current and ambient temperature are normalized to a range of [0,1]. The normalization formula is as follows:

[0066] ;

[0067] in , These are the maximum and minimum historical load currents of the equipment. , These represent the maximum and minimum historical ambient temperatures.

[0068] The normalized working condition feature vector Input the generator G of the conditional generative adversarial network trained in step S2. This generator employs an encoder-decoder structure, specifically a U-Net architecture with skip connections. The encoder part consists of four convolutional blocks, each containing two convolutional layers and one max-pooling layer, used for progressive downsampling to extract image features. At each level of the encoder, the conditional feature vector... The generator fuses the feature map with the current layer's image feature map through feature concatenation. Specifically, the feature vector is copied and expanded into a tensor with the same spatial size as the feature map, and then concatenated along the channel dimension. The generator's input also includes a random noise vector z sampled from a standard normal distribution to increase output diversity. During model training, z is randomly sampled; during model deployment and inference, z is fixed as a zero vector or a constant vector to ensure that a unique and reproducible baseline infrared thermal image is generated for the same working conditions. The generator's forward propagation process can be represented as:

[0069] ;

[0070] in This is the generated standard baseline infrared thermal image.

[0071] The decoder section of the generator consists of four upsampling blocks. Each upsampling block contains one transposed convolutional layer and two convolutional layers. Skip connections are used to concatenate the feature maps of the corresponding encoder layers with the upsampled feature maps to recover the spatial details of the image. The decoder output layer uses a 1×1 convolutional layer to map the feature maps into a single-channel grayscale image, with pixel values ​​ranging from the original infrared thermal image. Figure 1 "Zhi" represents the predicted normal temperature value at the corresponding location on the equipment surface under this operating condition.

[0072] Finally, the generator output is compared with the current infrared thermal image acquired in step S1. Normal reference infrared thermogram with the same pixel size The pixel value of this baseline heatmap (x,y) represents the operating conditions The generated baseline thermal image should represent the predicted normal temperature at coordinates (x, y) on the device surface. This baseline thermal image will then be sent to step S4 for image spatial registration.

[0073] S4. Perform image spatial registration between the current infrared thermal image and the normal reference infrared thermal image to establish a spatial mapping relationship between the two.

[0074] Furthermore, in S4, the step of performing image spatial registration between the current infrared thermal image and the normal reference infrared thermal image to establish the spatial mapping relationship between the two specifically includes:

[0075] Multiple feature points are extracted from the current infrared thermal image and the normal reference infrared thermal image respectively, and a feature descriptor with scale and rotation invariance is generated for each feature point;

[0076] The feature descriptors of the two images are matched to establish the correspondence between feature points;

[0077] Based on the matched feature point pairs, calculate the spatial transformation matrix between the two images;

[0078] The normal reference infrared thermal image is geometrically transformed according to the spatial transformation matrix to align it with the pixel coordinate system of the current infrared thermal image;

[0079] The number of matched feature point pairs is checked. If the number is lower than the preset matching threshold, the registration is determined to be unsuccessful, the subsequent difference operation is stopped, and the UAV is instructed to re-acquire the current infrared thermal image.

[0080] Specifically, the current infrared thermal image acquired in step S1 is used as a reference. and the normal reference infrared thermal image generated in step S3 Feature points are extracted. This embodiment employs a scale-invariant feature transform algorithm. First, a Gaussian difference pyramid is constructed, and local extrema are detected in each scale space as candidate feature points. For each candidate feature point, a three-dimensional quadratic function is fitted to accurately locate it to the sub-pixel level, and low-contrast points and edge response points are filtered out. Then, a principal direction is assigned to each feature point, obtained based on gradient histogram statistics, making the descriptor rotation-invariant. With each feature point as the center, its neighborhood is divided into 4×4 sub-regions. Gradient histograms in 8 directions are calculated within each sub-region, forming a 4×4×8=128-dimensional feature descriptor vector d. This descriptor is scale-invariant and rotation-invariant.

[0081] The feature descriptors of the two images are matched. A nearest neighbor distance ratio matching strategy is used: for For each feature point in the dataset, calculate its descriptor and... Find the nearest Euclidean distance among all feature point descriptors. and the second closest distance ,like If the match is found to be correct, then the pair is accepted. The above method establishes a set of correspondences between feature points in two images. ,in The coordinates of feature points on the current heatmap K represents the coordinates of the corresponding feature point on the baseline heatmap, and K is the number of matching pairs.

[0082] Based on the set of matching feature points Calculate the spatial transformation matrix between the two images. Since the power equipment is a rigid body and the viewpoint change is limited when the drone is hovering, this embodiment uses a rigid transformation model. The transformation matrix H includes the rotation angle. Translational displacement ( , ):

[0083] ;

[0084] Where (x, y) are the pixel coordinates in the baseline heatmap, , The coordinates are the coordinates corresponding to the current heatmap coordinate system after transformation. The transformation parameters that minimize the reprojection error are determined using the least squares method.

[0085] ;

[0086] Normal reference infrared thermogram Geometric transformation is performed according to the aforementioned spatial transformation matrix H. In this embodiment, bilinear interpolation is used for resampling to obtain the registered reference heatmap. To make it consistent with the current infrared thermal image Align the pixel coordinate system.

[0087] After registration, verify the number K of matching feature point pairs. If K is lower than the preset matching threshold... In this embodiment, If the value is 10, registration is considered failed, indicating a lack of sufficient common structural features between the two images to establish a reliable spatial mapping. At this point, subsequent difference operations are stopped, and the drone is controlled to re-acquire the current infrared thermal image, returning to step S1. If registration fails three times consecutively, the current inspection is terminated, and the device is marked as "registration unavailable," automatically retrying on the next inspection. If the registration is successful, the reference heatmap after registration will be generated. Compared with the current heatmap Proceed to step S5 to perform pixel-by-pixel difference calculation.

[0088] S5. Perform pixel-by-pixel differential calculations on the registered current infrared thermal image and the normal reference infrared thermal image to obtain the residual thermal image.

[0089] Furthermore, in S5, the specific steps for obtaining the residual heatmap include:

[0090] Read the temperature value of each pixel in the current registered infrared thermal image, and the temperature value of the corresponding pixel in the normal registered reference infrared thermal image;

[0091] For each pixel location, the temperature value of the current infrared thermal image is subtracted from the temperature value of the normal reference infrared thermal image to obtain the residual value of that pixel.

[0092] The residual values ​​of all pixels are arranged spatially according to the original image to form a residual heatmap, where the value of each pixel represents the deviation between the measured temperature at that location and the normal reference temperature.

[0093] Specifically, read the registered current infrared thermal image output in step S4. and the registered normal reference infrared thermogram The two images have the same pixel size, denoted as H×W, where H is the number of pixels in the height direction and W is the number of pixels in the width direction. For each pixel coordinate (x, y), where... Obtain each and The temperature value. For each pixel location, perform a subtraction operation to obtain the residual value of that pixel. :

[0094] ;

[0095] in The unit is Celsius. When A value >0 indicates that the measured temperature at that location is higher than the normal reference temperature; when When ≈0, it indicates that the measured temperature is consistent with the normal reference; when When <0, it indicates that the measured temperature is lower than the normal reference temperature.

[0096] The residual values ​​r(x,y) of all pixels are arranged in a two-dimensional matrix according to the spatial arrangement of the original image, namely the residual heatmap R, which also has the size of H×W:

[0097] ;

[0098] In the residual heatmap R, the value of each pixel represents the deviation between the measured temperature at that location and the normal reference temperature. Positive deviations indicate potential thermal defect areas.

[0099] After generating the residual heatmap R, it is used as the input for step S6 for adaptive threshold segmentation and extraction of thermal defect anomaly regions.

[0100] S6. Based on the residual values ​​of each pixel in the residual heatmap, an adaptive threshold segmentation method is used to extract connected regions with residual values ​​greater than the dynamic threshold. After morphological post-processing, the thermal defect abnormal region is obtained.

[0101] Furthermore, in S6, the step of extracting connected regions with residual values ​​greater than the dynamic threshold using the adaptive threshold segmentation method specifically includes:

[0102] Calculate the mean and standard deviation of all pixel residual values ​​in the historical residual heatmap of the device, and set the current dynamic threshold to the sum of K times the mean and standard deviation;

[0103] Pixels with residual values ​​greater than the dynamic threshold in the residual heatmap are marked as candidate abnormal pixels;

[0104] Connectivity analysis is performed on candidate anomalous pixels to extract all connected pixel regions as candidate anomalous regions;

[0105] Morphological post-processing is performed on candidate anomaly regions;

[0106] The areas preserved after morphological post-processing are identified as thermal defect anomalous areas, and the area and maximum temperature rise of each area are calculated.

[0107] Furthermore, the specific steps for morphological post-processing of candidate anomaly regions include:

[0108] Calculate the pixel area of ​​each candidate anomaly region and remove isolated regions whose area is smaller than the preset minimum defect area;

[0109] Calculate the minimum Euclidean distance between every two candidate anomaly regions, and merge adjacent regions whose distance is less than a preset spacing threshold into the same region;

[0110] The output retains the thermal defect anomaly regions after area filtering and region merging.

[0111] Specifically, first, determine the dynamic segmentation threshold of the current residual heatmap. Then, calculate the global mean of pixel residual values ​​in all historical residual heatmaps of this device. and standard deviation The specific statistical method is as follows: all pixel residual values ​​of all historical residual heatmaps (each with size H×W) are flattened and merged into a one-dimensional array. The arithmetic mean μ and sample standard deviation σ of this array are calculated. For the first inspection or when the number of historical residual heatmaps is less than 3, a preset fixed threshold is used. As an alternative, when the historical residual heatmap accumulates to 3 or more times, the current dynamic threshold is calculated. for:

[0112] ;

[0113] Where K is a preset coefficient, and in this embodiment, K=3 is chosen so that the residual value is greater than +3 The pixels in question have a very low statistical probability of being random noise.

[0114] The residual value of each pixel in the residual heatmap R obtained in step S5. With the current dynamic threshold In comparison, if If a pixel is found to be an anomalous pixel, it is marked as such; otherwise, it is marked as a normal pixel. The marking results form a binary mask image B.

[0115] ;

[0116] Connectivity analysis is performed on candidate anomalous pixels in the binary mask B. An eight-connected neighborhood criterion is used, meaning that two candidate anomalous pixels adjacent in the horizontal, vertical, or diagonal directions are considered to belong to the same connected region. Each connected region is labeled using a two-pass scanning algorithm, assigned a unique number L to each region, and the set of coordinates of all pixels contained within that region is recorded. Output all connected regions as a set of candidate outliers. , where N is the total number of connected regions.

[0117] Morphological post-processing is performed on the candidate anomaly region set A. First, the morphological post-processing of each candidate anomaly region is calculated. pixel area That is, the number of pixels contained in that region. If Less than the preset minimum defect area threshold If the area is identified as an isolated noise point, it is removed from A. In this embodiment, we take... =5 pixels. Under the conditions of image resolution of 1024×768 and shooting distance of 3 to 5 meters, the physical area corresponding to 5 pixels is approximately 0.5 to 1.0 square centimeters. The minimum defect area threshold can be dynamically adjusted according to the actual detection accuracy requirements, or the physical size threshold can be converted into a pixel threshold through calibration.

[0118] For the remaining candidate outlier regions after area filtering, calculate the minimum Euclidean distance between every two regions. For each region... and Its minimum distance is defined as:

[0119] ;

[0120] in Let be the Euclidean distance. Less than the preset spacing threshold In this embodiment, =3 pixels, then the area and The regions are merged into a single region, and the set of coordinates for the merged region is: ∪ Repeat this process until the minimum distance between all regions is greater than or equal to... .

[0121] The regions retained after area filtering and region merging are determined as the final thermal defect anomaly regions. For each thermal defect anomaly region, its area (number of pixels) and maximum temperature rise are calculated. The maximum temperature rise is defined as the maximum value of the pixel residual values ​​within that region.

[0122] ;

[0123] in This is the set of pixel coordinates for this region.

[0124] Output the location contour, area, and maximum temperature rise of each thermal defect abnormal area for trend analysis and predictive maintenance decision-making in step S7.

[0125] S7. Record the area change trend of the thermal defect abnormal area during multiple inspections. When the trend meets the preset conditions, shorten the inspection cycle of the equipment and output an active warning.

[0126] Furthermore, in S7, when the trend meets preset conditions, the steps to shorten the inspection cycle of the device and output an active warning specifically include:

[0127] The area of ​​the thermal defect anomaly area obtained from each inspection and the corresponding timestamp are stored in the health record of the equipment, forming an area sequence arranged in chronological order;

[0128] When the length of the area sequence reaches the preset minimum number of consecutive times N, extract the area values ​​of the most recent N consecutive times;

[0129] Determine whether the area values ​​of N consecutive times simultaneously satisfy the condition that the area of ​​each subsequent time is not less than the area of ​​the previous time, and that the cumulative growth rate of the area of ​​the Nth time relative to the area of ​​the first time exceeds a preset growth threshold.

[0130] If the above conditions are met, the equipment inspection cycle will be shortened from the current cycle value to the shortened cycle value.

[0131] Generate proactive early warning information, which includes the device number, the area sequence of the most recent N consecutive times, the cumulative growth rate, and the recommended operation and maintenance measures, and push the information to the operation and maintenance personnel's terminal.

[0132] Specifically, the area of ​​the thermal defect anomaly region obtained from each inspection is recorded as... , where t is the inspection timestamp. The health record database of the device is stored to form an area sequence arranged in chronological order. When the length of the area sequence reaches a preset minimum number of consecutive counts N, extract the area values ​​of the most recent N consecutive counts. In this embodiment, N=3.

[0133] Determine whether the N consecutive area values ​​simultaneously satisfy the following two conditions: Condition 1: Monotonically increasing. That is, for i = 1, 2, ..., N−1, the following holds true. .

[0134] Condition 2: The cumulative growth rate exceeds the preset growth threshold. The cumulative growth rate R is defined as the relative increase in area of ​​the Nth time relative to the area of ​​the first time:

[0135] ;

[0136] in This refers to the area inspected most recently. This represents the area inspected during the first inspection N times ago. In this embodiment, we take... =50%.

[0137] If both of the above conditions are met, the equipment is determined to be experiencing a trend of deterioration. In this case, the equipment's inspection cycle is adjusted from the current cycle value. Shorten to the new cycle value The period shortening rate is positively correlated with the cumulative growth rate R, specifically:

[0138] ;

[0139] in As the maximum growth rate limit is set, this embodiment takes... =100%, meaning when the cumulative growth rate exceeds 100%, the period shortens to 20% of the current period value. The shortened period value. It is limited to between 20% and 80% of the initial period value.

[0140] Generate proactive early warning information. This information includes the device ID and the area sequence of the most recent N consecutive times. Cumulative growth rate R, current period value The document also includes recommended maintenance measures. In this embodiment, the recommended measure is "Please arrange for personnel to conduct an on-site review of the thermal defect area of ​​the equipment." This information is pushed to the maintenance personnel's terminal via wireless network and simultaneously highlighted on the UAV inspection management system interface.

[0141] If, in subsequent inspections, the area of ​​the abnormal region tends to stabilize or decline (i.e., the cumulative growth rate decreases to less than 1%), If the following conditions are no longer met (and the monotonically increasing condition is no longer satisfied), the system gradually restores the inspection cycle of the device to the initial set value and marks the warning as lifted. In this embodiment, the recovery strategy is: each time the stability condition is met, the inspection cycle is extended to 1.2 times the current cycle until the initial cycle value is reached; if the stability condition is met three times consecutively, it can also be directly restored to the initial cycle. If, during continuous observation after shortening the cycle, the area of ​​the abnormal region continues to increase and exceeds the fault alarm threshold, for example, the maximum temperature rise exceeds... If the system fails to detect the fault, it will immediately generate a fault alarm and lock in the minimum inspection cycle, such as once every 7 days, until manual intervention is required to confirm or the repair is completed.

[0142] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for image recognition during unmanned aerial vehicle (UAV) inspections, characterized in that, include: S1. Control the infrared thermal imager carried by the drone to collect the current infrared thermal image of the power equipment to be tested, and simultaneously collect the unique number of the equipment, the current load current and the current ambient temperature; S2. Retrieve the historical inspection dataset of the device according to the unique number. The dataset contains paired historical infrared thermal images and operating parameters. Use the dataset to train a conditional generative adversarial network. The network takes the operating parameters as input and outputs the corresponding normal baseline infrared thermal image. S3. Input the current load current and current ambient temperature into the trained conditional generative adversarial network to generate a normal reference infrared thermal image under the current operating conditions. S4. Perform image spatial registration between the current infrared thermal image and the normal reference infrared thermal image to establish a spatial mapping relationship between the two. S5. Perform pixel-by-pixel difference calculation on the registered current infrared thermal image and the normal reference infrared thermal image to obtain the residual thermal image; S6. Based on the residual values ​​of each pixel in the residual heatmap, an adaptive threshold segmentation method is used to extract connected regions with residual values ​​greater than the dynamic threshold, and the thermal defect abnormal region is obtained after morphological post-processing. S7. Record the area change trend of the thermal defect abnormal area during multiple inspections. When the trend meets the preset conditions, shorten the inspection cycle of the equipment and output an active warning.

2. The UAV inspection image recognition method according to claim 1, characterized in that, In step S2, the step of outputting the corresponding normal reference infrared thermal image specifically includes: Based on the unique number, retrieve all historical inspection records of the device from the historical inspection database. Each record contains a historical infrared thermal image and the corresponding historical load current and historical ambient temperature, forming a paired dataset. The number of samples in the paired dataset is counted. If the number of samples is lower than a preset threshold, a cold start strategy is executed to expand the training samples. Using operating condition parameters as input and corresponding historical infrared thermal images as output ground truth, a conditional generative adversarial network is trained. The network includes a generator and a discriminator. Through alternating optimization, the generator can output the corresponding normal baseline infrared thermal image based on the input operating condition parameters. The trained conditional generative adversarial network is evaluated using a reserved test dataset. When the structural similarity index and peak signal-to-noise ratio between the generated heatmap and the real heatmap both meet the preset quality standards, the model is confirmed to be ready for use.

3. The UAV inspection image recognition method according to claim 2, characterized in that, The steps of implementing the cold start strategy to expand the training samples specifically include: Obtain the three-dimensional geometric model and material thermal properties of the device, and establish an electrothermal coupled finite element simulation model. In the simulation model, different combinations of load current and ambient temperature are set as boundary conditions. The surface temperature field distribution of the equipment under each combination is generated by finite element calculation and rendered as a synthetic infrared thermal image. The synthesized infrared thermal image is paired with the corresponding load current and ambient temperature, and added to the paired dataset as supplementary training samples.

4. The UAV inspection image recognition method according to claim 2, characterized in that, The step of implementing the cold start strategy to expand the training samples also includes: Obtain pre-trained source conditional generative adversarial network models for devices of the same model; Using a small amount of existing real historical inspection data from the device as fine-tuning samples, the generator and discriminator of the source model are incrementally trained in a limited number of rounds to adapt the model to the individual thermal characteristics of the current device. The fine-tuned model is used as the conditional generative adversarial network for the current device, and subsequent real inspection data is collected to gradually replace the simulation samples or supplement the fine-tuned samples.

5. The UAV inspection image recognition method according to claim 1, characterized in that, In step S3, the step of generating a normal reference infrared thermal image under the current operating condition specifically includes: The current load current and the current ambient temperature are combined into an input operating condition feature vector; The input working condition feature vector is input into the generator of the conditional generative adversarial network. The generator adopts an encoder-decoder structure and fuses the working condition feature vector with the image features step by step during the encoding process. The generator outputs a normal reference infrared thermal image with the same pixel size as the current infrared thermal image, wherein each pixel value represents the predicted normal temperature value of the corresponding location on the surface of the device under the operating conditions.

6. The UAV inspection image recognition method according to claim 1, characterized in that, In step S4, the step of performing image spatial registration between the current infrared thermal image and the normal reference infrared thermal image to establish a spatial mapping relationship between the two specifically includes: Multiple feature points are extracted from the current infrared thermal image and the normal reference infrared thermal image respectively, and a feature descriptor with scale and rotation invariance is generated for each feature point; The feature descriptors of the two images are matched to establish the correspondence between feature points; Based on the matched feature point pairs, calculate the spatial transformation matrix between the two images; The normal reference infrared thermal image is geometrically transformed according to the spatial transformation matrix to align it with the pixel coordinate system of the current infrared thermal image; The number of matched feature point pairs is checked. If the number is lower than the preset matching threshold, the registration is determined to be unsuccessful, the subsequent differential operation is stopped, and the UAV is instructed to re-acquire the current infrared thermal image.

7. The UAV inspection image recognition method according to claim 1, characterized in that, In step S5, the step of obtaining the residual heatmap specifically includes: Read the temperature value of each pixel in the current registered infrared thermal image, and the temperature value of the corresponding pixel in the normal registered reference infrared thermal image; For each pixel location, the temperature value of the current infrared thermal image is subtracted from the temperature value of the normal reference infrared thermal image to obtain the residual value of that pixel. The residual values ​​of all pixels are arranged spatially according to the original image to form a residual heatmap, where the value of each pixel represents the deviation between the measured temperature at that location and the normal reference temperature.

8. The UAV inspection image recognition method according to claim 1, characterized in that, In step S6, the step of extracting connected regions with residual values ​​greater than the dynamic threshold using an adaptive threshold segmentation method specifically includes: The mean and standard deviation of all pixel residual values ​​in the historical residual heatmap of the device are statistically analyzed, and the current dynamic threshold is set to the sum of K times the mean and standard deviation. Pixels in the residual heatmap whose residual value is greater than the dynamic threshold are marked as candidate abnormal pixels; Connectivity analysis is performed on the candidate abnormal pixels to extract all connected pixel regions as candidate abnormal regions; Morphological post-processing is performed on the candidate abnormal regions; The areas preserved after morphological post-processing are identified as thermal defect anomalous areas, and the area and maximum temperature rise of each area are calculated.

9. The UAV inspection image recognition method according to claim 8, characterized in that, The specific steps for performing morphological post-processing on the candidate anomaly regions include: Calculate the pixel area of ​​each candidate anomaly region and remove isolated regions whose area is smaller than the preset minimum defect area; Calculate the minimum Euclidean distance between every two candidate anomaly regions, and merge adjacent regions whose distance is less than a preset spacing threshold into the same region; The output retains the thermal defect anomaly regions after area filtering and region merging.

10. The UAV inspection image recognition method according to claim 1, characterized in that, In step S7, when the trend meets preset conditions, the step of shortening the inspection cycle of the device and outputting an active warning specifically includes: The area of ​​the thermal defect anomaly area obtained from each inspection and the corresponding timestamp are stored in the health record of the equipment, forming an area sequence arranged in chronological order; When the length of the area sequence reaches the preset minimum number of consecutive times N, the area values ​​of the most recent N consecutive times are extracted; Determine whether the area values ​​of the N consecutive times simultaneously satisfy the condition that the area of ​​each subsequent time is not less than the area of ​​the previous time, and the cumulative growth rate of the area of ​​the Nth time relative to the area of ​​the first time exceeds a preset growth threshold. If the above conditions are met, the inspection cycle of the equipment will be shortened from the current cycle value to the shortened cycle value. Generate proactive early warning information, which includes the device number, the most recent consecutive N area sequences, the cumulative growth rate, and suggested maintenance measures, and push the information to the maintenance personnel's terminal.

Images