Unmanned aerial vehicle wind power blade defect dynamic detection method based on AI vision

Abstract

The invention discloses an unmanned aerial vehicle wind power blade defect dynamic detection method based on AI vision, and relates to the technical field of JSLYMC, and the method comprises the following steps: S1, collecting an original dynamic image sequence; s2, generating an aligned image sequence; s3, generating a standardized blade surface image sequence; s4, constructing a blade topological structure diagram; s5, inputting the graph structure data into the improved TransGAT model, and outputting a defect candidate set; s6, inputting the defect candidate set into the FairMOT model, and outputting a defect space-time trajectory; and S7, generating a structured detection report. The method overcomes the limitations of dependence on manual inspection, poor dynamic adaptability and insufficient identification precision in a traditional wind power blade defect detection method, and provides an efficient and accurate solution for unmanned aerial vehicle wind power blade automatic inspection and intelligent maintenance decision.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence visual inspection technology, and in particular to a dynamic detection method for defects in wind turbine blades using unmanned aerial vehicles based on AI vision. Background Technology

[0002] With the rapid development of the wind power industry and the continuous increase in installed capacity, wind turbine blades, as the core components for capturing wind energy, directly determine the safety and economic benefits of the entire wind power system. In modern wind power operation and maintenance management, accurate detection and real-time monitoring of blade defects are not only important means to prevent catastrophic accidents, but also a key link in realizing the transformation of wind farms from "periodic passive maintenance" to "proactive maintenance based on conditions." However, most current methods for detecting wind turbine blade defects rely on manual high-altitude inspections, telescope observations, or offline analysis based on simple image processing, which are high-risk and inefficient. While these traditional methods can meet basic operation and maintenance needs, they lack the ability to capture and intelligently analyze the dynamic characteristics of blades under high-speed rotation conditions, making them unsuitable for the real-time monitoring needs of complex meteorological environments and large-scale blade movements in wind farms.

[0003] The main limitations of traditional wind turbine blade defect detection methods lie in their poor adaptability to detection environments and insufficient robustness in target recognition. Existing methods typically rely on static image assumptions or simple image differencing to determine the blade surface condition. These methods often fail significantly when faced with motion blur caused by high-speed blade rotation, dynamic background interference, and drastic changes in lighting conditions. When the acquired blade image data exhibits multi-scale small targets, complex background textures, nonlinear geometric distortions, and strong time-varying features, the generalization performance and adaptability of traditional detection algorithms are severely limited. In particular, when dealing with minute defects such as micro-cracks, spalling, and fiber breakage on the blade surface, traditional general-purpose target detection algorithms struggle to efficiently and accurately extract defect features, leading to high false negative rates or frequent false alarms, severely impacting the accurate assessment of the blade's structural health.

[0004] Furthermore, traditional methods often neglect the spatiotemporal continuity of defects within a continuous time series and the physical topology of the blade during defect detection, making it difficult to comprehensively utilize inter-frame motion correlation information and prior knowledge of the blade surface structure. For example, during dynamic inspection by UAVs, the blades continuously rotate and the imaging perspective constantly changes. Traditional single-frame image analysis methods cannot effectively establish the temporal correspondence of defects, making it difficult to form a complete spatiotemporal trajectory of the defects. Even when some methods employ deep learning techniques, they fail to fully explore the graph structure topological features and deep feature relationships in the dynamic motion patterns of blade images, making it difficult to achieve efficient, accurate, and real-time defect localization, tracking, and three-dimensional quantitative evaluation.

[0005] Therefore, how to provide a dynamic detection method for wind turbine blade defects based on AI vision is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] This invention proposes a dynamic defect detection method for wind turbine blades using AI vision. By combining the FairMOT model with an improved TransGAT model, it can more accurately detect and identify defects and dynamic evolution trends of wind turbine blades. This method can not only automatically extract the spatiotemporal features and texture details of wind turbine blades under high-speed rotation, but also combine the blade surface topology information. Through frequency domain enhanced graph attention units and edge-constrained multi-head attention units, it achieves effective fusion of topological and temporal features, significantly improving the robustness and real-time performance of wind turbine blade defect detection. By introducing a spatiotemporal context enhancement sublayer, a relative position encoding injection sublayer, and a causal multi-head attention computation sublayer, the FairMOT backbone network model can adaptively fuse geometric motion priors and effectively solve the problem of temporal information leakage, capturing subtle changes of defect targets in dynamic backgrounds in real time. Combining the structure-aware inference of the improved TransGAT model with the cross-frame trajectory correlation capability of the FairMOT model, it achieves accurate generation and 3D inversion of defect spatiotemporal trajectories, thereby realizing dynamic detection and accurate quantification of wind turbine blade defects. This invention overcomes the limitations of traditional wind turbine blade defect detection methods, such as reliance on manual inspection, poor dynamic adaptability, and insufficient identification accuracy, and provides an efficient and accurate solution for automated inspection and intelligent maintenance decision-making of wind turbine blades using drones.

[0007] A dynamic detection method for defects in wind turbine blades using an unmanned aerial vehicle (UAV) based on AI vision, according to an embodiment of the present invention, includes the following steps:

[0008] S1. Deploy visible light imaging sensors on the UAV to continuously image the wind turbine blades in a rotating state and collect the original dynamic image sequence.

[0009] S2. Based on the original dynamic image sequence, calculate the inter-frame motion vector using the optical flow method and apply smoothness constraints, perform spatiotemporal registration operation, and output the aligned image sequence;

[0010] S3. Input the aligned image sequence into the rotating target adaptive perspective correction network, construct the perspective transformation matrix based on the blade rotation phase estimation parameters, perform geometric distortion correction operation, and output a standardized blade surface image sequence.

[0011] S4. Construct a blade topology graph based on a standardized blade surface image sequence, map image superpixel regions as nodes, map spatial adjacency relationships and texture prior relationships as edges, and generate graph structure data.

[0012] S5. Input the graph structure data into the improved TransGAT model, use the frequency domain enhanced attention unit to extract robust node features, map the topological edge relationship to the attention bias through the edge constraint multi-head attention unit to perform explicit modulation, and construct the feature bidirectional propagation path through the bidirectional spatiotemporal awareness fusion unit. After fusing the global structural context, perform structural awareness reasoning and output a defect candidate set.

[0013] S6. Input the defect candidate set into the FairMOT model, extract and enhance the spatiotemporal features, output the detection results and Re-ID features in parallel through the dual-task prediction head layer, and output the spatiotemporal trajectory of the defect by combining appearance similarity and motion distance through the cross-frame trajectory association layer.

[0014] S7. Extract the defect quantification feature set based on the defect spatiotemporal trajectory, and combine it with the spatiotemporally aligned UAV precise pose calculation to obtain the global three-dimensional spatial coordinates and normal angle, and generate a structured inspection report.

[0015] Optionally, S2 specifically includes:

[0016] S21. Extract feature points from the original dynamic image sequence, calculate the dense optical flow field of adjacent inter-frame pixels using a block matching algorithm, establish inter-frame pixel-level motion mapping relationship, and generate an initial motion vector field.

[0017] S22. Based on the preset continuity and smoothness constraints of the motion vector field, the RANSAC algorithm is used to remove abnormal motion data in the initial motion vector field to obtain an optimized motion vector field, and the inter-frame global transformation matrix is ​​calculated based on the optimized motion vector field.

[0018] S23. Use the inter-frame global transformation matrix to perform geometric reverse distortion and grayscale interpolation on each frame of the original dynamic image sequence to eliminate inter-frame jitter and displacement errors, and output the motion-compensated aligned image sequence.

[0019] Optionally, S3 specifically includes:

[0020] S31. Input the aligned image sequence into the feature pyramid backbone network, extract multi-scale leaf appearance features, fuse the temporal context information after motion compensation, and output a target feature map containing spatial geometric structure features.

[0021] S32. Input the target feature map into the blade rotation phase estimation subnetwork, predict the blade rotation angle and center offset through the convolutional regression layer, and construct the perspective transformation matrix based on the rotation phase estimation parameters.

[0022] S33. Perform geometric reverse distortion operation on the target feature map using the perspective transformation matrix, perform bilinear interpolation sampling and pixel filling, complete the perspective distortion correction of the blade surface image, and output a globally coordinate standardized blade surface image sequence.

[0023] Optionally, S4 specifically includes:

[0024] S41. Perform region segmentation processing on the globally coordinate-normalized blade surface image sequence, generate a set of superpixel regions by combining the preset texture gradient and color consistency constraints, and extract local texture statistical features for each superpixel region.

[0025] S42. Calculate the geometric distance and connectivity between regions based on the spatial location distribution of superpixel region sets, construct spatial adjacency relationships, and construct texture prior relationships based on the similarity calculation results of local texture statistical features.

[0026] S43. Map the set of superpixel regions to nodes, and map spatial adjacency and texture prior relationships to edges connecting nodes, constructing a leaf topology graph containing node features and edge weights.

[0027] Optionally, the improved TransGAT model includes a frequency domain enhanced graph attention unit, an edge-constrained multi-head attention unit, and a bidirectional spatiotemporal awareness fusion unit:

[0028] The frequency domain enhanced graph attention unit is used to map the node feature tensor of the blade topology graph to the frequency domain, extract the frequency domain statistical feature tensor that is insensitive to changes in illumination, and perform channel-level adaptive weighted fusion with the original time domain node feature tensor to obtain an enhanced node feature tensor that includes anti-interference capability.

[0029] The edge-constrained multi-head attention unit is used to receive the enhanced node feature tensor and construct a heterogeneous attention mechanism composed of a texture-aware head, a geometry-aware head, and a spatial-aware head. Using the edge weight matrix of the leaf topology graph, the spatial adjacency relationship and the texture prior relationship are encoded as bias terms and superimposed on the attention score calculation process. The feature aggregation between nodes is explicitly modulated so that the aggregation weight responds to the semantic features and topological constraints of the nodes at the same time, and outputs a multi-source heterogeneous feature tensor containing edge constraints.

[0030] The bidirectional spatiotemporal perception fusion unit is used to construct a propagation path that diffuses from the central node to neighboring nodes and an aggregation path that converges from neighboring nodes to the central node, using a multi-source heterogeneous feature tensor as input. Through a bidirectional message passing mechanism, the output of the spatial perception head is used to dynamically mask the aggregation range of the texture perception head and integrate long-distance dependencies to output a high-order node feature tensor that fuses global structural information and local texture details.

[0031] The high-order node feature tensor is input into the fully connected decoder for dimensionality reduction, and the defect classification task and coordinate regression task are executed in parallel. The probability distribution of defect categories is output through the Softmax activation function, and the position coordinates of the defect candidate region are output through the identity activation function. Finally, a defect candidate set containing position coordinates and category attributes is generated.

[0032] Optionally, the bidirectional message passing mechanism includes:

[0033] Define the central node as the diffusion and propagation direction of the source node, and define the neighboring nodes as the convergence and aggregation direction of the source node;

[0034] Extract the spatial correlation score matrix output by the spatial sensing head, map it to the 0 to 1 interval using the Sigmoid function, and generate a binary spatial mask matrix based on the set spatial distance threshold.

[0035] The binarized spatial mask matrix is ​​multiplied element-wise with the initial aggregation weight matrix generated by the texture-aware head. The aggregation weights of the corresponding positions that are outside the spatial perception range are reset to 0 using the elements with values ​​of 0 in the spatial mask matrix.

[0036] Based on the corrected aggregation weight matrix, the neighborhood node features are weighted and summed along the convergence path to generate local update features, while the self-attention mechanism is used to extract long-distance dependency features between nodes.

[0037] The local update features and long-distance dependency features are concatenated along the channel dimension, and then linearly transformed and dimension-restored through a fully connected layer. Finally, the transformation result is residually connected with the input multi-source heterogeneous feature tensor to output a high-order node feature tensor that integrates global structural information and local texture details.

[0038] Optionally, the FairMOT model includes an improved backbone feature extraction layer, a dual-task prediction head layer, and a cross-frame trajectory association layer:

[0039] The improved backbone feature extraction layer includes a spatiotemporal context enhancement sublayer, a relative position encoding injection sublayer, and a causal multi-head attention computation sublayer. It is used to receive image features corresponding to the defect candidate set, extract shared low-level features through convolution operations, and process them sequentially through the spatiotemporal context enhancement sublayer, the relative position encoding injection sublayer, and the causal multi-head attention computation sublayer to output a shared spatiotemporal feature map that conforms to temporal causal logic and incorporates geometric motion priors.

[0040] The dual-task prediction head layer is connected to the output of the improved backbone feature extraction layer, and includes a parallel detection branch and a re-identification branch.

[0041] The detection branch is used to predict the center point, size and center point offset of the defect target through heat map regression based on the shared spatiotemporal feature map, and generate defect detection results.

[0042] The re-identification branch is used to extract the high-dimensional feature vector corresponding to the center point of each defect target based on the shared spatiotemporal feature map, and generate a Re-ID feature set for distinguishing different defect identities;

[0043] The cross-frame trajectory association layer is used to calculate appearance similarity based on the Re-ID feature set, and to use Kalman filtering to predict motion state and calculate motion distance in combination with the position coordinates in the defect detection results; a joint cost matrix combining appearance similarity and motion distance is constructed, and data association is performed using the Hungarian algorithm to assign unique trajectory identifiers to mutually related defect targets, and trajectory smoothing and denoising processing is performed to output the spatiotemporal trajectory of the defect with a unique identifier.

[0044] Optionally, step S7 includes the following steps:

[0045] S71. Based on the spatiotemporal trajectory of defects, analyze the pixel coordinate changes and morphological features of defect candidate regions in the image sequence, calculate the physical size parameters and area ratio of defects, combine the defect category to perform weighted scoring, and output a defect quantitative feature set containing physical size and damage severity.

[0046] S72. Spatiotemporally align the image frame timestamps corresponding to the defect spatiotemporal trajectory, the UAV flight log pose data, and the preset blade geometric model. Calculate the transformation matrix from the image coordinate system to the global measurement coordinate system and output the spatial mapping relationship between the UAV's precise pose and the blade model under the unified coordinate system.

[0047] S73. Based on the precise pose of the UAV and the camera intrinsic parameters, a collinearity equation constraint is constructed. The two-dimensional coordinates of the defect image are back-projected onto the three-dimensional surface of the blade geometric model. By solving the intersection of the ray and the surface, the global three-dimensional spatial coordinates and normal angle of the defect are obtained.

[0048] S74. Encapsulate the defect quantification feature set with global three-dimensional spatial coordinates and normal angles according to a predetermined data structure to generate a multi-dimensional structured inspection report containing defect location, shape, attributes and risk rating.

[0049] The beneficial effects of this invention are:

[0050] (1) This invention adopts the FairMOT model, which significantly improves the feature representation ability and tracking stability of wind turbine blade defects in dynamic scenarios. Traditional multi-target tracking algorithms often suffer from blurred feature extraction and trajectory drift when dealing with high-speed rotating targets due to the drastic displacement of the target and dynamic changes in the background. To solve these problems, this invention innovatively introduces a spatiotemporal context enhancement sublayer and a causal multi-head attention computation sublayer, which can effectively integrate local motion details and long-distance temporal dependencies. At the same time, a causal masking mechanism is used to prevent future information leakage, ensuring the real-time performance and logical consistency of the inference process. Meanwhile, a relative position encoding injection sublayer is adopted, combined with sparse optical flow field encoding geometric motion priors, so that the model can accurately capture the relative motion trend of the defect target between consecutive frames, thereby significantly improving the feature robustness and trajectory correlation accuracy of the model in complex dynamic environments.

[0051] (2) This invention effectively solves the problem of accurate identification and structural perception reasoning of minute defects on the surface of wind turbine blades in complex texture backgrounds by adopting an improved TransGAT model. Traditional detection methods usually ignore the spatial adjacency relationship and texture prior association between regions on the blade surface, making it difficult to accurately distinguish between noise interference and real defects. The improved TransGAT model constructs a blade topology diagram, uses frequency domain enhanced graph attention units to extract robust node features that are insensitive to changes in illumination, and combines edge-constrained multi-head attention units to map the topological edge relationship into attention bias, thereby achieving explicit modulation of feature aggregation between nodes. This method breaks through the limitations of traditional convolutional neural networks in utilizing image structured information, and can adaptively aggregate global structural context and local texture details, greatly improving the accuracy and anti-interference ability of blade defect detection, and providing efficient technical support for the refined intelligent operation and maintenance of wind turbine blades. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is an overall flowchart of a dynamic detection method for wind turbine blade defects based on AI vision proposed in this invention.

[0054] Figure 2 This is a flowchart illustrating the working principle of the improved TransGAT model, which is based on AI vision for dynamic detection of defects in wind turbine blades using unmanned aerial vehicles (UAVs). Detailed Implementation

[0055] The invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0056] refer to Figure 1 and Figure 2 A dynamic detection method for defects in wind turbine blades using drones based on AI vision includes the following steps:

[0057] S1. Deploy visible light imaging sensors on the UAV to continuously image the wind turbine blades in a rotating state and collect the original dynamic image sequence.

[0058] S2. Based on the original dynamic image sequence, calculate the inter-frame motion vector using the optical flow method and apply smoothness constraints, perform spatiotemporal registration operation, and output the aligned image sequence;

[0059] S3. Input the aligned image sequence into the rotating target adaptive perspective correction network, construct the perspective transformation matrix based on the blade rotation phase estimation parameters, perform geometric distortion correction operation, and output a standardized blade surface image sequence.

[0060] S4. Construct a blade topology graph based on a standardized blade surface image sequence, map image superpixel regions as nodes, map spatial adjacency relationships and texture prior relationships as edges, and generate graph structure data.

[0061] S5. Input the graph structure data into the improved TransGAT model, use the frequency domain enhanced attention unit to extract robust node features, map the topological edge relationship to the attention bias through the edge constraint multi-head attention unit to perform explicit modulation, and construct the feature bidirectional propagation path through the bidirectional spatiotemporal awareness fusion unit. After fusing the global structural context, perform structural awareness reasoning and output a defect candidate set.

[0062] S6. Input the defect candidate set into the FairMOT model, extract and enhance the spatiotemporal features, output the detection results and Re-ID features in parallel through the dual-task prediction head layer, and output the spatiotemporal trajectory of the defect by combining appearance similarity and motion distance through the cross-frame trajectory association layer.

[0063] S7. Extract the defect quantification feature set based on the defect spatiotemporal trajectory, and combine it with the spatiotemporally aligned UAV precise pose calculation to obtain the global three-dimensional spatial coordinates and normal angle, and generate a structured inspection report.

[0064] In this embodiment, S2 specifically includes:

[0065] S21. Feature points are extracted from the original dynamic image sequence. The dense optical flow field of pixels between adjacent frames is calculated using a block matching algorithm to establish a pixel-level motion mapping relationship between frames and generate an initial motion vector field. The block matching algorithm includes taking the target pixel in the current frame image as the center point and dividing a rectangular region of a preset size as the current macroblock. In the next frame image, the initial search window is determined with the motion propagation result of the target pixel as the center. The search radius of the initial search window is dynamically adjusted according to the local texture entropy value of the current macroblock to generate an actual search area. Multiple candidate blocks are generated in the actual search area based on a preset search path. The grayscale difference between the current macroblock and all corresponding pixels in each candidate block is accumulated using the mean square error formula to calculate the matching cost set. The matching cost with the smallest value is selected from the matching cost set, and the corresponding candidate block is locked as the best matching block. The coordinate difference between the center coordinates of the best matching block and the center coordinates of the current macroblock is calculated, and the coordinate difference is recorded as the motion vector of the target pixel. All motion vectors constitute the initial motion vector field.

[0066] S22. Based on the preset continuity and smoothness constraints of the motion vector field, the RANSAC algorithm is used to remove abnormal motion data in the initial motion vector field to obtain an optimized motion vector field. The inter-frame global transformation matrix is ​​then calculated based on the optimized motion vector field. The RANSAC algorithm includes randomly sampling motion vector data from the initial motion vector field to obtain a minimum sample set of motion vector data, constructing a coordinate observation matrix and a target coordinate vector, solving a system of linear equations using the least squares method to obtain the specific parameter matrix of the geometric transformation model, and performing matrix multiplication on the pixel coordinates of the remaining motion vector data in the current frame with the specific parameter matrix of the geometric transformation model to calculate the mapping to the next frame. The predicted coordinates of the frame are calculated; the Euclidean distance between the predicted coordinates and the actual coordinates of the remaining motion vector data in the next frame is calculated, and the Euclidean distance is used as the projection error; motion vector data with projection errors less than a preset error threshold are marked as inliers, the number of inliers is counted, and the corresponding geometric transformation model parameters are recorded as the optimal model parameter set; the random sampling, parameter matrix solving, Euclidean distance calculation, and inlier counting process are repeated until the preset convergence condition is reached; abnormal motion data are removed based on the inlier subset corresponding to the optimal model parameter set to obtain the optimized motion vector field, and the matrix solving operation is re-executed using the coordinate data contained in the inlier subset to generate the inter-frame global transformation matrix;

[0067] S23. Using the inter-frame global transformation matrix, perform geometric inverse distortion and grayscale interpolation on each frame of the original dynamic image sequence to eliminate inter-frame jitter and displacement errors, and output a motion-compensated aligned image sequence. The geometric inverse distortion and grayscale interpolation include constructing a pixel coordinate grid of the reference frame image, performing matrix multiplication on the pixel coordinate grid using the inverse of the inter-frame global transformation matrix, mapping it to the current frame image plane to obtain the sampling point position with non-integer coordinates, dividing the sampling point position into an integer coordinate part and a fractional coordinate offset, determining the weight coefficient of neighboring pixels based on the fractional coordinate offset, performing a weighted summation operation on the grayscale values ​​of neighboring pixels at the sampling point position using the weight coefficients, calculating the interpolated pixel grayscale value at the non-integer coordinates, and filling the interpolated pixel grayscale value into the corresponding pixel position of the reference frame image to generate a pixel-level aligned motion-compensated image.

[0068] This step calculates the dense optical flow field using a block matching algorithm to generate an initial motion vector field and establishes a pixel-level inter-frame motion mapping relationship. It uses the RANSAC algorithm to remove outlier data and fits to generate an inter-frame global transformation matrix, improving the accuracy and robustness of the transformation parameters. Combining geometric inverse warping and grayscale interpolation, it outputs a motion-compensated aligned image sequence, eliminating inter-frame jitter and displacement errors.

[0069] In this embodiment, S3 specifically includes:

[0070] S31. Input the aligned image sequence into the feature pyramid backbone network to extract multi-scale leaf appearance features, fuse the temporal context information after motion compensation, and output a target feature map containing spatial geometric structure features; the feature pyramid backbone network includes inputting the aligned image sequence into the bottom convolutional neural network for convolution operation to extract high-resolution bottom geometric edge features; inputting the bottom geometric edge features into the intermediate convolutional network for downsampling and feature abstraction to extract mid-level semantic features containing leaf texture details; inputting the mid-level semantic features into the top convolutional network for deep feature extraction to obtain high-level global morphological features with a large receptive field; performing an upsampling operation on the high-level global morphological features to restore the feature map size to be consistent with the mid-level semantic features, and performing pixel-by-pixel addition and fusion with the mid-level semantic features to generate enhanced features that fuse spatial details; performing an upsampling operation on the enhanced features again to restore the feature map size to be consistent with the bottom geometric edge features, and integrating temporal context information through cascading operations to generate a target feature map containing rich spatial geometric structure features.

[0071] S32. Input the target feature map into the blade rotation phase estimation sub-network, predict the blade rotation angle and center offset through a convolutional regression layer, and construct a perspective transformation matrix based on the rotation phase estimation parameters. The blade rotation phase estimation sub-network includes inputting the target feature map into a spatial pooling layer for downsampling, removing spatial redundancy information, and extracting a global feature vector with translation invariance; inputting the global feature vector into a multilayer perceptron network for nonlinear feature transformation through the ReLU function, and outputting target regression parameters containing rotation angle values ​​and center offset values; calculating the corresponding sine and cosine values ​​using the rotation angle values, and constructing an affine transformation matrix by combining the center offset values; extending the affine transformation matrix to a homogeneous coordinate space to generate a perspective transformation matrix containing translation and rotation information; transforming the coordinates of the blade key points in the image using the perspective transformation matrix to calculate the predicted coordinate set after rotation; calculating the Euclidean distance loss based on the predicted coordinate set and the preset real label coordinates, and updating the network layer parameters in reverse using the gradient descent algorithm; when the loss function value decreases and meets the preset convergence threshold, outputting the perspective transformation matrix generated in the current iteration.

[0072] S33. Perform a geometric inverse distortion operation on the target feature map using a perspective transformation matrix, perform bilinear interpolation sampling and pixel filling to complete the perspective distortion correction of the blade surface image, and output a globally coordinate-standardized blade surface image sequence. The geometric inverse distortion operation includes constructing an inverse transformation matrix of the perspective transformation matrix, mapping each pixel coordinate point in the output blade image to the corresponding floating-point coordinate position in the input target feature map; determining the four nearest integer coordinate points of the floating-point coordinate position in the input target feature map, and extracting the pixel feature values ​​of the four nearest integer coordinate points; calculating the horizontal and vertical offsets of the floating-point coordinate position relative to the four nearest integer coordinate points, and using the complements of the horizontal and vertical offsets as the corresponding linear interpolation weight coefficients; performing weighted multiplication operations on the pixel feature values ​​of the four nearest integer coordinate points with the corresponding linear interpolation weight coefficients, and summing the product results to generate the corrected pixel value of the current pixel coordinate point; filling the corrected pixel value into the corresponding position in the output blade image, traversing all pixel coordinate points of the output blade image, and obtaining a globally coordinate-standardized blade surface image sequence.

[0073] This step establishes a precise mapping relationship between the blade image and the global standard coordinate system by constructing a perspective transformation matrix and performing a geometric inverse warp operation. Pixel resampling and filling are performed using bilinear interpolation, effectively solving the problem of non-integer coordinate values ​​caused by geometric transformation and ensuring the continuity and smoothness of the corrected image. A globally coordinate-standardized sequence of blade surface images is output, eliminating perspective distortion caused by changes in shooting angle and providing a standardized geometric reference for subsequent feature extraction and phase detection.

[0074] In this embodiment, S4 specifically includes:

[0075] S41. Perform region segmentation processing on the globally coordinate-normalized blade surface image sequence, generate a set of superpixel regions by combining preset texture gradient and color consistency constraints, and extract local texture statistical features for each superpixel region; the region segmentation processing includes converting the globally coordinate-normalized blade surface image sequence to the CIELab color space, calculating the texture gradient magnitude of each pixel in the image in the horizontal and vertical directions; dividing the image into a grid at preset intervals and selecting cluster centers as initial seed points, calculating the color Euclidean distance and spatial coordinate distance between each pixel and its neighboring seed points, and weighted summing the color distance and spatial distance to generate a comprehensive value. The overall metric distance is adjusted using texture gradient magnitude as a boundary constraint to suppress cross-edge cluster merging. Each pixel is assigned to the region of the seed point with the smallest overall metric distance to generate an initial superpixel region. A connectivity check is performed on the initial superpixel region, merging small regions with fewer than a preset threshold of pixels. The cluster center positions are iteratively updated until convergence, generating the final set of superpixel regions. The local binary pattern feature values ​​of all pixels within the superpixel region are statistically analyzed. The arithmetic mean and standard deviation of the local binary pattern feature values ​​are calculated, and the arithmetic mean and standard deviation are combined to generate local texture statistical features describing the texture distribution characteristics of the region.

[0076] S42. Calculate the geometric distance and connectivity between regions based on the spatial location distribution of the superpixel region set, construct spatial adjacency relationships, and construct texture prior relationships based on the similarity calculation results of local texture statistical features. The construction of spatial adjacency relationships and the calculation of texture prior relationships include traversing the superpixel region set, calculating the Euclidean distance between the center pixel coordinates of any two superpixel regions, and marking regions with an Euclidean distance less than a preset neighborhood radius as spatial neighbors. For regions marked as spatial neighbors, extract the common edge pixel set at the boundary of the two regions, count the proportion of the number of common edge pixels to the total number of region boundary pixels, and determine regions with a proportion greater than a preset connectivity threshold as geometrically connected regions. Calculate the cosine similarity of the local texture statistical features between geometrically connected regions, and use the cosine similarity value as a texture similarity measure. Compare the texture similarity measure with a preset texture similarity threshold. When the texture similarity measure is greater than or equal to the preset texture similarity threshold, determine that the two regions belong to homogeneous texture regions and establish texture association edges. Based on the geometric connectivity determination results and the texture association edge determination results, generate a region relationship topology diagram containing spatial adjacency attributes and texture similarity attributes.

[0077] S43. Map the set of superpixel regions to nodes, and map spatial adjacency relationships and texture prior relationships to edges connecting nodes, constructing a leaf topology graph containing node features and edge weights; the construction of the leaf topology graph includes assigning a unique node index to each superpixel region in the set of superpixel regions, storing the local texture statistical features, spatial coordinate positions, and region area attributes of the superpixel regions as node feature vectors; generating adjacency connecting edges of nodes based on spatial adjacency relationships, setting the initial weight of the adjacency connecting edges to a preset constant; generating texture connecting edges of nodes based on texture prior relationships, calculating the cosine similarity value between the local texture statistical features of the corresponding regions of two nodes, normalizing the cosine similarity value and using it as the weight value of the texture connecting edge; merging the adjacency connecting edges and texture connecting edges into the edge set of the node, if two nodes simultaneously have spatial adjacency relationships and texture prior relationships, then performing weighted fusion processing on the edge weights; traversing all nodes to generate a leaf topology graph containing a set of node features and a set of edge weights.

[0078] This step generates a set of superpixel regions using a region segmentation algorithm, clustering image pixels into semantically meaningful node units, effectively reducing computational redundancy in image data. It constructs a topological structure graph by combining spatial adjacency and texture similarity, establishing dual geometric and textural constraints between regions, thus enhancing the structured representation of features. The output is a blade topological structure graph containing node features and edge weights, providing standardized structured input for subsequent graph neural network processing and improving the ability to model complex surface structures of blades.

[0079] In this embodiment, the improved TransGAT model includes a frequency domain enhanced graph attention unit, an edge-constrained multi-head attention unit, and a bidirectional spatiotemporal awareness fusion unit:

[0080] The frequency domain enhanced graph attention unit is used to map the node feature tensor of the blade topology graph to the frequency domain, extract the frequency domain statistical feature tensor that is insensitive to illumination changes, and perform channel-level adaptive weighted fusion with the original time domain node feature tensor to obtain an enhanced node feature tensor with anti-interference capability. Specifically, it includes performing a one-dimensional discrete Fourier transform operation on the input node feature tensor to map the node features from the time domain to the frequency domain, obtaining a complex frequency domain feature tensor containing amplitude and phase information; performing a modulo operation on the complex frequency domain feature tensor to extract the frequency domain amplitude feature tensor representing the texture distribution structure as the frequency domain statistical feature tensor. A multilayer perceptron network model with two fully connected layers is constructed. The frequency domain amplitude feature tensor and the original time domain node feature tensor are concatenated along the channel dimension. The concatenated feature vector is input into the multilayer perceptron network model for ReLU nonlinear transformation, and the output frequency domain attention weight coefficients have the same dimension as the number of input feature channels. The frequency domain attention weight coefficients and the frequency domain amplitude feature tensor are multiplied one-way to generate a weighted frequency domain feature tensor. The weighted frequency domain feature tensor and the original time domain node feature tensor are added element-wise and then normalized to generate an enhanced node feature tensor with anti-interference capability.

[0081] The edge-constrained multi-head attention unit is used to receive the enhanced node feature tensor and construct a heterogeneous attention mechanism composed of a texture-aware head, a geometry-aware head, and a spatial-aware head. Using the edge weight matrix of the leaf topology graph, the spatial adjacency relationship and the texture prior relationship are encoded as bias terms and superimposed on the attention score calculation process. The feature aggregation between nodes is explicitly modulated so that the aggregation weight responds to the semantic features and topological constraints of the nodes at the same time, and outputs a multi-source heterogeneous feature tensor containing edge constraints.

[0082] The heterogeneous attention mechanism includes inputting enhanced node feature tensors into three independent attention branches—texture-aware head, geometry-aware head, and spatial-aware head—in parallel. A linear transformation layer within the texture-aware head generates query, key, and value tensors, and the transpose matrix product of the query and key tensors is calculated to generate an initial attention score matrix based on semantic texture. The geometry-aware head and spatial-aware head extract coordinate geometric information and relative position encoding from node features, respectively. The geometry-aware head extracts coordinate geometric information from node features, calculates Euclidean distance between nodes, and generates a geometric correlation score matrix representing spatial position distribution. The spatial-aware head extracts relative position encoding from node features, encodes relative coordinate deviations between nodes using sine and cosine functions, and calculates a spatial correlation score matrix between nodes. The edge weights of the blade topology graph are extracted. The algorithm analyzes the node connectivity and weight distribution data contained in the edge weight matrix. It extracts the connection weights representing spatial adjacency and the similarity weights representing texture priors from the edge weight matrix, and performs a linear transformation using a learnable linear transformation layer. The transformed values ​​are mapped to the same numerical range as the attention score matrix, generating topological bias terms corresponding to spatial and texture constraints. These topological bias terms are then superimposed onto the corresponding positions of the initial attention score matrix, geometric correlation score matrix, and spatial correlation score matrix, with scores for disconnected node positions set to negative infinity. The superimposed scores are normalized using the Softmax activation function, and the aggregation weight coefficients between nodes are calculated. Based on these aggregation weight coefficients, the value tensor and the features of each branch are weighted and summed, outputting a multi-source heterogeneous feature tensor containing edge constraints.

[0083] The bidirectional spatiotemporal perception fusion unit is used to construct a propagation path that diffuses from the central node to neighboring nodes and an aggregation path that converges from neighboring nodes to the central node, using a multi-source heterogeneous feature tensor as input. Through a bidirectional message passing mechanism, the output of the spatial perception head is used to dynamically mask the aggregation range of the texture perception head and integrate long-distance dependencies to output a high-order node feature tensor that fuses global structural information and local texture details.

[0084] The high-order node feature tensor is input into the fully connected decoder for dimensionality reduction, and the defect classification task and coordinate regression task are executed in parallel. The probability distribution of defect categories is output through the Softmax activation function, and the position coordinates of the defect candidate region are output through the identity activation function. Finally, a defect candidate set containing position coordinates and category attributes is generated.

[0085] The fully connected decoder employs a multilayer perceptron structure, comprising several cascaded fully connected layers and ReLU nonlinear activation layers. This structure maps high-dimensional node features to a low-dimensional, pre-defined shared semantic space. In the classification branch, a linear projection layer mapped to the total number of categories is constructed to transform the shared semantic features. A Softmax normalization function is used to output the posterior probability distribution for each category, representing the confidence level of the defect category to which the node belongs. In the regression branch, a linear projection layer mapped to the coordinate parameter dimension is constructed to directly perform linear transformations on the shared semantic features. An identity activation function is used to maintain numerical continuity, outputting the center point coordinates and bounding box width and height parameters of the defect candidate region. The posterior probabilities output from the classification branch and the coordinate parameters output from the regression branch are aligned and cascaded along the node dimension. A non-maximum suppression algorithm is used to remove redundant detection boxes with excessive overlap, and a confidence threshold is applied for filtering to generate a defect candidate set containing precise location coordinates and category attributes.

[0086] This step extracts frequency-domain statistical features that are insensitive to changes in illumination through a frequency-domain enhanced graph attention unit, and adaptively weights and fuses the frequency-domain amplitude features with the original temporal features, effectively suppressing uneven illumination and noise interference. An edge-constrained multi-head attention unit is used to construct a texture, geometry, and spatial heterogeneous perception mechanism, encoding the edge weights of the topological graph as attention bias terms, establishing a strong correlation between node semantic features and graph topological constraints, and enhancing the model's ability to express features of complex textures and geometric structures. A bidirectional spatiotemporal perception fusion unit is used to construct a bidirectional propagation path of diffusion and convergence, and a spatial mask is used to dynamically correct the texture perception range, integrating local texture details and long-distance global dependencies, improving the structuring degree and semantic richness of high-order features. The fully connected decoder, through multi-task joint optimization and post-processing to remove redundant detections, outputs a high-precision defect candidate set, achieving accurate localization and classification of minor defects and irregular boundaries on the blade, significantly improving the robustness and accuracy of the defect detection system.

[0087] In this embodiment, the bidirectional message passing mechanism includes:

[0088] Define the central node as the diffusion and propagation direction of the source node, and define the neighboring nodes as the convergence and aggregation direction of the source node;

[0089] Extract the spatial correlation score matrix output by the spatial sensing head, map it to the 0 to 1 interval using the Sigmoid function, and generate a binary spatial mask matrix based on the set spatial distance threshold.

[0090] The binarized spatial mask matrix is ​​multiplied element-wise with the initial aggregation weight matrix generated by the texture-aware head. The aggregation weights of the corresponding positions that are outside the spatial perception range are reset to 0 using the elements with values ​​of 0 in the spatial mask matrix.

[0091] Based on the corrected aggregation weight matrix, the neighborhood node features are weighted and summed along the convergence path to generate local update features, while the self-attention mechanism is used to extract long-distance dependency features between nodes.

[0092] The local update features and long-distance dependency features are concatenated along the channel dimension, and then linearly transformed and dimension-restored through a fully connected layer. Finally, the transformation result is residually connected with the input multi-source heterogeneous feature tensor to output a high-order node feature tensor that integrates global structural information and local texture details.

[0093] This step establishes a bidirectional propagation path between the central node and neighboring nodes, constructing a mechanism for the bidirectional flow of feature information in both local and global domains. This effectively balances the perception of local details with the understanding of global structure. A spatial mask matrix is ​​used to correct the texture aggregation weights, resetting invalid weights beyond the spatial perception range to zero, eliminating noise interference from irrelevant nodes, and enhancing the local geometric consistency of feature aggregation. By combining the splicing and fusion of locally updated features and long-range dependent features, a dual association constraint of short-range context and long-range semantics is established, enriching the semantic expression of node features. A high-order node feature tensor that integrates global structural information and local texture details is output, providing a feature representation with both local accuracy and global completeness for subsequent graph neural network processing, thus improving the recognition accuracy of complex defect regions in the blade.

[0094] In this embodiment, the FairMOT model includes an improved backbone feature extraction layer, a dual-task prediction head layer, and a cross-frame trajectory association layer:

[0095] The improved backbone feature extraction layer includes a spatiotemporal context enhancement sublayer, a relative position encoding injection sublayer, and a causal multi-head attention computation sublayer. It is used to receive image features corresponding to the defect candidate set, extract shared low-level features through convolution operations, and process them sequentially through the spatiotemporal context enhancement sublayer, the relative position encoding injection sublayer, and the causal multi-head attention computation sublayer to output a shared spatiotemporal feature map that conforms to temporal causal logic and incorporates geometric motion priors.

[0096] The spatiotemporal context enhancement sublayer is used to construct a parallel three-dimensional convolutional branch containing multiple sets of different dilation rates. The branch performs convolution operations on the input features to obtain the corresponding spatiotemporal feature maps. The spatiotemporal feature maps output by each branch are concatenated along the channel dimension and channel fusion and dimensionality reduction are performed through a 1x1x1 convolutional kernel to generate enhanced features of the fused context. The enhanced features are then added to the input features through a residual connection, and the enhanced low-level features are output.

[0097] The relative position encoding injection sublayer constructs query, key, and value tensors based on the enhanced low-level features. According to the spatiotemporal coordinate index of the query and key tensors in the feature map, it calculates the relative distance difference between the time dimension and the spatial dimension. Then, using the relative position encoding vector corresponding to the index of the preset learnable position embedding parameter table, it maps the encoding vector to a position bias matrix and linearly superimposes it onto the dot product result of the query and key tensors to generate the original attention score matrix containing geometric motion priors.

[0098] The causal multi-head attention computation sublayer constructs a causal mask matrix with the same size as the original attention score matrix. By traversing the spatiotemporal position index, the position values ​​representing future moments are set to negative infinity, and the position values ​​representing historical and current moments are set to 0. The causal mask matrix and the original attention score matrix are added element by element. The modified score matrix is ​​normalized using the Softmax activation function to obtain attention weight coefficients that conform to temporal causal logic. The value tensor is weighted and aggregated based on the attention weight coefficients to output a shared spatiotemporal feature map that integrates geometric motion priors and strictly satisfies temporal causal relationships.

[0099] The dual-task prediction head layer is connected to the output of the improved backbone feature extraction layer, and includes a parallel detection branch and a re-identification branch.

[0100] The detection branch is used to predict the center point, size, and center point offset of the defect target based on the shared spatiotemporal feature map through heatmap regression, generating defect detection results. Specifically, it includes performing deconvolution upsampling on the shared spatiotemporal feature map to restore it to the original input image resolution, obtaining a high-resolution prediction feature map. In the classification prediction head, a 1x1 convolution kernel is used to map the high-resolution prediction feature map to a keypoint heatmap of the total number of defect categories. A Gaussian kernel function is used to encode the coordinates of the real defect center point to generate training targets. The FocalLoss function is used to supervise the fitting of the predicted heatmap and the real heatmap, and the position with the largest response value in the heatmap is extracted as the predicted center point coordinates of the defect target. In the regression prediction head, two independent 1x1 convolution branches predict the width and height parameters and the sub-pixel offset of the center point of the defect target, respectively. The SmoothL1 loss function is used to constrain the deviation between the predicted value and the real label. The positioning accuracy is corrected based on the predicted center point coordinates and the corresponding sub-pixel offset. The bounding box of the defect target is calculated by combining the predicted width and height parameters. Finally, the bounding box with a classification confidence score higher than a preset threshold is used as the final defect detection result.

[0101] The re-identification branch is used to extract high-dimensional feature vectors corresponding to the center point of each defect target based on the shared spatiotemporal feature map, generating a Re-ID feature set for distinguishing different defect identities. Specifically, it includes performing deconvolution upsampling on the shared spatiotemporal feature map to restore the feature map to the same resolution as the original input image; using ROIAlign region of interest alignment operation, based on the defect target bounding box coordinates output by the detection branch, cropping and adaptively pooling the region feature map from the upsampled feature map to a fixed size; constructing a feature embedding network containing several convolutional layers, batch normalization layers, and ReLU activation layers to perform deep semantic extraction on the region feature map, and using a global average pooling layer to compress the three-dimensional feature tensor into a one-dimensional feature vector; mapping the one-dimensional feature vector to a feature space of a preset dimension through a fully connected layer, and performing L2 normalization to constrain the magnitude of the feature vector, generating a Re-ID feature vector set for measuring similarity.

[0102] The cross-frame trajectory association layer is used to calculate appearance similarity based on the Re-ID feature set, and to predict motion state using Kalman filtering in conjunction with the position coordinates in the defect detection results to calculate motion distance; it constructs a joint cost matrix combining appearance similarity and motion distance, performs data association using the Hungarian algorithm, assigns unique trajectory identifiers to mutually associated defect targets, and performs trajectory smoothing and denoising processing to output defect spatiotemporal trajectories with unique identifiers; specifically, it includes calculating the cosine similarity between the defect target to be associated in the current frame and historical trajectory candidates based on the Re-ID feature set, as the appearance similarity cost; simultaneously, it initializes the Kalman filter, uses the state vector of the historical trajectory to predict the center position of the bounding box of the defect target in the current frame, and calculates the predicted... The Mahalanobis distance between the location and the detection location is used as the motion distance cost. The appearance similarity cost and the motion distance cost are linearly weighted according to the preset weight coefficients to construct a joint cost matrix. A matching threshold and occlusion handling strategy are set, and the Hungarian algorithm is used to find the globally optimal matching pair with the minimum total cost in the joint cost matrix. The state vector and covariance matrix of the Kalman filter are updated according to the matching results, and the identity information of the defect target is updated using the newly assigned trajectory identifier. For unmatched trajectories in multiple consecutive frames, the marking and deletion logic is executed. New trajectory identifiers are initialized for newly appearing defect targets. The generated trajectory sequence is subjected to Kalman smoothing to correct position jitter. Finally, the defect motion trajectory containing continuous spatiotemporal position and unique identity is output.

[0103] This step aggregates multi-scale spatiotemporal context information through parallel 3D convolutional branches with hybrid dilation rates, enhancing the network's ability to capture dynamic features of defects and effectively alleviating the limitations of a single receptive field in modeling temporal changes. By combining relative position encoding and causal attention mechanisms, geometric motion priors and strict temporal causal logic are injected into the feature aggregation process, explicitly constructing dynamic correlation constraints between spatiotemporal locations and improving the consistency and anti-interference capability of feature representation. A shared spatiotemporal feature map, incorporating geometric priors and conforming to causality, is output, providing a unified, high-quality feature representation for the parallel dual-task prediction head and enhancing the ability to model small defects in complex backgrounds. Based on the shared feature map, high-precision defect localization is achieved through heatmap regression and sub-pixel correction. Simultaneously, ROIAlign and deep metric learning are used to extract Re-ID features with discriminative power, establishing joint correlation constraints between appearance features and motion state. Combining the Hungarian algorithm and Kalman filtering for global optimal matching and trajectory smoothing, accurate tracking of defect identities across frames and stable output of motion trajectories are achieved, effectively solving the problem of target loss under occlusion and rapid movement.

[0104] In this embodiment, S7 specifically includes:

[0105] S71. Based on the spatiotemporal trajectory of the defect, extract the bounding box coordinate sequence of the defect target in each frame, map the pixel coordinates to the physical world coordinate system using calibration parameters, and calculate the movement velocity and displacement change of the defect using timestamps; construct a multi-dimensional scoring model including size, velocity, and area proportion based on the physical attributes and hazard weight of the defect category, and calculate the damage severity index of the defect; specifically, this includes locating the defect candidate region based on the spatiotemporal trajectory of the defect, calculating the pixel area of ​​the defect on the image plane using the center point coordinates and width and height parameters after Kalman filtering smoothing; and calculating the pixel-to-physical size conversion ratio based on the geometric calibration intrinsic parameters of the imaging system. The process involves converting the pixel width, height, and area of ​​the defect into actual physical size parameters; calculating the ratio of the physical area of ​​the defect to the total imaging area of ​​the image sensor to obtain the area ratio index of the defect; querying a preset weight coefficient table based on the defect category label to obtain the size weight, motion weight, and area weight of the corresponding category; linearly fusing the physical size parameters, motion speed change, and area ratio index using a weighted summation formula to generate a comprehensive damage score; and concatenating the physical size parameters, area ratio, motion change, and comprehensive damage score to output a defect quantitative feature set containing physical size and damage severity.

[0106] S72. Based on image frame timestamps and UAV flight log pose data, a time interpolation algorithm is used to map the UAV pose information to the acquisition time of each image frame, establishing precise synchronization in the time dimension. Combining a pre-defined blade geometry model and camera imaging intrinsic parameters, a coordinate transformation chain is constructed, including the UAV body coordinate system, camera coordinate system, and global measurement coordinate system. A rigid body transformation matrix is ​​used to solve the transformation relationship from image coordinates to the global measurement coordinate system. Specifically, this includes parsing the UAV flight log to obtain GPS coordinates and attitude angle data from the inertial navigation unit, matching image frames with their corresponding flight states using timestamp indexes, constructing a rotation matrix and translation vector from the camera coordinate system to the body coordinate system based on camera calibration parameters, and calculating the projection matrix from image pixels to global measurement coordinate points using the transformation matrix from the body coordinate system to the navigation coordinate system. Using the 3D spatial point cloud data of the blade geometry model as constraints, the parameters of the transformation matrix are optimized using the least squares method to eliminate time synchronization errors and sensor measurement drift. The output includes precise UAV pose data with a unified time and spatial reference, as well as a precise spatial mapping relationship from image pixel coordinates to the 3D coordinates of the blade model.

[0107] S73. Based on the UAV's precise pose data and camera intrinsic parameter matrix, construct collinearity equation constraints describing the collinearity of the camera's optical center, image pixels, and spatial object points. Specifically, this includes using the UAV's pose data to construct a rotation and translation transformation matrix from the global coordinate system to the camera coordinate system, and combining the camera intrinsic parameters to convert the image pixel coordinates into normalized imaging ray vectors in the camera coordinate system. Starting from the camera's optical center, emit a probe ray into space along the direction of the normalized imaging ray, and calculate the nearest Euclidean distance point between the ray and the three-dimensional surface of the blade geometric model as the intersection point. Extract the three-dimensional coordinate values ​​of the intersection point in the global coordinate system as the global three-dimensional spatial coordinates of the defect, and calculate the tangent plane and normal vector based on the surface geometric properties of the blade geometric model at the intersection point, and solve the normal angle of the defect surface normal relative to the global coordinate system. Finally, output the three-dimensional positioning result of the defect, which includes the three-dimensional spatial position coordinates and the surface normal angle.

[0108] S74. Encapsulate the defect quantification feature set with global three-dimensional spatial coordinates and normal angles according to a predetermined data structure to generate a multi-dimensional structured inspection report containing defect location, shape, attributes and risk rating.

[0109] This step maps image pixel coordinates to the physical world coordinate system based on the spatiotemporal trajectory of the defect using geometric calibration intrinsic parameters, and accurately calculates the movement velocity and displacement change of the defect by combining timestamps. A multi-dimensional scoring model is constructed based on the physical attributes and hazard weights of the defect category, generating a quantitative feature set including physical size and damage severity. Spatiotemporal alignment of UAV pose data and blade geometry model is achieved through time interpolation algorithms and a multi-level coordinate transformation chain, eliminating sensor measurement drift and synchronization errors, and establishing a precise mapping relationship from image coordinates to global measurement coordinates. Collinearity equation constraints are used to back-project the two-dimensional coordinates of the image onto the three-dimensional surface of the blade, obtaining the global three-dimensional spatial coordinates and normal angle of the defect, achieving millimeter-level precise positioning of minute defects. The quantitative features and three-dimensional positioning information are encapsulated according to a predetermined structure to generate a multi-dimensional structured inspection report containing location, shape, attributes, and risk rating, providing comprehensive and quantitative data support for wind turbine blade operation and maintenance decisions.

[0110] Example 1:

[0111] To verify the feasibility of this invention in automated wind turbine blade inspection, the method was applied to an intelligent unmanned aerial vehicle (UAV) blade inspection system at a wind farm (hereinafter referred to as "Site A"). In traditional wind turbine blade defect detection systems, defect identification is typically performed using manual inspection or traditional image processing methods. These methods are not only labor-intensive and inefficient, but also struggle to accurately locate minute defects in complex backgrounds and under varying lighting conditions. Furthermore, they cannot obtain precise three-dimensional spatial information about the defects, failing to meet the needs of refined operation and maintenance. To address these issues, Site A decided to adopt the spatiotemporal tracking and quantitative detection method for wind turbine blade defects based on the FairMOT model proposed in this invention.

[0112] During implementation, Station A first used a drone equipped with a high-resolution camera to collect image sequences of the wind turbine blades in rotation. After preprocessing operations such as image denoising and geometric distortion correction, a high-quality blade image dataset was formed. At the same time, the technicians at Station A combined the CAD model of the blades and the drone flight logs to accurately synchronize and calibrate the pose information of the acquired images, providing benchmark data for the three-dimensional localization of defects.

[0113] Station A effectively integrates multi-scale spatiotemporal context information by improving the spatiotemporal context enhancement sublayer and causal multi-head attention computation sublayer in the backbone feature extraction layer. It also injects geometric motion priors and strict temporal causal logic into the feature aggregation process, generating high-quality shared spatiotemporal feature maps, laying a solid foundation for subsequent tasks. Next, utilizing the detection branch in the dual-task prediction head layer, based on heatmap regression and sub-pixel correction techniques, it achieves high-precision localization and accurate size estimation of the center point of small defect targets, generating high-confidence defect detection results. Simultaneously, using the re-identification branch through ROIAlign and deep metric learning, it extracts Re-ID feature vectors with high identity discriminative power, effectively distinguishing visually similar defect targets.

[0114] In the core cross-frame association stage, this invention constructs a joint cost matrix combining appearance similarity and movement distance through a cross-frame trajectory association layer. It utilizes the Hungarian algorithm for global optimal matching and combines it with Kalman filtering for trajectory smoothing, achieving accurate tracking of the defect's cross-frame identity and outputting a continuous and stable spatiotemporal trajectory of the defect. Subsequently, based on the spatiotemporal trajectory and the precise pose of the UAV, this invention further calculates the defect's physical size, area ratio, and damage severity index. By constraining the collinearity equations, the two-dimensional coordinates of the image are back-projected onto the three-dimensional surface of the blade, obtaining the defect's global three-dimensional spatial coordinates and normal angle, generating a multi-dimensional structured inspection report containing location, shape, attributes, and risk rating.

[0115] During implementation, the technical team at site A discovered that, compared to traditional inspection methods and conventional tracking algorithms, the method of this invention significantly improves the accuracy and 3D localization capability of wind turbine blade defect detection. Traditional methods cannot effectively handle motion blur and target occlusion caused by blade rotation, and lack 3D quantization techniques. In contrast, the method of this invention effectively achieves stable tracking and accurate quantization of defects through causal attention mechanisms, joint correlation between appearance and motion, and 3D back-projection technology.

[0116] To further verify the actual performance of the method of the present invention, site A conducted a detailed comparative test between the method of the present invention and the traditional method. The specific performance data is shown in Table 1:

[0117] Table 1. Performance Comparison of Defect Detection and Quantification Methods for Wind Turbine Blades at Station A

[0118] index Traditional methods Method of the present invention Increase Defect location accuracy rate (%) 82.5 96.2 +13.7% Track ID switching count (times / minute) 15.4 0.8 -94.8% Physical dimension measurement error (mm) 12.5 2.1 -83.2% Root mean square error of 3D positioning (mm) 45.2 8.6 -81.0% Processing time for a single frame of image (ms) 180 35 -80.6% Defect missed detection rate (%) 18.3 2.5 -86.3% Number of false positive defects (per flight) 25.0 3.2 -87.2% Total time for inspection data processing (hours) 4.5 1.2 -73.3% Defect risk rating compliance rate (%) 78.0 93.5 +15.5%

[0119] As shown in Table 1, the performance of the wind turbine blade defect detection system was comprehensively improved after applying the method of this invention. The accuracy of defect location increased from 82.5% with traditional methods to 96.2%, and the number of trajectory ID switching times decreased from 15.4 times per minute to 0.8 times per minute, significantly improving tracking stability and effectively solving the problem of identity confusion caused by occlusion and intersection. The physical size measurement error decreased from 12.5 mm to 2.1 mm, and the root mean square error of three-dimensional positioning decreased from 45.2 mm to 8.6 mm, achieving high-precision quantification of defect morphology and providing accurate data support for the formulation of maintenance plans. The processing time for a single frame image decreased from 180 ms to 35 ms, greatly improving real-time processing capabilities. The defect false negative rate decreased from 18.3% to 2.5%, and the number of false positive defects decreased from 25.0 per flight to 3.2, significantly reducing false alarms and false negatives. The total time for processing inspection data was reduced from 4.5 hours to 1.2 hours, the defect risk rating compliance rate increased from 78.0% to 93.5%, and the efficiency of operation and maintenance decision-making was improved by 60.0%, effectively reducing wind turbine downtime and maintenance costs.

[0120] Using the method of this invention, wind farm A successfully achieved rapid and stable tracking and high-precision three-dimensional quantitative detection of wind turbine blade defects, effectively overcoming the identification difficulties under complex dynamic backgrounds, ensuring the accuracy and completeness of inspection data, significantly improving the intelligence and automation level of wind farm operation and maintenance, significantly reducing the workload of manual analysis, enhancing the timeliness and reliability of blade safety monitoring, and providing strong technical support for the intelligent operation and maintenance of wind power equipment.

[0121] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A dynamic detection method for defects in wind turbine blades using unmanned aerial vehicles (UAVs) based on AI vision, characterized in that, Includes the following steps: S1. Deploy visible light imaging sensors on the UAV to continuously image the wind turbine blades in a rotating state and collect the original dynamic image sequence. S2. Based on the original dynamic image sequence, calculate the inter-frame motion vector using the optical flow method and apply smoothness constraints, perform spatiotemporal registration operation, and output the aligned image sequence; S3. Input the aligned image sequence into the rotating target adaptive perspective correction network, construct the perspective transformation matrix based on the blade rotation phase estimation parameters, perform geometric distortion correction operation, and output a standardized blade surface image sequence. S4. Construct a blade topology graph based on a standardized blade surface image sequence, map image superpixel regions as nodes, map spatial adjacency relationships and texture prior relationships as edges, and generate graph structure data. S5. Input the graph structure data into the improved TransGAT model, use the frequency domain enhanced attention unit to extract robust node features, map the topological edge relationship to the attention bias through the edge constraint multi-head attention unit to perform explicit modulation, and construct the feature bidirectional propagation path through the bidirectional spatiotemporal awareness fusion unit. After fusing the global structural context, perform structural awareness reasoning and output a defect candidate set. S6. Input the defect candidate set into the FairMOT model, extract and enhance the spatiotemporal features, output the detection results and Re-ID features in parallel through the dual-task prediction head layer, and output the spatiotemporal trajectory of the defect by combining appearance similarity and motion distance through the cross-frame trajectory association layer. S7. Extract the defect quantification feature set based on the defect spatiotemporal trajectory, and combine it with the spatiotemporally aligned UAV precise pose calculation to obtain the global three-dimensional spatial coordinates and normal angle, and generate a structured inspection report.

2. The method for dynamic detection of defects in wind turbine blades using an unmanned aerial vehicle (UAV) based on AI vision, as described in claim 1, is characterized in that... S2 specifically includes: S21. Extract feature points from the original dynamic image sequence, calculate the dense optical flow field of adjacent inter-frame pixels using a block matching algorithm, establish inter-frame pixel-level motion mapping relationship, and generate an initial motion vector field. S22. Based on the preset continuity and smoothness constraints of the motion vector field, the RANSAC algorithm is used to remove abnormal motion data in the initial motion vector field to obtain an optimized motion vector field, and the inter-frame global transformation matrix is ​​calculated based on the optimized motion vector field. S23. Use the inter-frame global transformation matrix to perform geometric reverse distortion and grayscale interpolation on each frame of the original dynamic image sequence to eliminate inter-frame jitter and displacement errors, and output the motion-compensated aligned image sequence.

3. The method for dynamic detection of defects in wind turbine blades based on AI vision for unmanned aerial vehicles according to claim 1, characterized in that, S3 specifically includes: S31. Input the aligned image sequence into the feature pyramid backbone network, extract multi-scale leaf appearance features, fuse the temporal context information after motion compensation, and output a target feature map containing spatial geometric structure features. S32. Input the target feature map into the blade rotation phase estimation subnetwork, predict the blade rotation angle and center offset through the convolutional regression layer, and construct the perspective transformation matrix based on the rotation phase estimation parameters. S33. Perform geometric reverse distortion operation on the target feature map using the perspective transformation matrix, perform bilinear interpolation sampling and pixel filling, complete the perspective distortion correction of the blade surface image, and output a globally coordinate standardized blade surface image sequence.

4. The method for dynamic detection of defects in wind turbine blades using unmanned aerial vehicles based on AI vision, as described in claim 1, is characterized in that... S4 specifically includes: S41. Perform region segmentation processing on the globally coordinate-normalized blade surface image sequence, generate a set of superpixel regions by combining the preset texture gradient and color consistency constraints, and extract local texture statistical features for each superpixel region. S42. Calculate the geometric distance and connectivity between regions based on the spatial location distribution of superpixel region sets, construct spatial adjacency relationships, and construct texture prior relationships based on the similarity calculation results of local texture statistical features. S43. Map the set of superpixel regions to nodes, and map spatial adjacency and texture prior relationships to edges connecting nodes, constructing a leaf topology graph containing node features and edge weights.

5. The method for dynamic detection of defects in wind turbine blades using an unmanned aerial vehicle (UAV) based on AI vision, as described in claim 1, is characterized in that... The improved TransGAT model includes a frequency domain enhanced graph attention unit, an edge-constrained multi-head attention unit, and a bidirectional spatiotemporal awareness fusion unit. The frequency domain enhanced graph attention unit is used to map the node feature tensor of the blade topology graph to the frequency domain, extract the frequency domain statistical feature tensor that is insensitive to changes in illumination, and perform channel-level adaptive weighted fusion with the original time domain node feature tensor to obtain an enhanced node feature tensor that includes anti-interference capability. The edge-constrained multi-head attention unit is used to receive the enhanced node feature tensor and construct a heterogeneous attention mechanism composed of a texture-aware head, a geometry-aware head, and a spatial-aware head. Using the edge weight matrix of the leaf topology graph, the spatial adjacency relationship and the texture prior relationship are encoded as bias terms and superimposed on the attention score calculation process. The feature aggregation between nodes is explicitly modulated so that the aggregation weight responds to the semantic features and topological constraints of the nodes at the same time, and outputs a multi-source heterogeneous feature tensor containing edge constraints. The bidirectional spatiotemporal perception fusion unit is used to construct a propagation path that diffuses from the central node to neighboring nodes and an aggregation path that converges from neighboring nodes to the central node, using multi-source heterogeneous feature tensors as input. Through a two-way message passing mechanism, the output of the spatial perception head is used to dynamically mask the aggregation range of the texture perception head, and long-distance dependencies are integrated to output a high-order node feature tensor that fuses global structural information and local texture details. The high-order node feature tensor is input into the fully connected decoder for dimensionality reduction, and the defect classification task and coordinate regression task are executed in parallel. The probability distribution of defect categories is output through the Softmax activation function, and the position coordinates of the defect candidate region are output through the identity activation function. Finally, a defect candidate set containing position coordinates and category attributes is generated.

6. The method for dynamic detection of defects in wind turbine blades using an unmanned aerial vehicle (UAV) based on AI vision, as described in claim 5, is characterized in that... The bidirectional message passing mechanism includes: Define the central node as the diffusion and propagation direction of the source node, and define the neighboring nodes as the convergence and aggregation direction of the source node; Extract the spatial correlation score matrix output by the spatial sensing head, map it to the 0 to 1 interval using the Sigmoid function, and generate a binary spatial mask matrix based on the set spatial distance threshold. The binarized spatial mask matrix is ​​multiplied element-wise with the initial aggregation weight matrix generated by the texture-aware head. The aggregation weights of the corresponding positions that are outside the spatial perception range are reset to 0 using the elements with values ​​of 0 in the spatial mask matrix. Based on the corrected aggregation weight matrix, the neighborhood node features are weighted and summed along the convergence path to generate local update features, while the self-attention mechanism is used to extract long-distance dependency features between nodes. The local update features and long-distance dependency features are concatenated along the channel dimension, and then linearly transformed and dimension-restored through a fully connected layer. Finally, the transformation result is residually connected with the input multi-source heterogeneous feature tensor to output a high-order node feature tensor that integrates global structural information and local texture details.

7. The method for dynamic detection of defects in wind turbine blades based on AI vision according to claim 1, characterized in that, The FairMOT model includes an improved backbone feature extraction layer, a dual-task prediction head layer, and a cross-frame trajectory association layer: The improved backbone feature extraction layer includes a spatiotemporal context enhancement sublayer, a relative position encoding injection sublayer, and a causal multi-head attention computation sublayer. It is used to receive image features corresponding to the defect candidate set, extract shared low-level features through convolution operations, and process them sequentially through the spatiotemporal context enhancement sublayer, the relative position encoding injection sublayer, and the causal multi-head attention computation sublayer to output a shared spatiotemporal feature map that conforms to temporal causal logic and incorporates geometric motion priors. The dual-task prediction head layer is connected to the output of the improved backbone feature extraction layer, and includes a parallel detection branch and a re-identification branch. The detection branch is used to predict the center point, size and center point offset of the defect target through heat map regression based on the shared spatiotemporal feature map, and generate defect detection results. The re-identification branch is used to extract the high-dimensional feature vector corresponding to the center point of each defect target based on the shared spatiotemporal feature map, and generate a Re-ID feature set for distinguishing different defect identities; The cross-frame trajectory association layer is used to calculate appearance similarity based on the Re-ID feature set, and to use Kalman filtering to predict motion state and calculate motion distance in combination with the position coordinates in the defect detection results; a joint cost matrix combining appearance similarity and motion distance is constructed, and data association is performed using the Hungarian algorithm to assign unique trajectory identifiers to mutually related defect targets, and trajectory smoothing and denoising processing is performed to output the spatiotemporal trajectory of the defect with a unique identifier.

8. The method for dynamic detection of defects in wind turbine blades using an unmanned aerial vehicle (UAV) based on AI vision, as described in claim 1, is characterized in that... S7 includes the following steps: S71. Based on the spatiotemporal trajectory of defects, analyze the pixel coordinate changes and morphological features of defect candidate regions in the image sequence, calculate the physical size parameters and area ratio of defects, combine the defect category to perform weighted scoring, and output a defect quantitative feature set containing physical size and damage severity. S72. Spatiotemporally align the image frame timestamps corresponding to the defect spatiotemporal trajectory, the UAV flight log pose data, and the preset blade geometric model. Calculate the transformation matrix from the image coordinate system to the global measurement coordinate system and output the spatial mapping relationship between the UAV's precise pose and the blade model under the unified coordinate system. S73. Based on the precise pose of the UAV and the camera intrinsic parameters, a collinearity equation constraint is constructed. The two-dimensional coordinates of the defect image are back-projected onto the three-dimensional surface of the blade geometric model. By solving the intersection of the ray and the surface, the global three-dimensional spatial coordinates and normal angle of the defect are obtained. S74. Encapsulate the defect quantification feature set with global three-dimensional spatial coordinates and normal angles according to a predetermined data structure to generate a multi-dimensional structured inspection report containing defect location, shape, attributes and risk rating.

Images