Fan blade surface defect detection method and detection system

Abstract

The application discloses a fan blade surface defect detection method and a detection system, and relates to the technical field of defect visual detection. The technical problems of poor robustness, easy environmental interference and poor defect positioning accuracy of single-mode detection are solved. In the data layer, the spatio-temporal alignment of multi-source data is realized through RTK high-precision positioning and hardware synchronization. In the feature layer, the color-texture-shape features of visible light, the temperature rise-thermal response features of infrared and the internal reflection features of radar are extracted and fused. In the decision layer, the detection results of each mode are comprehensively judged by calculating the intersection-over-union ratio and setting multi-level threshold rules, so that the accuracy is improved and the false detection rate is reduced. The RTK real-time dynamic differential positioning and multi-view geometric back-projection technology are used to accurately map the defect pixel coordinates detected in the image to the blade three-dimensional digital twin, and the actual physical position of the defect is output, so that manual comparison and search are not needed, and the maintenance preparation time is shortened.

Description

Technical Field

[0001] This invention relates to the field of visual defect detection technology, specifically to a method and system for detecting surface defects on wind turbine blades. Background Technology

[0002] Wind turbine blades are key components in wind turbine generators that capture wind energy, and their surface condition directly affects the safety and power generation efficiency of the unit. Operating for extended periods in harsh environments such as wind, sandstorms, lightning strikes, humidity, and alternating loads, blades are highly susceptible to various defects, including leading-edge erosion, cracks, coating peeling, lightning strike damage, internal delamination, and adhesive failure. If these defects are not detected and repaired promptly, they can propagate, leading to blade breakage or even the collapse of the entire turbine.

[0003] Existing technologies utilize drones equipped with high-definition cameras or infrared thermal imagers to cruise and photograph blades, then employ image processing or deep learning algorithms to identify visible defects such as surface cracks and coating peeling. While this method improves detection efficiency and safety, it still suffers from the following limitations: First, it can only capture information from the outer surface of the blade, failing to detect hidden defects such as internal delamination and bonding failures; second, single-modal detection is highly susceptible to environmental interference such as lighting, shadows, and dirt, resulting in high false positive and false negative rates; third, the positioning accuracy is insufficient, making it difficult to precisely pinpoint the physical location of defects on giant blades; and fourth, it lacks quantitative assessment of defect severity and predictive maintenance recommendations. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for detecting surface defects in wind turbine blades, which solves the problems of poor robustness, susceptibility to environmental interference, and poor defect location accuracy in single-mode detection.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting surface defects in wind turbine blades, which specifically includes the following steps: Step 1: Set up an RTK reference station at the base of the wind turbine under test, generate a flight path based on the spatial position information of the wind turbine blades, and collect real-time thermal image sequences and visible light images; Step 2: Back-project the two-dimensional pixel coordinates of the visible light image to the three-dimensional point cloud, align the multi-source data, extract color features, texture features and shape features from the visible light image and concatenate them into a comprehensive feature vector, input it into a pre-trained classifier to output the defect probability, and mark the candidate regions with a probability greater than a preset threshold as defect candidate regions. Step 3: For each heat map sequence, extract the temperature change curve of each pixel over time, calculate the temperature rise rate, peak temperature and thermal response characteristic parameters, compare the thermal response curve of the defect candidate area with the thermal response curve of the undamaged reference area on the same blade, calculate the similarity score between the temperature difference curve and the zero temperature difference curve, and compare it with the preset threshold. If it is lower than the threshold, the candidate area is marked as a potential defect area. Step 4: Map the spatial locations of the visible light defect candidate region and the infrared potential defect region to the same image coordinate system, calculate the intersection-union ratio (IoU) of each pair of candidate regions, compare the IoU with at least one preset threshold, and according to the comparison results, merge the corresponding candidate regions into the final defect region or output them as independent defects, calculate the fusion confidence and generate defect labels, and output the bounding box, fusion confidence and fusion label of each final defect region.

[0006] As a further aspect of the present invention, the method of back-projecting the two-dimensional pixel coordinates of a visible light image to a three-dimensional point cloud is as follows: Obtain the spatial coordinates of the UAV at each data acquisition moment. With attitude angle Using the pre-calibrated visible light camera intrinsic parameter matrix K and distortion coefficients, arbitrary pixel coordinates (u, v) are back-projected onto a 3D point in the global coordinate system. Its formula is ,in For the attitude rotation matrix of the UAV, d is the gimbal rotation matrix, and d is the depth value.

[0007] As a further aspect of the present invention, the method for extracting color features, texture features, and shape features from a visible light image is as follows: Color characteristics: Calculate the color histogram and color moments in the RGB and HSV color spaces respectively; Texture features: Contrast, correlation, energy, and homogeneity are calculated using local binary mode and gray-level co-occurrence matrix. Shape features: Canny edge detection is used to extract the defect contour, and the morphological parameters such as the roundness, aspect ratio, and convex hull area of ​​the contour are calculated.

[0008] As a further aspect of the present invention, the method for screening the defect candidate region is as follows: For each feature dimension, the mean and standard deviation are calculated on a pre-collected large sample of background regions, and then the feature values ​​of the candidate regions are transformed. ,in and represents the statistical values ​​of the background region, where i represents the type of feature dimension; The standardized color features, texture features, and shape features are concatenated into a comprehensive feature vector, which is then input into the defect classifier. The output probability of the candidate region is then compared with a probability threshold, and candidate regions with a probability greater than the threshold are marked as defect candidate regions.

[0009] As a further aspect of the present invention, the temperature rise rate represents the average slope within the first 0.5 seconds, the peak temperature represents the maximum temperature in the entire sequence, and the thermal response characteristic parameters represent the curve fitted to a first-order exponential model. The steady-state temperature rise A and time constant were obtained. , where the value of e is rounded to two decimal places.

[0010] As a further aspect of the present invention, the method for screening the potential defective regions is as follows: The thermal response curves of the candidate defect region are compared with those of a reference region on the same blade that is more than a preset distance from the defect and has no damage. Temperature difference curves are calculated, and the similarity between the two temperature difference curves is calculated using a dynamic time warping algorithm. Specifically, the DTW distance is defined as... Where K is the number of alignment points. This indicates the temperature difference curve at a certain alignment moment. The value at the given location is used, and the obtained DTW distance is normalized according to the formula. Normalization is performed, where L is the sequence length. To predict the maximum temperature difference, if the similarity is lower than the preset similarity threshold, the candidate defect region is marked as a potential defect region.

[0011] As a further aspect of the present invention, the method for calculating the intersection-union ratio of each pair of candidate regions is as follows: Obtain the set of candidate regions for visible light defects and collection of infrared potential defect regions For each pair of visible light defect candidate regions and infrared potential defect regions, calculate their loU according to the formula. The calculation is performed, where ∪ is the area of ​​the union, ∩ is the area of ​​the intersection of the two rectangles, and a and b represent the types of visible light defect candidate regions and infrared potential defect regions, respectively.

[0012] As a further aspect of the present invention, the method for outputting the bounding box, fusion confidence score, and fusion flag of each final defect region is as follows: Obtain the IOU threshold and compare the IOU value with the IOU threshold. If IOU ≥ θ high If the visible light and infrared detections both identify the same physical defect, then the two regions are merged into a final defect region. The bounding box of this final defect region is the union of the two regions, and the confidence level is calculated. If the confidence level of a visible light defect candidate region exceeds the confidence level threshold, and the confidence level of an infrared potential defect region also exceeds the confidence level threshold, it is marked as a visible light + infrared dual-confirmed defect; otherwise, it is marked as a single-mode high-confidence defect. If θ low <lou≤θ mid If the two regions overlap, they are determined to be partially overlapping. Then, the partially overlapping regions are merged to obtain the defect region. The bounding box of the defect region is the union of the two regions. The combined confidence score is the maximum of the two regions, and it is marked as a partially matching defect. If lou < θ low If the visible light and infrared detection areas correspond to different physical locations, it is determined to be an independent defect.

[0013] As a further aspect of the present invention, after all pairing processes are completed, the unmatched candidate regions are checked. For unmatched visible light regions, if their confidence level exceeds 0.8, they are considered as independent defects and marked as high-confidence isolated visible light curves. If the confidence level is less than 0.5, they are considered as false alarms and discarded. For unmatched infrared regions, if the confidence level exceeds 0.7, an isolated internal thermal anomaly is output, which may be an internal defect; if the confidence level is less than 0.5, it is discarded.

[0014] A wind turbine blade surface defect detection system includes: Positioning and navigation unit: includes a fixed RTK base station erected at the base of the wind turbine tower under test or at an adjacent stable location, and an RTK mobile station mounted on the UAV; the RTK base station broadcasts differential correction signals to the RTK mobile station in real time via a wireless data link to achieve millimeter-level real-time dynamic positioning of the UAV during flight. The UAV flight control unit is configured to receive flight path instructions generated by the ground operation terminal, cruise along the blade surface from the blade root to the blade tip in segments, and automatically detour to the next detection surface of the blade after completing single-surface detection to continue segmented cruise; the flight path is generated by a three-dimensional spatial path generation algorithm along the blade length direction according to preset waypoints based on the input wind turbine blade spatial position information. A multimodal data acquisition unit, mounted on a drone, includes: A high-resolution visible light camera is used to capture visible light images of the leaf surface; Infrared thermal imager, used to acquire infrared thermal image sequences of the blade surface; A thermal excitation source is used to apply a controllable heat flux to the blade surface when reaching a preset waypoint; Ground-penetrating radar detectors are used to collect radar echo data inside the blades; And sensors for acquiring vibration signals and acoustic emission signals; The visible light camera and the infrared thermal imager are triggered by a hardware synchronization signal to synchronously acquire images with the same sampling timing. Multi-source data spatiotemporal alignment and digital twin construction unit: Configured to acquire the spatial coordinates and attitude angles of the UAV at each acquisition moment, it back-projects the two-dimensional pixel coordinates of the visible light image to the three-dimensional spatial point cloud using pre-calibrated camera intrinsic parameters and distortion coefficients, and converts the reflection point position of the ground-penetrating radar echo to the same coordinate system through geometric relationships; an adaptive filter is used to suppress noise in the vibration and acoustic emission data; the highest sampling frequency among all sensors is used as the reference clock to perform time interpolation and resampling on the data of other sensors to achieve time axis synchronization; a high-precision three-dimensional point cloud and surface mesh model of the blade is constructed based on the spatial coordinate data, and the spatiotemporally aligned multi-source detection data is attached to the surface and internal voxel mesh of the three-dimensional model through texture mapping to form a visualized digital twin of the blade; Visible light defect candidate region screening unit: configured to extract color features, texture features and shape features from visible light images, concatenate the standardized features into a comprehensive feature vector, input it into a pre-trained classifier to output the defect probability, and mark the candidate regions with the probability greater than a preset threshold as defect candidate regions; Infrared potential defect region marking unit: configured to extract the temperature change curve of each pixel over time for each heat map sequence, calculate the temperature rise rate, peak temperature and thermal response characteristic parameters; compare the thermal response curve of the defect candidate region with that of the undamaged reference region on the same blade, calculate the temperature difference curve, and use a dynamic time warping algorithm to calculate the similarity score between the temperature difference curve and the zero temperature difference curve; compare the similarity score with a preset threshold, and if it is lower than the threshold, mark the candidate region as a potential defect region; Multimodal fusion decision and defect labeling unit: configured to map the spatial positions of visible light defect candidate regions and infrared potential defect regions to the same image coordinate system, calculate the intersection-union ratio (IoU) of each pair of candidate regions, compare the IoU with at least one preset threshold, and according to the comparison result, fuse the corresponding candidate regions into the final defect region or output them as independent defects, calculate the fusion confidence and generate defect labels, and output the bounding box, fusion confidence and fusion label of each final defect region.

[0015] This invention provides a method and system for detecting surface defects in wind turbine blades. Compared with existing technologies, it has the following advantages: This invention achieves spatiotemporal alignment of multi-source data through RTK high-precision positioning and hardware synchronization at the data layer; at the feature layer, it extracts color-texture-shape features of visible light, temperature rise-thermal response features of infrared, and internal reflection features of radar, and fuses them through random forest or deep network; at the decision layer, it comprehensively judges the detection results of each modality by calculating the intersection-union ratio and setting multi-level threshold rules, thereby improving accuracy and reducing false detection rate compared to single visible light detection.

[0016] This invention utilizes RTK real-time dynamic differential positioning and multi-view geometric back-projection technology to accurately map the pixel coordinates of defects detected in the image onto a three-dimensional digital twin of the blade, outputting the actual physical location of the defect. On-site maintenance personnel can directly locate the defect based on the three-dimensional coordinates without manual comparison, thus shortening maintenance preparation time. It integrates a visible light camera, an infrared thermal imager, a ground-penetrating radar, and an acoustic emission sensor on the same UAV platform, and achieves synchronous acquisition of data from all sensors at the same detection waypoint through a hardware synchronous triggering mechanism. Compared with the existing technology that separates surface and internal detection, this solution can simultaneously acquire images of the blade's appearance, thermal field distribution, internal radar reflection, and dynamic acoustic emission signals in a single flight mission, improving detection efficiency and defect detection rate. It can detect hidden defects such as internal delamination and early bonding failure that are easily missed by existing technologies. Attached Figure Description

[0017] Figure 1 This is a flowchart of the wind turbine blade surface defect detection method of the present invention; Figure 2 This is a block diagram of the wind turbine blade surface defect detection system of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] First Embodiment Please see Figure 1 This application provides a method for detecting surface defects on wind turbine blades, which specifically includes the following steps: Step 1: Set up an RTK reference station at the base of the wind turbine under test, generate a flight path based on the spatial position information of the wind turbine blades, and collect real-time thermal image sequences and visible light images; Step 2: Back-project the two-dimensional pixel coordinates of the visible light image to the three-dimensional point cloud, align the multi-source data, extract color features, texture features and shape features from the visible light image and concatenate them into a comprehensive feature vector, input it into a pre-trained classifier to output the defect probability, and mark the candidate regions with a probability greater than a preset threshold as defect candidate regions. Step 3: For each heat map sequence, extract the temperature change curve of each pixel over time, calculate the temperature rise rate, peak temperature and thermal response characteristic parameters, compare the thermal response curve of the defect candidate area with the thermal response curve of the undamaged reference area on the same blade, calculate the similarity score between the temperature difference curve and the zero temperature difference curve, and compare it with the preset threshold. If it is lower than the threshold, the candidate area is marked as a potential defect area. Step 4: Map the spatial locations of visible light defect candidate regions and infrared potential defect regions to the same image coordinate system, calculate the intersection-union ratio (IoU) of each pair of candidate regions, compare the IoU with at least one preset threshold, and based on the comparison results, fuse the corresponding candidate regions into the final defect region or output them as independent defects; calculate the fusion confidence and generate defect labels; output the bounding box, fusion confidence, and fusion label of each final defect region.

[0020] Second Embodiment As a second embodiment of the present invention, it is implemented based on the first embodiment, and the difference from the first embodiment is as follows; Step 1: Set up a fixed RTK reference station at the base of the wind turbine under test or at an adjacent stable location. After the reference station collects satellite observation data, it broadcasts differential correction signals in real time to the RTK mobile station carried by the UAV via a 4G / 5G network or data transmission radio. Input the spatial position information of the wind turbine blades, including longitude, latitude, altitude and yaw angle, into the system operation terminal. The operation terminal generates a flight path along the blade length direction according to the preset waypoints based on the three-dimensional spatial path generation algorithm. After receiving the flight path instructions, the UAV cruises along the blade surface in segments from the blade root to the blade tip. Upon reaching a preset waypoint, the system controls an illumination source or thermal excitation source to continuously apply a controllable heat flow to the blade surface. The thermal excitation source is a halogen lamp array or a pulsed laser diode, with a heating power of 200W to 500W and a continuous heating time of 0.5 seconds to 1.5 seconds. An infrared thermal imager collects a real-time thermal image sequence of the blade surface at a predetermined sampling frequency. The sampling frequency of the infrared thermal imager is not less than 60Hz, and the sampling frequency of the visible light camera is not less than 30Hz. The two are triggered by a hardware synchronization signal with a synchronization error of less than 1 millisecond. At the same time, under the same sampling sequence, the visible light high-definition camera synchronously collects images of the blade surface. After completing the single-sided inspection, the UAV automatically detours to the next inspection surface of the blade to continue segmented cruise. During the acquisition process, the system associates and stores the longitude, latitude, altitude, pitch angle, roll angle, and yaw angle of each acquisition point in real time.

[0021] Step 2: Obtain the spatial coordinates of the UAV at each data acquisition moment. With attitude angle Using the pre-calibrated visible light camera intrinsic parameter matrix K and distortion coefficients, arbitrary pixel coordinates (u, v) are back-projected onto a 3D point in the global coordinate system. Its formula is ,in For the attitude rotation matrix of the UAV, Here, d represents the gimbal pointing rotation matrix, and d is the depth value obtained from multi-view triangulation or laser ranging. Based on the above operations, the two-dimensional pixel coordinates of the visible light image are back-projected to the three-dimensional spatial point cloud. Then, through geometric relationships, the reflection point positions of the ground-penetrating radar echoes are converted to the same coordinate system. The specific processing method is as follows: According to radar two-way travel time Dielectric constant of blade composite materials Calculate the distance from the antenna to the reflection point. Then, by combining the antenna installation offset and the beam direction angle, the coordinates of the reflection point are converted to the same global coordinate system as visible light. For the vibration and acoustic emission data collected during operation, real-time optical images of the wind turbine blades are recorded simultaneously, and a recursive least squares adaptive filter is used for noise suppression. The highest sampling frequency among all sensors is used as the reference clock. For sensor data with sampling frequencies lower than the reference, cubic spline interpolation is used for resampling: nearest neighbor interpolation is used for discrete event data; cubic spline interpolation is used for continuous numerical data, ensuring strict synchronization and alignment of multi-source data on the time axis. Based on the collected spatial coordinate data, a high-precision 3D point cloud / surface mesh model of the blade is constructed. The spatiotemporally aligned multi-source detection data is then attached to the surface and internal voxel mesh of the 3D model through texture mapping, forming a visualized digital twin of the blade. Appearance features such as color, texture, and shape are extracted from the visible light images, and the specific extraction methods are as follows: Color characteristics: Calculate the color histogram and color moments in the RGB and HSV color spaces respectively; Texture features: Contrast, correlation, energy, and homogeneity are calculated using local binary mode and gray-level co-occurrence matrix. Shape features: Canny edge detection is used to extract the defect contour, and morphological parameters such as the roundness, aspect ratio, and convex hull area of ​​the contour are calculated; Based on the aforementioned apparent features, candidate regions are screened. Specifically, for each feature dimension, the mean and standard deviation are calculated on a pre-collected large sample of background regions. Then, the feature values ​​of the candidate regions are transformed... ,in and , where i represents the statistical value of the background region, and i represents the feature dimension type. The standardized color features, texture features, and shape features are concatenated into a comprehensive feature vector, which is then input into the defect classifier to output the defect probability of the candidate region. Random forest is used as the main classifier, with the following parameter settings: number of decision trees is 100, maximum depth is 10, and minimum number of leaf samples is 2. Training data: more than 500 real defect samples are collected, and experts use annotation tools to accurately mark the defect bounding boxes. More than 1000 background interference samples are collected, and experts also mark them as non-defects. The above samples are divided into training set (70%), validation set (15%), and test set (15%). The output probability is compared with the probability threshold, and candidate regions with a probability greater than the probability threshold are marked as defect candidate regions.

[0022] Step 3: Extract the temperature rise curve and temperature field gradient features from the infrared thermal image, calculate the temperature difference curve between the defective area and the non-defective area, and compare their similarity. The specific processing method is as follows: For each heatmap sequence, the temperature-time curve T(t) of each pixel is extracted, and the temperature rise rate, peak temperature, and thermal response characteristic parameters are calculated. The temperature rise rate represents the average slope in the first 0.5 seconds, the peak temperature represents the maximum temperature in the entire sequence, and the thermal response characteristic parameters represent the curve fitted with a first-order exponential model. The steady-state temperature rise A and time constant were obtained. , where the value of e is rounded to two decimal places; Then, the thermal response curves of the candidate defect region are compared with those of a reference region on the same blade that is more than a preset distance from the defect and has no damage. The temperature difference curves are calculated, and the similarity between the two temperature difference curves is calculated using a dynamic time warping algorithm. Specifically, the DTW distance is defined as... Where K is the number of alignment points. This indicates the temperature difference curve at a certain alignment moment. The value at the given location is used, and the obtained DTW distance is normalized according to the formula. Normalization is performed, where L is the sequence length. To predict the maximum temperature difference, if the similarity is lower than the preset similarity threshold, the candidate defect region is marked as a potential defect region. The preset similarity threshold is obtained by calculating the DTW similarity of at least 10 known defect-free blade surface regions before detection, and taking the average value minus twice the standard deviation as the default threshold.

[0023] Step 4: Merge the spatial locations of visible light defect candidate regions and infrared potential defect regions, calculate their intersection-union ratio (IU / U), and the specific processing procedure is as follows: Obtain the set of candidate regions for visible light defects and collection of infrared potential defect regions Simultaneously, the corresponding defect confidence scores are obtained. For each pair of visible light defect candidate regions and infrared potential defect regions, their IOU is calculated according to the formula. The calculation is performed, where ∪ is the area of ​​the union, ∩ is the area of ​​the intersection of the two rectangles, and a and b represent the types of visible light defect candidate regions and infrared potential defect regions, respectively. Next, three IOU thresholds are set, θ low =0.3, θ mid =0.6, θ high =0.8, the specific IOU threshold here can be adjusted according to the actual situation, and the relationship between the IOU value and the IOU threshold should be compared. If lou≥θ high If the visible light and infrared detections both identify the same physical defect, then the two regions are merged into a final defect region. The bounding box of this final defect region is the union of the two regions, and the confidence level is calculated. If the confidence level of a visible light defect candidate region exceeds the confidence level threshold, and the confidence level of an infrared potential defect region also exceeds the confidence level threshold, it is marked as a visible light + infrared dual-confirmed defect; otherwise, it is marked as a single-mode high-confidence defect. If θ low <lou≤θ mid If the two regions overlap, they are determined to be partially overlapping. Then, the partially overlapping regions are merged to obtain the defect region. The bounding box of the defect region is the union of the two regions. The combined confidence score is the maximum of the two regions, and it is marked as a partially matching defect. If lou < θ low If the visible light and infrared detection areas correspond to different physical locations, it is determined to be an independent defect. After all matching processes are completed, check the unmatched candidate regions. For unmatched visible light regions, if their confidence level exceeds 0.8, they are considered independent defects and marked as high-confidence isolated visible light curves. If the confidence level is less than 0.5, they are considered false alarms and discarded. For unmatched infrared regions, if the confidence level exceeds 0.7, an isolated internal thermal anomaly is output, which may be an internal defect; if the confidence level is less than 0.5, it is discarded.

[0024] Third Embodiment As a third embodiment of the present invention, the focus is on combining the implementation process of the above embodiments.

[0025] Fourth embodiment Please see Figure 2 This application provides a wind turbine blade surface defect detection system, including: Positioning and navigation unit: includes a fixed RTK base station erected at the base of the wind turbine tower under test or at an adjacent stable location, and an RTK mobile station mounted on the UAV; the RTK base station broadcasts differential correction signals to the RTK mobile station in real time via a wireless data link to achieve millimeter-level real-time dynamic positioning of the UAV during flight. The UAV flight control unit is configured to receive flight path instructions generated by the ground operation terminal, cruise along the blade surface from the blade root to the blade tip in segments, and automatically detour to the next detection surface of the blade after completing single-surface detection to continue segmented cruise; the flight path is generated by a three-dimensional spatial path generation algorithm along the blade length direction according to preset waypoints based on the input wind turbine blade spatial position information. A multimodal data acquisition unit, mounted on a drone, includes: A high-resolution visible light camera is used to capture visible light images of the leaf surface; Infrared thermal imager, used to acquire infrared thermal image sequences of the blade surface; A thermal excitation source is used to apply a controllable heat flux to the blade surface when reaching a preset waypoint; A ground-penetrating radar detector is used to collect radar echo data inside the blades. A portable ground-penetrating radar detector is used here to facilitate the mounting of drones. And sensors for acquiring vibration signals and acoustic emission signals; The visible light camera and the infrared thermal imager are triggered by a hardware synchronization signal to synchronously acquire images with the same sampling timing. Multi-source data spatiotemporal alignment and digital twin construction unit: Configured to acquire the spatial coordinates and attitude angles of the UAV at each acquisition moment, it back-projects the two-dimensional pixel coordinates of the visible light image to the three-dimensional spatial point cloud using pre-calibrated camera intrinsic parameters and distortion coefficients, and converts the reflection point position of the ground-penetrating radar echo to the same coordinate system through geometric relationships; an adaptive filter is used to suppress noise in the vibration and acoustic emission data; the highest sampling frequency among all sensors is used as the reference clock to perform time interpolation and resampling on the data of other sensors to achieve time axis synchronization; a high-precision three-dimensional point cloud and surface mesh model of the blade is constructed based on the spatial coordinate data, and the spatiotemporally aligned multi-source detection data is attached to the surface and internal voxel mesh of the three-dimensional model through texture mapping to form a visualized digital twin of the blade; Visible light defect candidate region screening unit: configured to extract color features, texture features and shape features from visible light images, concatenate the standardized features into a comprehensive feature vector, input it into a pre-trained classifier to output the defect probability, and mark the candidate regions with the probability greater than a preset threshold as defect candidate regions; Infrared potential defect region marking unit: configured to extract the temperature change curve of each pixel over time for each heat map sequence, calculate the temperature rise rate, peak temperature and thermal response characteristic parameters; compare the thermal response curve of the defect candidate region with that of the undamaged reference region on the same blade, calculate the temperature difference curve, and use a dynamic time warping algorithm to calculate the similarity score between the temperature difference curve and the zero temperature difference curve; compare the similarity score with a preset threshold, and if it is lower than the threshold, mark the candidate region as a potential defect region; Multimodal fusion decision and defect labeling unit: configured to map the spatial positions of visible light defect candidate regions and infrared potential defect regions to the same image coordinate system, calculate the intersection-union ratio (IoU) of each pair of candidate regions, compare the IoU with at least one preset threshold, and according to the comparison result, fuse the corresponding candidate regions into the final defect region or output them as independent defects, calculate the fusion confidence and generate defect labels, and output the bounding box, fusion confidence and fusion label of each final defect region.

[0026] Some of the data in the above formulas are numerical calculations with dimensions removed, and the contents not described in detail in this specification are all prior art known to those skilled in the art.

[0027] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A method for detecting surface defects in wind turbine blades, characterized in that, The method specifically includes the following steps: Step 1: Set up an RTK reference station at the base of the wind turbine under test, generate a flight path based on the spatial position information of the wind turbine blades, and collect real-time thermal image sequences and visible light images; Step 2: Back-project the two-dimensional pixel coordinates of the visible light image to the three-dimensional point cloud, align the multi-source data, extract color features, texture features and shape features from the visible light image and concatenate them into a comprehensive feature vector, input it into a pre-trained classifier to output the defect probability, and mark the candidate regions with a probability greater than a preset threshold as defect candidate regions. Step 3: For each heat map sequence, extract the temperature change curve of each pixel over time, calculate the temperature rise rate, peak temperature and thermal response characteristic parameters, compare the thermal response curve of the defect candidate area with the thermal response curve of the undamaged reference area on the same blade, calculate the similarity score between the temperature difference curve and the zero temperature difference curve, and compare it with the preset threshold. If it is lower than the threshold, the candidate area is marked as a potential defect area. Step 4: Map the spatial locations of the visible light defect candidate region and the infrared potential defect region to the same image coordinate system, calculate the intersection-union ratio (IoU) of each pair of candidate regions, compare the IoU with at least one preset threshold, and according to the comparison results, merge the corresponding candidate regions into the final defect region or output them as independent defects, calculate the fusion confidence and generate defect labels, and output the bounding box, fusion confidence and fusion label of each final defect region.

2. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method of back-projecting the two-dimensional pixel coordinates of a visible light image to a three-dimensional point cloud is as follows: Obtain the spatial coordinates of the UAV at each data acquisition moment. With attitude angle Using the pre-calibrated visible light camera intrinsic parameter matrix K and distortion coefficients, arbitrary pixel coordinates (u, v) are back-projected onto a 3D point in the global coordinate system. Its formula is ,in For the attitude rotation matrix of the UAV, d is the gimbal rotation matrix, and d is the depth value.

3. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method for extracting color features, texture features, and shape features from visible light images is as follows: Color characteristics: Calculate the color histogram and color moments in the RGB and HSV color spaces respectively; Texture Features: Contrast, correlation, energy, and homogeneity are calculated using a local binary mode and gray-level co-occurrence matrix; Shape features: Canny edge detection is used to extract the defect contour, and the morphological parameters such as the roundness, aspect ratio, and convex hull area of ​​the contour are calculated.

4. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method for filtering the defect candidate region is as follows: For each feature dimension, the mean and standard deviation are calculated on a pre-collected large sample of background regions, and then the feature values ​​of the candidate regions are transformed. ,in and represents the statistical values ​​of the background region, where i represents the type of feature dimension; The standardized color features, texture features, and shape features are concatenated into a comprehensive feature vector, which is then input into the defect classifier. The output probability of the candidate region is then compared with a probability threshold, and candidate regions with a probability greater than the threshold are marked as defect candidate regions.

5. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The temperature rise rate represents the average slope within the first 0.5 seconds, the peak temperature represents the maximum temperature throughout the entire sequence, and the thermal response characteristic parameters represent the curve fitted to a first-order exponential model. The steady-state temperature rise A and time constant were obtained. , where the value of e is rounded to two decimal places.

6. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method for screening the potential defective regions is as follows: The thermal response curves of the candidate defect region are compared with those of a reference region on the same blade that is more than a preset distance from the defect and has no damage. Temperature difference curves are calculated, and the similarity between the two temperature difference curves is calculated using a dynamic time warping algorithm. Specifically, the DTW distance is defined as... Where K is the number of alignment points. This indicates the temperature difference curve at a certain alignment moment. The value at the given location is used, and the obtained DTW distance is normalized according to the formula. Normalization is performed, where L is the sequence length. To predict the maximum temperature difference, if the similarity is lower than the preset similarity threshold, the candidate defect region is marked as a potential defect region.

7. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method for calculating the intersection-union ratio of each pair of candidate regions is as follows: Obtain the set of candidate regions for visible light defects and collection of infrared potential defect regions For each pair of visible light defect candidate regions and infrared potential defect regions, calculate their loU according to the formula. The calculation is performed, where ∪ is the area of ​​the union, ∩ is the area of ​​the intersection of the two rectangles, and a and b represent the types of visible light defect candidate regions and infrared potential defect regions, respectively.

8. The method for detecting surface defects of wind turbine blades according to claim 1, characterized in that, The method for outputting the bounding box, fusion confidence score, and fusion flag for each final defect region is as follows: Obtain the IOU threshold and compare the IOU value with the IOU threshold. If IOU ≥ θ high If the visible light and infrared detections both identify the same physical defect, then the two regions are merged into a final defect region. The bounding box of this final defect region is the union of the two regions, and the confidence level is calculated. If the confidence level of a candidate region for a visible light defect exceeds the confidence threshold, and the confidence level of a potential infrared defect region also exceeds the confidence threshold, it is marked as a visible light + infrared double-confirmed defect; otherwise, it is marked as a single-mode high-confidence defect. If θ low <lou≤θ mid If the two regions overlap, they are determined to be partially overlapping. Then, the partially overlapping regions are merged to obtain the defect region. The bounding box of the defect region is the union of the two regions. The combined confidence score is the maximum of the two regions, and it is marked as a partially matching defect. If lou < θ low If the visible light and infrared detection areas correspond to different physical locations, it is determined to be an independent defect.

9. The method for detecting surface defects of wind turbine blades according to claim 8, characterized in that, After all pairing processes are completed, the unmatched candidate regions are checked. For unmatched visible light regions, if their confidence level exceeds 0.8, they are considered independent defects and marked as high-confidence isolated visible light curves. If the confidence level is less than 0.5, they are considered false alarms and discarded. For unmatched infrared regions, if the confidence level exceeds 0.7, an isolated internal thermal anomaly is output, which may be an internal defect; if the confidence level is less than 0.5, it is discarded.

10. A wind turbine blade surface defect detection system, executed by the wind turbine blade surface defect detection method according to any one of claims 1-9, characterized in that, include: Positioning and navigation units include a fixed RTK base station erected at the base of the wind turbine tower under test or at an adjacent stable location, and an RTK mobile station mounted on a drone; The RTK base station broadcasts differential correction signals to the RTK mobile station in real time via a wireless data link to achieve millimeter-level real-time dynamic positioning of the UAV during flight. The UAV flight control unit receives flight path instructions generated by the ground operation terminal, cruises along the blade surface from the blade root to the blade tip in segments, and automatically detours to the next detection surface of the blade after completing single-surface detection to continue segmented cruise; the flight path is generated by a three-dimensional spatial path generation algorithm along the blade length direction according to preset waypoints based on the input wind turbine blade spatial position information. A multimodal data acquisition unit, mounted on a drone, includes: A high-resolution visible light camera is used to capture visible light images of the leaf surface; Infrared thermal imager, used to acquire infrared thermal image sequences of the blade surface; A thermal excitation source is used to apply a controllable heat flux to the blade surface when reaching a preset waypoint; Ground-penetrating radar detectors are used to collect radar echo data inside the blades; And sensors for acquiring vibration signals and acoustic emission signals; The visible light camera and the infrared thermal imager are triggered by a hardware synchronization signal to synchronously acquire images with the same sampling timing. Multi-source data spatiotemporal alignment and digital twin construction unit: Acquire the spatial coordinates and attitude angles of the UAV at each acquisition moment; use pre-calibrated camera intrinsic parameters and distortion coefficients to back-project the two-dimensional pixel coordinates of the visible light image to the three-dimensional spatial point cloud; convert the reflection point position of the ground-penetrating radar echo to the same coordinate system through geometric relationships; use an adaptive filter to suppress noise in vibration and acoustic emission data; use the highest sampling frequency among all sensors as the reference clock to perform time interpolation and resampling on the data of other sensors to achieve time axis synchronization; construct a high-precision three-dimensional point cloud and surface mesh model of the blade based on the spatial coordinate data; attach the spatiotemporally aligned multi-source detection data to the surface and internal voxel mesh of the three-dimensional model through texture mapping to form a visualized digital twin of the blade; Visible light defect candidate region screening unit: extracts color features, texture features and shape features from visible light images, concatenates the standardized features into a comprehensive feature vector, inputs it into a pre-trained classifier to output the defect probability, and marks the candidate regions with the probability greater than a preset threshold as defect candidate regions; Infrared potential defect region marking unit: For each heat map sequence, extract the temperature change curve of each pixel over time, calculate the temperature rise rate, peak temperature, and thermal response characteristic parameters; compare the thermal response curve of the candidate defect region with that of the undamaged reference region on the same blade, calculate the temperature difference curve, and use a dynamic time warping algorithm to calculate the similarity score between the temperature difference curve and the zero temperature difference curve; compare the similarity score with a preset threshold, and if it is lower than the threshold, mark the candidate region as a potential defect region; Multimodal fusion decision and defect labeling unit: Maps the spatial positions of visible light defect candidate regions and infrared potential defect regions to the same image coordinate system, calculates the intersection-union ratio (IoU) of each pair of candidate regions, compares the IoU with at least one preset threshold, and according to the comparison result, fuses the corresponding candidate regions into the final defect region or outputs them as independent defects, calculates the fusion confidence and generates defect labels, and outputs the bounding box, fusion confidence and fusion label of each final defect region.

Images