A fan blade inner cavity defect identification and positioning method based on image fusion

Abstract

This invention discloses a method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion, belonging to the field of wind power equipment operation and maintenance technology. The method includes: S1, acquiring multi-source images and location data and performing preprocessing; S2, fusing and panoramic stitching of multi-view visible light images; S3, performing cross-modal feature fusion of visible light and infrared images to obtain a global representation of defect features; S4, achieving automatic defect identification and classification based on an improved YOLOv11 model; S5, completing global deduplication and precise location of defects by combining pixel coordinates and spatial coordinates; S6, automatically generating multi-dimensional statistical analysis and inspection reports; S7, establishing a model feedback optimization mechanism to achieve adaptive iteration. This invention improves the accuracy and efficiency of defect detection through the innovative combination of multi-source image fusion and deep learning technology, realizing the full-process automation of wind turbine blade internal cavity inspection, and providing strong support for intelligent operation and maintenance of wind farms.

Description

Technical Field

[0001] This invention belongs to the field of wind power equipment operation and maintenance technology, and particularly relates to a method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion. Background Technology

[0002] With the rapid development of the wind power industry, the capacity of individual wind turbines is constantly increasing. As the core component for capturing wind energy, the blades' operating status directly affects the turbine's power generation efficiency and safety stability. Wind turbine blades are mostly made of composite materials such as fiberglass reinforced resin and operate in complex outdoor environments for extended periods, needing to withstand multiple loads including strong winds, temperature differences, and ultraviolet radiation. This makes them prone to defects such as bulging, wrinkles, delamination, and cracks within the blade cavity. If these defects are not detected and repaired in a timely manner, they may continue to expand, leading to serious malfunctions such as blade breakage, resulting in significant economic losses and safety risks.

[0003] Currently, the detection of defects in the internal cavity of wind turbine blades mainly relies on traditional detection methods, which have many technical bottlenecks: Traditional manual inspection requires professionals to carry flaw detection equipment into the blade cavity, or to make indirect judgments through external inspection. This is not only labor-intensive and inefficient, but also dependent on the experience of the inspectors, making it prone to missed or false detections due to blind spots and human negligence. In particular, the accuracy of identifying minute and complex defects is difficult to guarantee. Physical detection methods such as ultrasonic, X-ray, and infrared imaging can improve the detection accuracy to a certain extent, but the equipment is expensive and the operation is complex, making it difficult to achieve efficient inspection of large-scale wind farms. Moreover, these technologies are mostly designed for single types of defects and have limited ability to identify complex defects such as composite cracks and material deterioration. At the same time, the detection process is time-consuming, affecting the normal operation time of the wind turbine. Existing diagrams Image-based detection methods rely on a single perspective or type of image for defect identification, resulting in limited image perspective and incomplete defect feature representation. Furthermore, existing deep learning models largely depend on training with real defect samples, but the cost and scarcity of blade internal cavity defect samples lead to insufficient model generalization ability, making it difficult to adapt to defect detection needs of different scales and shapes. They also suffer from defect localization bias and duplicate labeling, failing to meet the accuracy requirements of industrial-grade inspections. Existing detection methods often require manual intervention for data processing, defect verification, and report generation, lacking fully automated inspection solutions. Moreover, most models have fixed architectures, unable to adaptively optimize based on new defect samples, making it difficult to adapt to the diverse changes in blade internal cavity defect types over the long term.

[0004] Existing methods for detecting defects in the internal cavity of wind turbine blades suffer from low efficiency, poor accuracy, insufficient generalization ability, and low automation, failing to meet the large-scale, high-precision, and high-efficiency inspection requirements of modern wind farms. Therefore, there is an urgent need to develop a defect identification and localization method that integrates multi-source image information, optimizes model training strategies, and achieves full-process automation to improve the accuracy and efficiency of wind turbine blade internal cavity defect detection and ensure the safe and stable operation of wind turbines. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion, addressing the problems of existing detection methods such as single image perspective, defect location deviation, low recognition rate of complex defects, and insufficient generalization ability for multiple types of defects. This invention addresses the practical needs of wind turbine blade internal cavity inspection by integrating panoramic stitching of multi-view visible light images with feature fusion technology of infrared-visible light images, combined with an improved deep learning model to achieve accurate defect identification and location.

[0006] To achieve the above objectives, this invention provides a method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion, comprising the following steps: S1. Acquire multi-source images and location data, and preprocess and calibrate the acquired multi-source image data; S2. Fusion and panoramic stitching of the multi-view visible light images acquired and preprocessed in S1; S3. Perform cross-modal feature fusion on the visible light and infrared images acquired and preprocessed in S1 and map them to the spatial location corresponding to the complete visible light panoramic image generated in S2 to obtain a global representation of the defect features on the panoramic image. S4. Establish a defect detection model based on YOLOv11, improve and train the model, and automatically identify and classify defects; S5. Combining the image pixel coordinates obtained in S4 with the spatial coordinate information of the inspection robot's position sensor obtained in S1, perform global unique deduplication of defects and precise spatial positioning of the blade's inner cavity. S6. Automatically generate multi-dimensional statistical analysis of defects and standardized electronic inspection reports; S7. Establish a feedback optimization mechanism for the defect detection model, and optimize the model by combining newly added inspection data and defect samples.

[0007] Preferably, S1 specifically includes: S101. The inspection robot is equipped with a visible light camera that includes front view, left front view, right front view, gimbal main view and rear view. It collects multi-view visible light images of different positions in the blade cavity according to the preset inspection mode. The collected visible light image data includes camera view, shooting time and inspection task information. S102. The infrared thermal imaging camera mounted on the inspection robot collects infrared images at the same position and angle as the visible light images, and records the temperature information of the defect area. S103. Using the high-precision position sensor of the inspection robot, the spatial coordinate information of the blade cavity corresponding to each frame of image is collected in real time. At the same time, it records the inspection depth, travel speed and vehicle posture auxiliary data of the inspection robot; S104. The collected multi-source image data and location data are stored in real time to the vehicle's USB flash drive for local data backup, and simultaneously uploaded to the inspection platform; S105. To address the noise, distortion, and viewing angle deviation present in the acquired visible light and infrared images, noise removal, distortion calibration, and image enhancement processing are performed.

[0008] Preferably, the specific content of S105 is as follows: An adaptive median filtering algorithm is used for visible light images to dynamically adjust the size of the filtering window according to the image noise distribution, remove random noise and retain the edge details of the blade cavity; for infrared images, a Gaussian filtering algorithm combined with a wavelet denoising algorithm is used to remove thermal imaging noise and retain the defect feature areas of temperature abrupt changes. Based on the camera intrinsic parameter calibration results, radial and tangential distortion calibrations are performed on visible light images and infrared images respectively to eliminate image distortion caused by lens distortion; through a spatial registration algorithm, the infrared image and the visible light image at the same position are registered at the pixel level to make the spatial positions of the two types of images correspond one-to-one. Contrast enhancement and edge enhancement are performed on defect areas in visible light images, and gradient thresholds are set to extract the structural edges of the blade cavity; temperature normalization and thermal feature enhancement are performed on infrared images to highlight the temperature difference between defect areas and normal areas.

[0009] Preferably, the specific content of S2 is as follows: For the four visible light images (front view, left front view, right front view, and gimbal main view) collected by the inspection robot at a certain position in the blade cavity, a feature point matching and seamless fusion algorithm is used to extract SIFT feature points from each view image. Geometric transformation and stitching of the visible light images are performed through homography matrix. Then, brightness equalization and edge fusion algorithms are used to eliminate stitching seams and generate a 360° visible light panoramic image of that position, which fully displays the entire blade cavity at that position. Based on the spatial coordinate information of the inspection robot's position sensor, the 360° visible light panoramic image of different positions inside the blade is stitched together segment by segment according to the spatial order of the inspection path. Global feature matching and stitching error correction technology is used to eliminate the stitching offset caused by the movement of the inspection robot, thereby generating a complete visible light panoramic image of the entire wind turbine blade's inner cavity.

[0010] Preferably, the specific content of S3 is as follows: Based on the clear display of defect geometric features in visible light images and the highlighting of defect thermal features in infrared images, a dual-branch feature fusion network is constructed to extract complementary defect features from the two types of images: Two parallel convolutional neural network branches are constructed: a visible light feature branch and an infrared feature branch. The visible light feature branch uses a lightweight CNN network to extract the geometric features of the defects. The infrared feature branch uses an improved CNN network with a temperature attention mechanism to extract the thermal features of the defects and suppress invalid features in the background region. In the feature output layer of the two branches, a feature fusion layer is set up. A weighted fusion and channel attention mechanism is adopted to automatically learn the weight allocation of visible light features and infrared features. The two types of features are fused at the channel level and spatial level to generate an infrared-visible light fusion defect feature map. The fused feature map is reduced in dimensionality and enhanced in terms of features to remove redundant features and highlight the core features of the defects. The infrared-visible light fused defect feature map of each location is accurately mapped to the corresponding position of the complete visible light panoramic image generated by S2 based on the spatial coordinate information of the inspection robot's position sensor, so as to obtain a full-domain and accurate defect fusion feature representation on the panoramic image.

[0011] Preferably, the specific content of S4 is as follows: S401. Establish a defect detection model based on YOLOv11 and improve the model; A Gaussian noise / salt-and-pepper noise injection layer is added between the feature fusion layer and the detection head to simulate sensor noise and interference factors such as uneven lighting during the inspection process. Attention weights are set to address the characteristic differences of four types of defects: bulges, wrinkles, delamination, and cracks, so that the model focuses on the core features of each type of defect. A weighted multi-task total loss function is constructed using CIoU loss and class-weighted cross-entropy loss. The expression is as follows: ; In the formula, Position the bounding box; Target confidence level; Classify defects by category; Weights for the bounding box localization loss; Weight the target confidence level loss; Defect category classification loss weights; S402. Train and optimize the defect detection model improved from S401; The system integrates real blade internal cavity defect images, virtual defect images generated by a two-layer generative adversarial network, and defect-free images, and divides them into training, validation, and test sets according to proportions. The first stage uses simple samples with large defect sizes and clear backgrounds for training, allowing the model to learn basic defect morphology. The second stage uses difficult samples with small defect sizes and blurry / occluded features for training, improving the model's ability to withstand interference. The third stage uses real defect samples for fine-tuning, bridging the distribution differences between virtual and real samples. Optimize the trained model using the validation set; S403. Input the infrared-visible light fusion defect feature map generated in S3 into the trained defect detection model. The model automatically outputs the defect category, confidence level, and pixel coordinates of the image.

[0012] Preferably, the specific content of S5 is as follows: Based on the physical dimensional parameters of the wind turbine blades, a mapping relationship is established between the pixel coordinate system of the image and the spatial coordinate system of the blade's inner cavity. The defect pixel coordinates output by the model are then converted into the actual spatial coordinates of the blade's inner cavity. ; Based on the actual spatial coordinates, category and size characteristics of the defect, the Euclidean distance clustering algorithm is used to automatically remove duplicate marks of the same defect in images from different viewpoints and positions, calculate the minimum bounding box of the defect region and determine the unique global identifier of the defect. The actual spatial coordinates of the deduplicated defects are accurately superimposed onto the complete visible light panoramic image of the wind turbine blade cavity generated in S2. At the same time, the type, size and temperature information of the defects are marked to achieve the visualized location of the defects in the entire inner cavity of the blade.

[0013] Preferably, the specific content of S6 is as follows: S601. Based on the defect identification and location results, perform three types of statistical analysis: The percentage of defects classified by type is as follows: bulges, wrinkles, delamination, and cracks. The number and distribution density of defects in different regions of the blade tip, middle section and tail section were statistically analyzed according to the defect area distribution. By combining historical inspection data, the defect development trend analysis can be performed on the changing trends of the number and size of defects on the same blade and at the same wind turbine station, thus enabling early warning of defects. S602. Manual review and modification: Based on the manual review interface, maintenance personnel review, modify, and annotate automatically identified defects, and the review results are updated to the defect database simultaneously. S603, Automatically generate electronic inspection reports.

[0014] Preferably, the electronic inspection report in S6 includes the following content: Environmental information to be monitored: inspection time, fan number, blade parameters, ambient temperature and humidity, and the working status of the inspection robot; Fault location distribution map: A panoramic view of the entire blade cavity with superimposed defect locations; Defect supporting information: original infrared image, visible light image and fused image of each defect point; Detailed defect information: spatial coordinates, type, size, temperature information, and confidence level of the defect; Defect statistics: number of various defects, regional distribution ratio, and development trend analysis.

[0015] Preferably, the specific content of S7 is as follows: New defect samples that have been manually reviewed are added to the defect sample library, and the category, characteristics and spatial location information of the defects are supplemented and labeled. By using newly added defect samples, the trained defect detection model is lightly fine-tuned and the weights of the infrared-visible light feature fusion network are updated, enabling the model to learn the features of new defects. The defect identification and location engine operates separately from other parts of the inspection platform in a decoupled manner.

[0016] Therefore, the present invention employs the above-mentioned image fusion-based method for identifying and locating defects in the internal cavity of wind turbine blades, which has the following beneficial effects: (1) By integrating multi-view visible light panoramic stitching and infrared-visible light cross-modal feature fusion technology, a 360° visible light panoramic image of the blade cavity is generated through SIFT feature point matching and seamless fusion, which solves the problem of single viewpoint and incomplete display of the whole cavity information in traditional detection images; a dual-branch feature fusion network is constructed, which combines the defect geometric features of the visible light image and the thermal features of the infrared image for complementary extraction, and automatically allocates feature weights through the channel attention mechanism, effectively highlighting the core features of the defect and suppressing background redundant information. Compared with the single image detection method, it realizes the whole-domain and accurate characterization of defect features, and greatly reduces the probability of missing defects in complex cavity environments; (2) Improve the YOLOv11 model to adapt to the blade defect detection scenario, solve the problems of large scale changes and uneven category distribution, and combine high-precision position sensing and global deduplication algorithm to realize the visualization and accurate positioning of defects, and meet the requirements of industrial-grade detection. (3) The training dataset was expanded by fusing real defect images with virtual defect images generated by a two-layer generative adversarial network, and a three-stage training strategy was adopted to effectively bridge the distribution differences between virtual and real samples and improve the model’s generalization ability for defects of different scales and shapes. The model feedback optimization mechanism was established to solve the problem of insufficient ability of traditional detection models to identify new defects, and the adaptive iterative optimization of the model was realized, which can adapt to the diverse changes in the types of defects in the inner cavity of wind turbine blades in the long term. (4) The spatial coordinates, categories, sizes, and temperature information of defects are accurately superimposed onto the complete visible light panoramic image of the blade cavity, which intuitively shows the full-area distribution of defects and provides clear and accurate defect location references for operation and maintenance personnel, greatly reducing the difficulty and time cost of on-site maintenance; combined with historical inspection data to analyze the development trend of defects, it can realize early warning of defects, effectively reduce the downtime of wind turbine blade failure, improve the safety and reliability of wind turbine operation, and reduce the overall operation and maintenance cost of wind farm.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion, according to the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0020] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0021] The following is combined Figure 1 The embodiments of the present invention will be described in detail below.

[0022] Example This embodiment takes the internal cavity inspection of 10 wind turbines (blades are 75 meters long glass fiber reinforced resin blades) in a large wind farm as the application scenario. It is necessary to detect four types of defects: bulges, wrinkles, delamination and cracks. The defect identification accuracy is required to be ≥95% and the positioning accuracy is ≤±10cm.

[0023] Hardware configuration: Inspection robot: IP54 protection rating, weight 7.8kg, supports independent four-wheel drive, turning on the spot, maximum climbing angle of 30°, maximum obstacle crossing height of 4cm, full load range of 1.8km, communication distance of 250m; equipped with high-precision position sensor, the positioning accuracy inside the blade is ±8cm.

[0024] Image acquisition equipment: The robot is equipped with five 8-megapixel (3840×2160) visible light cameras (front view, left front view, right front view, gimbal main view, and rear view), each camera is equipped with a supplementary light (supporting four levels of intensity adjustment); it is also equipped with an infrared thermal imaging camera with a temperature measurement range of -20~150℃ and a temperature resolution of 0.1℃, which can achieve synchronous acquisition at the same position and from the same angle as the visible light cameras.

[0025] Auxiliary equipment: 10-inch mobile operating platform (5-hour battery life), supporting robot remote control, real-time image display and inspection mode switching; vehicle-mounted USB flash drive (128G capacity) for local data backup; intelligent server (configuration meets requirements: 12-core 24-thread CPU, 32G DDR4 memory, 4TB SATA hard drive, 3080 graphics card) for model training and data processing.

[0026] Software and data preparation: Model training dataset: It integrates 3000 real images of defects in the internal cavity of wind turbine blades (including 800 images of bulges, 750 images of wrinkles, 700 images of delamination, and 750 images of cracks), 2000 virtual defect images (1200 images of primary defects and 800 images of complex defects) generated based on a two-layer generative adversarial network, and 1000 defect-free images, and divides them into training set, validation set and test set in an 8:1:1 ratio.

[0027] Preset parameter configuration: Image preprocessing: The initial window size for the adaptive median filter in visible light images was set to 3×3, and the standard deviation for the Gaussian filter in infrared images was set to 0.8; contrast limiting factor... =0.8, gradient threshold =0.3.

[0028] Model loss function weights: =7.5, =1.0, =0.5.

[0029] Inspection mode parameters: In semi-automatic inspection mode, the inspection depth is set to the full length of the blade, the travel speed is 0.3m / s, the number of consecutive photos is 3, the interval between photos is 0.5m, and the camera fill light intensity is set to level 2.

[0030] Implementation steps: S1. Perform multi-source image and location data acquisition, and perform image preprocessing and distortion calibration; A semi-automatic inspection mode was set up via a mobile operating platform, and inspection tasks were assigned to the inspection robots corresponding to 10 wind turbines. The robots entered the blade interior according to a preset path and simultaneously completed the following data collection tasks: The visible light camera acquires multi-view images of different positions at the front, middle and rear ends of the blade from the perspectives of front view, left front view, right front view, gimbal main view and rear view. Each image contains metadata such as camera view, shooting time and inspection task number. An infrared thermal imaging camera simultaneously acquires infrared images from the same location and angle, recording temperature information of the defect area; The position sensor acquires the spatial coordinates of the blade's internal cavity corresponding to each frame of image in real time. Simultaneously record inspection depth, travel speed, and vehicle posture data; All collected data is stored in real time to the vehicle's USB flash drive, and after the inspection is completed, it can be uploaded to the intelligent server with one click via the USB flash drive.

[0031] The intelligent server automatically processes the uploaded multi-source images: Noise Removal: For visible light images, adaptive median filtering is used, and the size of the filtering window (3×3~7×7) is dynamically adjusted according to the noise distribution to remove random noise and retain edge details; for infrared images, Gaussian filtering combined with wavelet denoising is used to remove thermal imaging noise and retain areas of temperature abrupt change. Distortion calibration: Based on the camera intrinsic parameter calibration results (intrinsic parameter matrix K, radial distortion coefficients k1=-0.032, k2=0.011, tangential distortion coefficients p1=0.002, p2=-0.001), radial and tangential distortion calibrations are performed on the visible light image and the infrared image respectively; pixel-level registration between the infrared image and the visible light image at the same location is achieved through a spatial registration algorithm; Image enhancement: Contrast enhancement and edge enhancement are performed on defect areas in visible light images to extract the edges of the blade's internal cavity structure; temperature normalization is performed on infrared images to highlight the temperature difference between defective and normal areas.

[0032] S2, Multi-view visible light image fusion and panoramic stitching; Single-location 360° panoramic fusion: For each acquisition location, SIFT feature points are extracted from four visible light images: front view, left front view, right front view, and gimbal main view. Geometric transformation and stitching are achieved through homography matrix. Then, brightness equalization and edge fusion are performed to eliminate stitching seams and generate a 360° visible light panoramic image of that location. Full-blade panoramic stitching: Based on the spatial coordinate information of the position sensor, the 360° panoramic images of different positions are stitched together segment by segment according to the inspection path. Global feature matching and stitching error correction technology are used to eliminate robot motion offset and generate a complete visible light panoramic image of the inner cavity of each blade of 10 wind turbines.

[0033] S3, infrared-visible cross-modal feature fusion; Feature extraction: Through a dual-branch feature fusion network, the visible light feature branch (MobileNetV3) extracts the geometric features of defects (shape, edge, size), and the infrared feature branch (improved CNN + temperature attention mechanism) extracts the thermal features of defects (temperature distribution, temperature gradient). Feature fusion: Through weighted fusion and channel attention mechanism of feature fusion layer, two types of feature weights are automatically assigned (geometric feature weight 0.6, thermal feature weight 0.4) to generate infrared-visible light fused defect feature map; Feature mapping: The fused feature map of each location is accurately mapped to the complete visible light panoramic image of the corresponding blade based on the spatial coordinate information to obtain a global representation of the defect features.

[0034] S4. Defect identification and classification; The fused feature map is input into the trained and optimized improved YOLOv11 model, and the model automatically outputs the defect category, confidence score, and pixel coordinates: The model simulates sensor noise interference through a noise injection layer, and combines CIoU loss and class-weighted cross-entropy loss to accurately identify defect types. The test results of 10 wind turbine blades showed that the accuracy rate of bulge identification was 96.8%, wrinkle identification was 97.2%, delamination identification was 95.5%, and crack identification was 98.1%, all of which met the requirement of ≥95%.

[0035] S5. Global deduplication and precise location of defects; Coordinate transformation: Based on the physical dimension parameters of the wind turbine blades, a mapping relationship between the pixel coordinate system and the spatial coordinate system is established to convert the defect pixel coordinates into actual spatial coordinates. ; Global deduplication: The Euclidean distance clustering algorithm is used to deduplicat the duplicate markers of the same defect in images from different viewpoints. The clustering threshold is set to 15cm to determine the unique global identifier of the defect. Visualized localization: The spatial coordinates, category, size, and temperature information of the deduplicated defects are overlaid onto a panoramic visible light image of the entire blade, visually displaying the defect distribution. For example, a degumming defect was detected in the middle section of blade number WFL-003, with spatial coordinates (18.6m, 1.2m, 0.7m), a size of 25mm × 40mm, and a temperature 3.2℃ higher than the surrounding area.

[0036] S6. Defect Statistics and Report Generation; Multi-dimensional statistics: According to the defect type, a total of 12 bulges, 8 wrinkles, 15 delaminations, and 10 cracks were detected in 10 wind turbines; According to the regional distribution, the defects in the middle section of the blade accounted for 62%, the front end 25%, and the tail end 13%; Combined with historical data, it was found that the crack defect size of 3 wind turbines increased by 5% to 8% compared with the last inspection, and an early warning was issued. Manual review: Operation and maintenance personnel review the automatically identified defects through the platform's manual review interface, correct two misjudged minor scratches (not cracks), and the review results are updated to the database simultaneously; Report generation: The system automatically generates a standardized electronic inspection report, which includes inspection environment information, defect location distribution map, original infrared / visible light / fusion image of each defect, spatial coordinates, size, temperature and statistical analysis results, and pushes them to the wind farm operation and maintenance management terminal in real time.

[0037] S7, Model Feedback Optimization Two misjudged samples and three newly added novel micro-debonding samples were added to the defect sample library after manual review. The original model was then lightly tweaked and the feature fusion network weights were updated to enable the model to learn new defect features and complete adaptive optimization.

[0038] Implementation results: Inspection efficiency: The inspection time for a single wind turbine blade has been reduced from 5 hours to 2 hours using traditional manual inspection, and the data upload and report generation time is ≤10 minutes, resulting in an overall improvement in inspection efficiency of 60%. Detection accuracy: The average identification accuracy of the four types of defects is 96.9%, the defect location accuracy is ±7.2cm, and there are no missed defects; Engineering value: By analyzing defect trends, three potential risk defects were predicted in advance, avoiding wind turbine failures and downtime; the automated process reduced manual workload by 80%, significantly reducing operation and maintenance costs.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion, characterized in that, Includes the following steps: S1. Acquire multi-source images and location data, and preprocess and calibrate the acquired multi-source image data; S2. Fusion and panoramic stitching of the multi-view visible light images acquired and preprocessed in S1; S3. Perform cross-modal feature fusion on the visible light and infrared images acquired and preprocessed in S1 and map them to the spatial location corresponding to the complete visible light panoramic image generated in S2 to obtain a global representation of the defect features on the panoramic image. S4. Establish a defect detection model based on YOLOv11, improve and train the model, and automatically identify and classify defects; S5. Combining the image pixel coordinates obtained in S4 with the spatial coordinate information of the inspection robot's position sensor obtained in S1, perform global unique deduplication of defects and precise spatial positioning of the blade's inner cavity. S6. Automatically generate multi-dimensional statistical analysis of defects and standardized electronic inspection reports; S7. Establish a feedback optimization mechanism for the defect detection model, and optimize the model by combining newly added inspection data and defect samples.

2. The method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion according to claim 1, characterized in that, S1 specifically includes: S101. The inspection robot is equipped with a visible light camera that includes front view, left front view, right front view, gimbal main view and rear view. It collects multi-view visible light images of different positions in the blade cavity according to the preset inspection mode. The collected visible light image data includes camera view, shooting time and inspection task information. S102. The infrared thermal imaging camera mounted on the inspection robot collects infrared images at the same position and angle as the visible light images, and records the temperature information of the defect area. S103. Using the high-precision position sensor of the inspection robot, the spatial coordinate information of the blade cavity corresponding to each frame of image is collected in real time. At the same time, it records the inspection depth, travel speed and vehicle posture auxiliary data of the inspection robot; S104. The collected multi-source image data and location data are stored in real time to the vehicle's USB flash drive for local data backup, and simultaneously uploaded to the inspection platform; S105. To address the noise, distortion, and viewing angle deviation present in the acquired visible light and infrared images, noise removal, distortion calibration, and image enhancement processing are performed.

3. The method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion according to claim 2, characterized in that, The specific content of S105 is as follows: An adaptive median filtering algorithm is used for visible light images to dynamically adjust the size of the filtering window according to the image noise distribution, remove random noise and retain the edge details of the blade cavity; for infrared images, a Gaussian filtering algorithm combined with a wavelet denoising algorithm is used to remove thermal imaging noise and retain the defect feature areas of temperature abrupt changes. Based on the camera intrinsic parameter calibration results, radial and tangential distortion calibrations are performed on visible light images and infrared images respectively to eliminate image distortion caused by lens distortion; through a spatial registration algorithm, the infrared image and the visible light image at the same position are registered at the pixel level to make the spatial positions of the two types of images correspond one-to-one. Contrast enhancement and edge enhancement are performed on defect areas in visible light images, and gradient thresholds are set to extract the structural edges of the blade cavity. Temperature normalization and thermal feature enhancement are performed on infrared images to highlight the temperature difference between defective and normal areas.

4. The method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion according to claim 3, characterized in that, The specific details of S2 are as follows: For the four visible light images (front view, left front view, right front view, and gimbal main view) collected by the inspection robot at a certain position in the blade cavity, a feature point matching and seamless fusion algorithm is used to extract SIFT feature points from each view image. Geometric transformation and stitching of the visible light images are performed through homography matrix. Then, brightness equalization and edge fusion algorithms are used to eliminate stitching seams and generate a 360° visible light panoramic image of that position, which fully displays the entire blade cavity at that position. Based on the spatial coordinate information of the inspection robot's position sensor, the 360° visible light panoramic image of different positions inside the blade is stitched together segment by segment according to the spatial order of the inspection path. Global feature matching and stitching error correction technology is used to eliminate the stitching offset caused by the movement of the inspection robot, thereby generating a complete visible light panoramic image of the entire wind turbine blade's inner cavity.

5. The method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion according to claim 4, characterized in that, The specific details of S3 are as follows: Based on the clear display of defect geometric features in visible light images and the highlighting of defect thermal features in infrared images, a dual-branch feature fusion network is constructed to extract complementary defect features from the two types of images: Construct two parallel convolutional neural network branches: a visible light feature branch and an infrared feature branch. The visible light feature branch uses a lightweight CNN network to extract the geometric features of the defects; The infrared feature branch employs an improved CNN network with an added temperature attention mechanism to extract thermal features of defects and suppress invalid features in the background region. In the feature output layer of the two branches, a feature fusion layer is set up. A weighted fusion and channel attention mechanism is adopted to automatically learn the weight allocation of visible light features and infrared features. The two types of features are fused at the channel level and spatial level to generate an infrared-visible light fusion defect feature map. Dimensionality reduction and feature enhancement are performed on the fused feature map to remove redundant features and highlight the core features of the defect. The infrared-visible light fused defect feature map of each location is accurately mapped to the corresponding position of the complete visible light panoramic image generated by S2 based on the spatial coordinate information of the inspection robot's position sensor, so as to obtain a full-domain and accurate defect fusion feature representation on the panoramic image.

6. The method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion according to claim 5, characterized in that, The specific details of S4 are as follows: S401. Establish a defect detection model based on YOLOv11 and improve the model; A Gaussian noise / salt-and-pepper noise injection layer is added between the feature fusion layer and the detection head to simulate sensor noise and interference factors such as uneven lighting during the inspection process. Attention weights are set to address the characteristic differences of four types of defects: bulges, wrinkles, delamination, and cracks, so that the model focuses on the core features of each type of defect. A weighted multi-task total loss function is constructed using CIoU loss and class-weighted cross-entropy loss. The expression is as follows: ； In the formula, Position the bounding box; Target confidence level; Classify defects by category; Weights for the bounding box localization loss; Weight the target confidence level loss; Defect category classification loss weights; S402. Train and optimize the defect detection model improved from S401; The system integrates real blade internal cavity defect images, virtual defect images generated by a two-layer generative adversarial network, and defect-free images, and divides them into training, validation, and test sets according to proportions. The first stage uses simple samples with large defect sizes and clear backgrounds for training, allowing the model to learn basic defect morphology. The second stage uses difficult samples with small defect sizes and ambiguity / occlusion to train the model and improve its anti-interference ability; the third stage uses real defect samples for fine-tuning to bridge the distribution difference between virtual and real samples. Optimize the trained model using the validation set; S403. Input the infrared-visible light fusion defect feature map generated in S3 into the trained defect detection model. The model automatically outputs the defect category, confidence level, and pixel coordinates of the image.

7. The method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion according to claim 6, characterized in that, The specific details of S5 are as follows: Based on the physical dimensional parameters of the wind turbine blades, a mapping relationship is established between the pixel coordinate system of the image and the spatial coordinate system of the blade's inner cavity. The defect pixel coordinates output by the model are then converted into the actual spatial coordinates of the blade's inner cavity. ; Based on the actual spatial coordinates, category and size characteristics of defects, the Euclidean distance clustering algorithm is used to automatically remove duplicate marks of the same defect in images from different viewpoints and positions, calculate the minimum bounding box of the defect region and determine the unique global identifier of the defect. The actual spatial coordinates of the deduplicated defects are accurately superimposed onto the complete visible light panoramic image of the wind turbine blade cavity generated in S2. At the same time, the type, size and temperature information of the defects are marked to achieve the visualized location of the defects in the entire inner cavity of the blade.

8. The method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion according to claim 7, characterized in that, The specific details of S6 are as follows: S601. Based on the defect identification and location results, perform three types of statistical analysis: The percentage of defects classified by type is as follows: bulges, wrinkles, delamination, and cracks. The number and distribution density of defects in different regions of the blade tip, middle section and tail section were statistically analyzed according to the defect area distribution. By combining historical inspection data, the defect development trend analysis can be performed on the changing trends of the number and size of defects on the same blade and at the same wind turbine station, thus enabling early warning of defects. S602. Manual review and modification: Based on the manual review interface, maintenance personnel review, modify, and annotate automatically identified defects, and the review results are updated to the defect database simultaneously. S603, Automatically generate electronic inspection reports.

9. The method for identifying and locating defects in the internal cavity of wind turbine blades based on image fusion according to claim 8, characterized in that, The electronic inspection report in S6 includes the following: Environmental information to be monitored: inspection time, fan number, blade parameters, ambient temperature and humidity, and the working status of the inspection robot; Fault location distribution map: A panoramic view of the entire blade cavity with superimposed defect locations; Defect supporting information: original infrared image, visible light image, and fused image of each defect point; Detailed defect information: spatial coordinates, type, size, temperature information, and confidence level of the defect; Defect statistics: number of various defects, regional distribution ratio, and development trend analysis.

10. The method for identifying and locating defects in the inner cavity of wind turbine blades based on image fusion according to claim 7, characterized in that, The specific details of S7 are as follows: New defect samples that have been manually reviewed are added to the defect sample library, and the category, characteristics and spatial location information of the defects are supplemented and labeled. By using newly added defect samples, the trained defect detection model is fine-tuned in a lightweight manner, and the weights of the infrared-visible light feature fusion network are updated, enabling the model to learn the features of new defects. The defect identification and location engine operates separately from other parts of the inspection platform in a decoupled manner.

Images