Fan blade online monitoring method and system based on unmanned aerial vehicle inspection

Abstract

The invention discloses a fan blade online monitoring method based on unmanned aerial vehicle inspection, and belongs to the technical field of fan blade inspection. In order to solve the problems of low efficiency and possible existence of a detection blind area of an existing inspection mode, the method is characterized in that a flight path of an unmanned aerial vehicle is planned according to a three-dimensional digital model of a fan blade, and the unmanned aerial vehicle carrying a multispectral camera is controlled to fly along the path and collect image data; after the data are transmitted to a ground server in real time, crack and corrosion characteristics are identified through a convolutional neural network, and a defect alarm is generated. The method is mainly used for automatic defect detection and state monitoring of the surface of the fan blade, and has the capabilities of path planning based on a three-dimensional model, multispectral image fusion identification, breakpoint resume and defect development trend analysis.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade inspection technology. More specifically, this invention relates to an online monitoring method and system for wind turbine blades based on unmanned aerial vehicle (UAV) inspection. Background Technology

[0002] During the operation and maintenance of wind turbine blades, regular inspections are crucial for detecting surface defects. Currently, manual inspection is the most common method, typically requiring technicians to observe from the ground using high-powered telescopes or approach the blades via suspended platforms, ropes, or other equipment. This method suffers from low efficiency; a comprehensive inspection takes a considerable amount of time and is highly dependent on the experience and fatigue of the personnel, potentially leading to missed defects. Furthermore, working at heights carries inherent safety risks and is often impossible under adverse weather conditions.

[0003] In recent years, drone technology has been attempted to be applied to wind turbine blade inspection. In some methods, the drone's flight path relies on preset fixed waypoints or manual remote control. This approach may struggle to ensure complete coverage of all outer surface areas of blades with complex curved surfaces, especially those with significant curvature variations, easily creating blind spots. This is because the three-dimensional geometric features of the blade are not fully considered in the path planning, resulting in insufficient adaptability of the flight path to the blade surface.

[0004] In image acquisition, some applications use only a single type of sensor, such as a visible light camera. Visible light images are less effective at identifying minute defects like cracks in poor lighting conditions or when surface contamination is present. Furthermore, defects like corrosion are not readily apparent under visible light in their early stages, meaning the reliability of defect identification is somewhat dependent on environmental factors and the type of defect.

[0005] In the data transmission and processing stage, drones operating in windy, field environments may experience unstable wireless communication links. Without effective data caching and management mechanisms, data loss may occur during signal interruptions, and subsequent transmission recovery lacks a priority strategy, impacting the timeliness of critical data. Furthermore, if only simple alarms are provided after defect identification without comparison with historical data, it becomes difficult to assess defect trends, hindering maintenance decision-making. Summary of the Invention

[0006] One object of the present invention is to solve at least the above-mentioned problems and to provide at least the advantages that will be described later.

[0007] To achieve these objectives and other advantages according to the present invention, an online monitoring method for wind turbine blades based on unmanned aerial vehicle (UAV) inspection is provided, comprising: Based on the three-dimensional digital model of the wind turbine blade, the flight path of the UAV is calculated by the path planning algorithm. The flight path consists of multiple waypoints in succession, with the horizontal spacing between waypoints between 0.5 and 2m, the flight altitude between 10 and 30m, and the flight path covering the entire outer surface of the wind turbine blade. Control the drone equipped with a multispectral camera to fly along the flight path at a speed of 1 to 5 m / s, and trigger the multispectral camera to collect image data of the wind turbine blades during the flight. The multispectral camera simultaneously collects images in the visible light band and infrared band, and the exposure time for image acquisition is between 1 / 2000 and 1 / 500 s. The collected image data is transmitted to the ground data processing server in real time via the data communication module on the drone. The ground data processing server receives image data and uses a convolutional neural network model to process the image data. The convolutional neural network model is trained to identify crack and corrosion features on the surface of the wind turbine blades. When the convolutional neural network model identifies crack or corrosion features, it generates defect alarm information, which includes the defect location coordinates and defect type. The defect alarm information is then sent to the monitoring system via a network communication interface.

[0008] Preferably, the process of calculating the flight path of the UAV using a path planning algorithm specifically includes: A three-dimensional digital model of the wind turbine blade is obtained, wherein the three-dimensional digital model is a three-dimensional triangular mesh model; Traverse the model from leaf root to leaf tip and calculate the radius of curvature of each local region along the traversal path; The horizontal spacing between waypoints is dynamically adjusted based on the radius of curvature: when the radius of curvature is ≤10 meters, the horizontal spacing between waypoints is adjusted to 0.5~1m; when the radius of curvature is >10m and ≤50m, the horizontal spacing between waypoints is adjusted to 1~2m. A waypoint sequence is generated based on the adjusted waypoint spacing, and the waypoint sequence is smoothed using spline curves to generate the final UAV flight path.

[0009] Preferably, the convolutional neural network model is a multimodal fusion network structure, which extracts features from visible light band images and infrared band images respectively through parallel visible light feature extraction branches and infrared feature extraction branches; The visible light feature extraction branch is used to extract the texture and morphological features of the wind turbine blade surface, and the infrared feature extraction branch is used to extract the abnormal heat distribution features of the wind turbine blade surface. The texture and morphological features are fused with the thermal distribution anomaly features through the feature fusion layer in the network, and the crack features and corrosion features are simultaneously identified by the shared fully connected classification layer based on the fused features.

[0010] Preferably, the specific parameters of the multimodal fusion network structure are as follows: The visible light feature extraction branch adopts the ResNet-18 network structure, and its input image size is 512×512 pixels, containing 3 color channels; The infrared feature extraction branch contains four sequentially connected convolutional blocks. Each convolutional block consists of a convolutional layer with a kernel size of 3×3 pixels, a batch normalization layer, and a ReLU activation function layer. The number of output channels of the four convolutional blocks are 64 channels, 128 channels, 256 channels, and 512 channels, respectively. Each convolutional block is followed by a max pooling layer with a pooling window of 2×2 pixels. The feature fusion layer uses a channel splicing method to splice the 512-channel feature map of the visible light branch with the 512-channel feature map of the infrared branch to form a 1024-channel fused feature map, and then uses a convolutional layer with 512 kernels and a size of 1×1 pixel to perform dimensionality reduction processing on the fused feature map. The shared fully connected classification layer receives a 512-dimensional feature vector obtained after global average pooling, and outputs the recognition results of crack features and corrosion features simultaneously through an output layer containing two neurons and a Sigmoid activation function.

[0011] Preferably, the process of generating defect alarm information specifically includes: Generate structured defect alarm information, which includes defect size, model recognition confidence, and timestamp, in addition to defect location coordinates and defect type. The current defect alert information is correlated with the historical defect database. If a record of the same type of defect exists in the preset historical time period at the current defect location, the alert is marked as an emerging defect. Based on the dimensional change rate of the aforementioned developmental defect, the severity level of the defect is dynamically assessed using a severity calculation model; The complete alarm information, including the severity level of the defect and the development defect marker, will be sent to the monitoring system through the network communication interface according to the priority corresponding to its severity level.

[0012] Preferably, the step of dynamically assessing the severity level of a defect based on its dimensional change rate using a severity calculation model specifically includes: Query the historical defect database for the historical dimensions and timestamps of the current defect at the same location; The size change rate of the defect is calculated based on the historical size and the current defect size, as well as the historical timestamp and the current timestamp. Determine the corresponding deterioration rate coefficient based on the numerical range of the dimensional change rate; The severity index is calculated by multiplying the current defect size by the deterioration rate coefficient. The severity level of the developmental defect is dynamically assessed and output based on the preset threshold range in which the severity index falls.

[0013] Preferably, the process of transmitting the acquired image data to the ground data processing server in real time also includes: The drone has a pre-set data cache area for temporarily storing the collected image data and the corresponding drone GPS coordinates and waypoints at the time of collection; It continuously monitors the signal strength of the wireless communication link, and automatically switches the transmission mode from real-time transmission to local buffering when the signal strength is lower than the preset threshold. When the signal strength is not lower than a preset threshold, the breakpoint resume process is initiated: the flight control system on the UAV communicates with the data buffer and determines whether the waypoint corresponding to the cached image data is located in the preset key area of ​​the wind turbine blade according to the waypoint identifier; the preset key area includes at least the leading edge and root of the blade; image data marked as corresponding to the preset key area is retrieved and transmitted first, and then the remaining cached image data are transmitted in sequence.

[0014] This invention also provides an online monitoring system for wind turbine blades based on unmanned aerial vehicle (UAV) inspection, comprising: The path planning module calculates the UAV's flight path based on the 3D digital model of the wind turbine blades using a path planning algorithm. The flight path consists of multiple waypoints arranged continuously, with horizontal spacing between waypoints ranging from 0.5 to 2 meters and a flight altitude ranging from 10 to 30 meters. The flight path covers the entire outer surface of the wind turbine blades. The flight control and image acquisition module controls the UAV equipped with a multispectral camera to fly along the flight path at a speed of 1 to 5 m / s. During the flight, it triggers the multispectral camera to acquire image data of the wind turbine blades. The multispectral camera simultaneously acquires images in the visible light band and infrared band. The exposure time for image acquisition is between 1 / 2000 and 1 / 500 s. The data communication module, integrated on the UAV, transmits the collected image data to the ground data processing server in real time via wireless communication. The image processing and defect recognition module, which is deployed in the ground data processing server, is used to receive the image data and process the image data using a convolutional neural network model, wherein the convolutional neural network model is trained to identify crack features and corrosion features on the surface of the wind turbine blades. The defect alarm and reporting module is used to generate defect alarm information when the convolutional neural network model identifies crack or corrosion features. The defect alarm information includes the defect location coordinates and defect type, and is sent to the monitoring system through the network communication interface.

[0015] This invention offers at least the following advantages: By using path planning based on a three-dimensional digital model, it can generate flight paths covering the entire outer surface of wind turbine blades. This planning method ensures comprehensive inspection without blind spots, and dynamically adjusts waypoint spacing according to blade curvature, improving inspection efficiency while maintaining acquisition accuracy. Simultaneous acquisition of visible and infrared images using a multispectral camera provides complementary data sources for defect identification. Joint analysis of the two types of image features through a multimodal fusion neural network comprehensively utilizes texture morphology and thermal distribution information, improving the accuracy of identifying defects such as cracks and corrosion. The system features data caching and breakpoint resume mechanisms, ensuring data security even under poor communication conditions. When a defect is identified, the system not only generates detailed structured alarms but also assesses the defect development trend by comparing historical data, providing a quantitative basis for maintenance decisions and supporting more targeted maintenance arrangements.

[0016] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0017] Figure 1 This is a flowchart of the online monitoring method for wind turbine blades based on UAV inspection as described in this invention; Figure 2 This is a schematic diagram of the structure of the wind turbine blade online monitoring system based on UAV inspection as described in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0019] like Figure 1 As shown, this invention provides an online monitoring method for wind turbine blades based on UAV inspection, comprising: S1. Based on the three-dimensional digital model of the wind turbine blade, the flight path of the UAV is calculated through the path planning algorithm. The flight path consists of multiple waypoints in succession. The horizontal spacing between waypoints is between 0.5 and 2m. This range is set based on the common size of wind turbine blades and the resolution of multispectral cameras, taking into account both coverage efficiency and image clarity. The flight altitude is between 10 and 30m. This altitude range is suitable for the length of typical wind turbine blades, avoiding the risk of collision if too close and affecting image quality if too far. The flight path covers the entire outer surface of the wind turbine blade. Specifically, flight path planning is based on a 3D digital model of the wind turbine blades, which can be a 3D triangular mesh model obtained through a laser scanner. The path planning algorithm can employ the A* algorithm or the Fast Random Tree algorithm. The horizontal spacing between waypoints can be selected as 0.5 m, 1 m, or 2 m, and the flight altitude can be selected as 10 m, 20 m, or 30 m to ensure coverage of the entire outer surface of the blades. The UAV can be equipped with a multispectral camera, which can include visible light and infrared sensors. The flight speed can be selected as 1 m / s, 3 m / s, or 5 m / s, and the exposure time can be selected as 1 / 2000 s, 1 / 1000 s, or 1 / 500 s. The UAV fuselage can be made of carbon fiber composite material to reduce weight. The flight control module and GPS module are installed inside the UAV fuselage for precise navigation and positioning.

[0020] S2. Control the drone equipped with a multispectral camera to fly along the flight path at a speed of 1~5m / s, and trigger the multispectral camera to collect image data of the wind turbine blades during the flight. The multispectral camera simultaneously collects visible light band images and infrared band images, and the exposure time for image acquisition is between 1 / 2000 and 1 / 500s. Specifically, during the operation, a 3D model is first loaded. The algorithm generates a sequence of waypoints based on the model's curvature and smooths it using spline curves to form a continuous path. Subsequently, the UAV flies along the path, triggering the camera to acquire images at designated waypoints. Parameter settings are completed through ground control software, adjusting the waypoint spacing and altitude according to the blade size and environmental conditions.

[0021] S3. The collected image data is transmitted to the ground data processing server in real time through the data communication module on the UAV; Specifically, the data communication module can be integrated onto the UAV, employing a 4G or 5G wireless communication module to transmit image data to a ground data processing server in real time. The UAV has a data buffer, which can use a solid-state drive as storage media, to temporarily store image data along with its corresponding GPS coordinates and waypoints. The signal strength threshold for the wireless communication link is set to -90 dBm. When the signal strength falls below this value, it automatically switches to local buffer mode; when the signal strength is not lower than this value, it initiates a breakpoint resume transmission process, prioritizing the transmission of image data from critical areas such as the leading edge and root of the blades. The communication module is mounted on the top of the UAV fuselage to optimize signal reception. During operation, the UAV continuously monitors the signal strength and adjusts the transmission mode according to preset logic. The parameter setting method is based on on-site communication testing to determine the threshold and transmission priority.

[0022] S4. The ground data processing server receives image data and processes it using a convolutional neural network model. The convolutional neural network model is trained to identify crack and corrosion features on the surface of the wind turbine blades. Specifically, the server is installed in the wind farm control room and interacts with the monitoring system via a wired connection. The ground data processing server can be equipped with a multi-core processor and a GPU accelerator card to run a convolutional neural network model, which can be trained based on the TensorFlow or PyTorch framework to identify crack and corrosion features.

[0023] S5. When the convolutional neural network model identifies crack or corrosion features, it generates defect alarm information, which includes the defect location coordinates and defect type. The defect alarm information is then sent to the monitoring system via the network communication interface.

[0024] Specifically, when the server identifies a defect, it generates structured alert information, including the defect's location coordinates, type, size, confidence level, and timestamp. The network communication interface can use an Ethernet or Wi-Fi module to send the alert to the monitoring system.

[0025] During operation, the server receives image data, invokes a neural network model for inference, and queries against a historical defect database to mark progressive defects. The severity calculation model assesses the defect level based on the rate of size change. For example, the threshold for the rate of size change is set to 0.1 mm / d, and the rate of deterioration coefficient is set to 1.0 to 2.0 based on the range of the rate of change. The severity index is obtained by multiplying the current size by the coefficient, and finally, the severity level is output according to the threshold range. Parameter settings are optimized through historical data analysis and machine learning.

[0026] This method generates flight paths covering the entire outer surface of wind turbine blades through path planning based on a 3D digital model of the blades. This precise path planning ensures thorough inspection without blind spots, avoiding areas that may be missed during manual inspections, thereby improving the comprehensiveness of the inspection and the integrity of the data collection.

[0027] This method automates data acquisition by controlling a drone equipped with a multispectral camera to fly along a predetermined path and collect images. The simultaneous acquisition of visible and infrared images provides a rich data source for subsequent analysis. This automated acquisition method reduces reliance on manual inspections from elevated positions, helping to lower operational risks and shorten inspection time.

[0028] By utilizing the data communication module on the drone to achieve real-time transmission of image data, on-site data can be quickly sent to a ground server for processing. This process reduces data latency, creating conditions for timely detection of potential hazards. The real-time transmission characteristic makes the monitoring process more timely.

[0029] By using a trained convolutional neural network model to analyze images on a ground-based data processing server, cracks and corrosion features on the blade surface can be automatically identified. This intelligent identification method reduces reliance on manual interpretation and helps improve the objectivity and consistency of defect identification.

[0030] When a defect is identified, the system automatically generates an alarm message containing the location and type of defect and sends it via the network interface. This automated process enables a rapid response from problem detection to early warning reporting, providing timely information support for subsequent maintenance decisions and facilitating the scheduling of targeted repairs.

[0031] In another embodiment, the process of calculating the flight path of the UAV using a path planning algorithm specifically includes: A three-dimensional digital model of the wind turbine blade is obtained, wherein the three-dimensional digital model is a three-dimensional triangular mesh model; Traverse the model from leaf root to leaf tip and calculate the radius of curvature of each local region along the traversal path; Based on the radius of curvature, the horizontal spacing of waypoints is dynamically adjusted: when the radius of curvature is ≤10 meters, the horizontal spacing of waypoints is adjusted to 0.5~1m; when the radius of curvature is >10m and ≤50m, the horizontal spacing of waypoints is adjusted to 1~2m. The above radius of curvature thresholds are determined based on the statistical analysis of typical surface changes of wind turbine blades and the flight stability test of UAVs to ensure the accuracy of image acquisition in high curvature areas. A waypoint sequence is generated based on the adjusted waypoint spacing, and the waypoint sequence is smoothed using spline curves to generate the final UAV flight path.

[0032] Specifically, the 3D digital model can be a 3D triangular mesh model of the wind turbine blade obtained through a laser scanner or photogrammetry system. The mesh side length of this model can be selected between 0.1 m and 0.5 m. The model data can be stored in the computer of the ground control station. The computer processor can be a multi-core CPU, and the memory capacity can be selected as 16 GB or 32 GB. During operation, the path planning algorithm traverses the 3D model from the blade root towards the blade tip. The traversal method can be either depth-first search or breadth-first search, and the traversal step size can be selected as 0.2 m, 0.5 m, or 1 m. During the traversal, the system records the center point coordinates and normal vector information of each local area. Parameter settings are completed through professional 3D processing software, and the traversal step size is determined according to the actual size of the blade.

[0033] The radius of curvature can be calculated using a curvature estimation algorithm based on a triangular mesh. The search radius involved in the calculation can be selected as 0.5 m, 1 m, or 2 m. The calculation module can be integrated into the path planning software of the ground control station and run on the aforementioned computer platform. When the radius of curvature is ≤10 m, the horizontal waypoint spacing is adjusted to a value between 0.5 and 1 m, which can be selected as 0.5 m, 0.8 m, or 1 m; when the radius of curvature is >10 m and ≤50 m, the waypoint spacing is adjusted to a value between 1 and 2 m, which can be selected as 1 m, 1.5 m, or 2 m. During operation, the algorithm calculates the average radius of curvature for each traversed local area and automatically selects the corresponding waypoint spacing value based on preset threshold conditions. The parameter setting method is determined based on the statistical analysis results of the curvature of different parts of the wind turbine blades.

[0034] An initial waypoint sequence is generated based on the adjusted waypoint spacing, with each waypoint containing three-dimensional coordinate information. Smoothing can be performed using cubic spline or B-spline curve algorithms, with a smoothing factor of 0.1, 0.2, or 0.3. The path planning software can run on a ground control station, and the generated final flight path file can be transferred to the UAV's flight controller. The flight controller can be an off-the-shelf product integrating IMU and GPS modules, installed inside the UAV fuselage. During operation, the smoothing algorithm reads the waypoint sequence and inserts interpolation points within each waypoint segment to form a continuous and smooth flight path. Parameter settings are determined through multiple simulation experiments to ensure a balance between path smoothness and flight stability.

[0035] The path planning method described above, by obtaining a three-dimensional triangular mesh model of the wind turbine blade, can accurately reflect the actual geometry of the blade. The systematic traversal from the blade root to the blade tip ensures the integrity of the model data, providing an accurate spatial information foundation for subsequent path calculations.

[0036] This method calculates the radius of curvature of each local region along the traversal path and dynamically adjusts the horizontal spacing of waypoints accordingly. Smaller waypoint spacing is used in regions with smaller radii of curvature, while larger spacing is used in regions with gentler curvature. This adaptive adjustment mechanism ensures that the generated flight path closely conforms to the geometric features of the blade surface. After generating a waypoint sequence based on the adjusted waypoint spacing, spline curves are used for smoothing. This step connects discrete waypoints into a continuous and smooth flight path, helping to reduce sharp turns and jerks during UAV flight. The final path considers both the requirements of data acquisition accuracy and the stability and efficiency of flight.

[0037] In another embodiment, the convolutional neural network model is a multimodal fusion network structure, which extracts features from visible light band images and infrared band images respectively through parallel visible light feature extraction branches and infrared feature extraction branches. The visible light feature extraction branch is used to extract the texture and morphological features of the wind turbine blade surface, and the infrared feature extraction branch is used to extract the abnormal heat distribution features of the wind turbine blade surface. The texture and morphological features are fused with the thermal distribution anomaly features through the feature fusion layer in the network, and the crack features and corrosion features are simultaneously identified by the shared fully connected classification layer based on the fused features.

[0038] Specifically, the multimodal fusion network structure can be deployed on a ground data processing server, which can be equipped with a multi-core central processing unit and a graphics processing unit.

[0039] The visible light feature extraction branch can be based on the ResNet-18 network structure, and its input image size can be selected as 512×512 pixels, containing 3 color channels; The infrared feature extraction branch may contain four sequentially connected convolutional blocks. Each convolutional block consists of a convolutional layer with a kernel size of 3×3 pixels, a batch normalization layer, and a ReLU activation function layer. The number of output channels of the four convolutional blocks are 64 channels, 128 channels, 256 channels, and 512 channels, respectively. Each convolutional block is followed by a max pooling layer with a pooling window of 2×2 pixels.

[0040] The network model can be implemented using deep learning frameworks such as TensorFlow or PyTorch, and readily available commercial server platforms can be used as server hardware. During operation, visible light and infrared images are simultaneously input into their respective feature extraction branches for parallel computation. Parameter settings are completed during the training phase using the stochastic gradient descent algorithm, with a learning rate that can be selected as 0.001 or 0.0001.

[0041] The visible light feature extraction branch and the infrared feature extraction branch can run in the server's GPU memory, and the GPU can be any readily available computing card model. During the operation, each branch independently performs convolution and pooling operations on the input image to generate high-dimensional feature maps. Parameter setting methods include using batch normalization layers to stabilize the training process, with batch sizes selectable as 32 or 64.

[0042] The feature fusion layer can use a channel splicing method to splice the 512-channel feature map of the visible light branch with the 512-channel feature map of the infrared branch to form a 1024-channel fused feature map, and then use a convolutional layer with 512 kernels and a size of 1×1 pixel to perform dimensionality reduction processing on the fused feature map. The shared fully connected classification layer receives a 512-dimensional feature vector obtained after global average pooling, and outputs the recognition results of crack features and corrosion features simultaneously through an output layer containing two neurons and a Sigmoid activation function.

[0043] The classification layer can be integrated into the server's neural network inference module. During operation, the fused feature maps are converted into feature vectors through global average pooling and then input into the fully connected layer for binary classification decisions. Parameter settings are achieved by minimizing the binary cross-entropy loss function, and the number of iterations can be selected as 100 or 200 rounds.

[0044] The described multimodal fusion network structure, through parallel visible light and infrared feature extraction branches, can simultaneously utilize the information advantages of two different spectral bands of images. Visible light images provide texture and morphological details of the blade surface, while infrared images reflect its thermal distribution characteristics. The combination of these two types of information provides a more comprehensive data foundation for defect identification.

[0045] The feature fusion layer in this network structure effectively integrates features from different branches, enabling the model to comprehensively judge crack and corrosion defects. The visible light branch uses a ResNet-18 network with good feature extraction capabilities, while the four convolutional blocks in the infrared branch focus on extracting thermal anomaly features. This collaborative processing approach helps capture different types of defect features.

[0046] After feature fusion and dimensionality reduction, the shared fully connected classification layer synchronously outputs the identification results of cracks and corrosion based on the fused feature vectors. This multi-task learning architecture enables the network to detect two types of defects in a single inference process, improving identification efficiency. The overall network structure design considers the balance between computational complexity and recognition accuracy, contributing to stable and reliable defect identification results in practical application environments.

[0047] In another embodiment, the process of generating defect alarm information specifically includes: Generate structured defect alarm information, which includes defect size, model recognition confidence, and timestamp, in addition to defect location coordinates and defect type. The current defect alert information is correlated with the historical defect database. If a record of the same type of defect exists in the preset historical time period at the current defect location, the alert is marked as an emerging defect. Based on the dimensional change rate of the aforementioned developmental defect, the severity level of the defect is dynamically assessed using a severity calculation model; The complete alarm information, including the severity level of the defect and the development defect marker, will be sent to the monitoring system through the network communication interface according to the priority corresponding to its severity level.

[0048] Specifically, defect alarm information can include defect location coordinates, defect type, defect size, model recognition confidence level, and timestamp. Defect location coordinates can use a geodetic coordinate system or a local coordinate system, with an accuracy of 0.1 m to 1 m. Defect size can record parameters such as length and width, with units selectable in millimeters or centimeters. The model recognition confidence threshold can be set to 0.7; data below this value requires manual review. The timestamp can use a Unix timestamp or ISO 8601 format. This information can be generated on a ground data processing server, which can be equipped with a database management system and a clock synchronization module. Preset historical time periods, such as 30 days, 60 days, or 90 days, can be configured according to the actual maintenance cycle. During operation, when the convolutional neural network identifies a defect, the system automatically extracts the above information and encapsulates it into structured data objects. Parameter settings are managed through configuration files, allowing adjustment of the accuracy requirements of each field according to actual application needs.

[0049] The severity calculation model can be built based on linear regression or exponential growth models. The dimensional change rate can be calculated using absolute or relative change rates, with time intervals in days. Severity levels can be divided into 3 or 5 levels, and priority relationships can be determined using a lookup table. The alarm sending module can be integrated into the network communication interface, using message queues or instant messaging protocols. The network communication interface can be an Ethernet card or a wireless communication module, installed in the server's expansion slot. During operation, the system first calculates the dimensional change rate, then determines the deterioration rate coefficient according to preset rules, with the coefficient range selectable between 1.0 and 3.0. The severity index is obtained by multiplying the current defect size by the deterioration rate coefficient, and the severity level is finally determined based on the threshold range of the index value. Parameter settings are completed through historical data analysis and logarithmic normalization.

[0050] Furthermore, the dynamic assessment of the defect severity level based on the dimensional change rate of the developmental defect using a severity calculation model specifically includes: Query the historical defect database for the historical dimensions and timestamps of the current defect at the same location; The size change rate of the defect is calculated based on the historical size and the current defect size, as well as the historical timestamp and the current timestamp. Determine the corresponding deterioration rate coefficient based on the numerical range of the dimensional change rate; The severity index is calculated by multiplying the current defect size by the deterioration rate coefficient. The severity level of the developmental defect is dynamically assessed and output based on the preset threshold range in which the severity index falls.

[0051] More specifically, the historical defect database can be a relational database or a time-series database, deployed on the storage system of a ground server. The preset historical time period can be 30 days, 60 days, or 90 days. During the query process, the system spatially matches the current defect location coordinates with historical records, with a matching radius set to 0.5 m. When the same type of defect is found at the same location, the system automatically marks the alarm as a developing defect and establishes a related record in the database. The database server can be an existing enterprise-grade storage device, equipped with redundant power supplies and a cooling system. During operation, the query algorithm first performs spatial index retrieval, then time range filtering, and finally defect type matching. Parameter settings are determined based on statistical analysis of the typical defect development cycle of wind turbine blades.

[0052] By correlating current defects with historical database queries, this method can identify defects of the same type that repeatedly occur in the same location within a preset time period. When such defects are detected, the system marks them as evolving defects, which helps track the evolution of defects and provides data support for predictive maintenance.

[0053] Based on the dimensional change rate of evolving defects, the system can dynamically assess the severity level of defects. By calculating the change rate between historical and current dimensions and combining it with a preset deterioration rate coefficient, a severity index reflecting the degree of danger of the defect is derived. This quantitative assessment method provides a basis for prioritizing maintenance work, allowing limited maintenance resources to be used to address the most urgent defects.

[0054] In another embodiment, the process of transmitting the acquired image data to the ground data processing server in real time also includes: The drone has a pre-set data cache area for temporarily storing the collected image data and the corresponding drone GPS coordinates and waypoints at the time of collection; It continuously monitors the signal strength of the wireless communication link, and automatically switches the transmission mode from real-time transmission to local buffering when the signal strength is lower than the preset threshold. When the signal strength is not lower than a preset threshold, the breakpoint resume process is initiated: the flight control system on the UAV communicates with the data buffer and determines whether the waypoint corresponding to the cached image data is located in the preset key area of ​​the wind turbine blade according to the waypoint identifier; the preset key area includes at least the leading edge and root of the blade; image data marked as corresponding to the preset key area is retrieved and transmitted first, and then the remaining cached image data are transmitted in sequence.

[0055] Specifically, the data cache can use a solid-state drive or an embedded multimedia card as the storage medium, with storage capacities of 64GB, 128GB, or 256GB. The cache is used to temporarily store acquired image data and its corresponding UAV GPS coordinates and waypoints at the time of acquisition. The GPS coordinate accuracy can reach 1m to 3m, and the waypoints can use digital sequence encoding. The storage medium can be installed on a shock-resistant bracket inside the UAV fuselage and connected to the flight control system via a data interface. During operation, the acquired image data is first written to the cache, along with the corresponding location information and timestamp. Parameter settings are configured through the flight control software, specifying the cache size and data storage format.

[0056] In the implementation of signal monitoring and transmission mode switching, the system continuously monitors the signal strength of the wireless communication link. The monitoring frequency can be selected as 1, 2, or 5 times per second. The signal strength threshold can be set to -85dBm, -90dBm, or -95dBm. When the signal strength is lower than the set threshold (e.g., -90dBm), the system automatically switches the transmission mode from real-time transmission to local buffering. The monitoring module can be integrated into the drone's wireless communication module, which can be a readily available 4G or 5G communication module, installed inside the antenna dome on the top of the drone's fuselage. During operation, the signal strength monitoring circuit collects the received signal strength indication value in real time and compares it with the preset threshold, controlling the data transmission path switching based on the comparison result. The parameter settings are determined based on on-site communication environment testing, considering the signal attenuation characteristics under different distances and obstacle conditions.

[0057] In the implementation of breakpoint resume and priority transmission, the system initiates the breakpoint resume process when the signal strength is not lower than a preset threshold. The flight control system communicates with the data buffer, determining whether the area corresponding to the buffered data belongs to a preset critical area based on waypoint identifiers. The critical area includes at least the leading edge and root of the blades. During transmission, image data from the critical area is retrieved and transmitted first, followed by the transmission of the remaining buffered data in sequence. The flight control system can use an existing autopilot installed in the UAV control cabin. During operation, the system first scans the waypoint identifiers in the buffer, matches them with the preset critical area waypoint list, and then organizes the data transmission sequence according to the principle of prioritizing critical areas. Parameter settings are configured through ground station software to define the range and priority rules of the critical areas.

[0058] By pre-setting a data buffer on the drone's end, the collected image data and its corresponding location information can be temporarily stored locally on the device. This measure provides a buffer for data transmission, effectively preventing data loss and ensuring the integrity of the collected data in the event of communication link instability.

[0059] By continuously monitoring the signal strength of the wireless communication link and automatically switching to local buffering mode when the signal strength falls below a preset threshold, this system can adapt to complex field communication environments. This strategy of dynamically adjusting the transmission mechanism ensures that data acquisition is not interrupted by temporary signal attenuation, thus improving the system's continuous operation capability under harsh communication conditions.

[0060] When communication signals recover to a usable level, the system's breakpoint resume transmission process intelligently prioritizes data based on waypoint markers. This process prioritizes transmitting image data of critical areas such as the leading edge and root of the blades, ensuring timely processing of important information. This differentiated transmission strategy helps optimize bandwidth utilization and improves the efficiency of detecting and responding to critical defects.

[0061] Based on the same inventive concept, this invention also provides an online monitoring system for wind turbine blades based on drone inspection. This online monitoring system can be a personal computer, a server, or other device that implements the aforementioned digital method for detecting duplicate clothing styles. Please see below. Figure 2 As shown, the wind turbine blade online monitoring system based on UAV inspection provided by the present invention includes: The path planning module calculates the UAV's flight path based on the 3D digital model of the wind turbine blades using a path planning algorithm. The flight path consists of multiple waypoints arranged continuously, with horizontal spacing between waypoints ranging from 0.5 to 2 meters and a flight altitude ranging from 10 to 30 meters. The flight path covers the entire outer surface of the wind turbine blades. The flight control and image acquisition module controls the UAV equipped with a multispectral camera to fly along the flight path at a speed of 1 to 5 m / s. During the flight, it triggers the multispectral camera to acquire image data of the wind turbine blades. The multispectral camera simultaneously acquires images in the visible light band and infrared band. The exposure time for image acquisition is between 1 / 2000 and 1 / 500 s. The data communication module, integrated on the UAV, transmits the collected image data to the ground data processing server in real time via wireless communication. The image processing and defect recognition module, which is deployed in the ground data processing server, is used to receive the image data and process the image data using a convolutional neural network model, wherein the convolutional neural network model is trained to identify crack features and corrosion features on the surface of the wind turbine blades. The defect alarm and reporting module is used to generate defect alarm information when the convolutional neural network model identifies crack or corrosion features. The defect alarm information includes the defect location coordinates and defect type, and is sent to the monitoring system through the network communication interface.

[0062] All relevant content of each step involved in the aforementioned embodiments of the wind turbine blade online monitoring method based on UAV inspection can be referenced to the functional description of the corresponding functional module of the wind turbine blade online monitoring system based on UAV inspection in the embodiments of this application, and will not be repeated here.

[0063] The module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0064] In the accompanying drawings of the system embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0065] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for online monitoring of wind turbine blades based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: Based on the three-dimensional digital model of the wind turbine blade, the flight path of the UAV is calculated by the path planning algorithm. The flight path consists of multiple waypoints in succession, with the horizontal spacing between waypoints between 0.5 and 2m, the flight altitude between 10 and 30m, and the flight path covering the entire outer surface of the wind turbine blade. Control the drone equipped with a multispectral camera to fly along the flight path at a speed of 1 to 5 m / s, and trigger the multispectral camera to collect image data of the wind turbine blades during the flight. The multispectral camera simultaneously collects images in the visible light band and infrared band, and the exposure time for image acquisition is between 1 / 2000 and 1 / 500 s. The collected image data is transmitted to the ground data processing server in real time via the data communication module on the drone. The ground data processing server receives image data and uses a convolutional neural network model to process the image data. The convolutional neural network model is trained to identify crack and corrosion features on the surface of the wind turbine blades. When the convolutional neural network model identifies crack or corrosion features, it generates defect alarm information, which includes the defect location coordinates and defect type. The defect alarm information is then sent to the monitoring system via a network communication interface.

2. The online monitoring method for wind turbine blades as described in claim 1, characterized in that, The process of calculating the flight path of the UAV using a path planning algorithm specifically includes: A three-dimensional digital model of the wind turbine blade is obtained, wherein the three-dimensional digital model is a three-dimensional triangular mesh model; Traverse the model from leaf root to leaf tip and calculate the radius of curvature of each local region along the traversal path; The horizontal spacing between waypoints is dynamically adjusted based on the radius of curvature: when the radius of curvature is ≤10 meters, the horizontal spacing between waypoints is adjusted to 0.5~1m; when the radius of curvature is >10m and ≤50m, the horizontal spacing between waypoints is adjusted to 1~2m. A waypoint sequence is generated based on the adjusted waypoint spacing, and the waypoint sequence is smoothed using spline curves to generate the final UAV flight path.

3. The online monitoring method for wind turbine blades as described in claim 1, characterized in that, The convolutional neural network model is a multimodal fusion network structure, which extracts features from visible light band images and infrared band images respectively through parallel visible light feature extraction branches and infrared feature extraction branches. The visible light feature extraction branch is used to extract the texture and morphological features of the wind turbine blade surface, and the infrared feature extraction branch is used to extract the abnormal heat distribution features of the wind turbine blade surface. The texture and morphological features are fused with the thermal distribution anomaly features through the feature fusion layer in the network, and the crack features and corrosion features are simultaneously identified by the shared fully connected classification layer based on the fused features.

4. The online monitoring method for wind turbine blades as described in claim 3, characterized in that, The specific parameters of the multimodal fusion network structure are as follows: The visible light feature extraction branch adopts the ResNet-18 network structure, and its input image size is 512×512 pixels, containing 3 color channels; The infrared feature extraction branch contains four sequentially connected convolutional blocks. Each convolutional block consists of a convolutional layer with a kernel size of 3×3 pixels, a batch normalization layer, and a ReLU activation function layer. The number of output channels of the four convolutional blocks are 64 channels, 128 channels, 256 channels, and 512 channels, respectively. Each convolutional block is followed by a max pooling layer with a pooling window of 2×2 pixels. The feature fusion layer uses a channel splicing method to splice the 512-channel feature map of the visible light branch with the 512-channel feature map of the infrared branch to form a 1024-channel fused feature map, and then uses a convolutional layer with 512 kernels and a size of 1×1 pixel to perform dimensionality reduction processing on the fused feature map. The shared fully connected classification layer receives a 512-dimensional feature vector obtained after global average pooling, and outputs the recognition results of crack features and corrosion features simultaneously through an output layer containing two neurons and a Sigmoid activation function.

5. The online monitoring method for wind turbine blades as described in claim 1, characterized in that, The process of generating defect alarm information specifically includes: Generate structured defect alarm information, which includes defect size, model recognition confidence, and timestamp, in addition to defect location coordinates and defect type. The current defect alert information is correlated with the historical defect database. If a record of the same type of defect exists in the preset historical time period at the current defect location, the alert is marked as an emerging defect. Based on the dimensional change rate of the aforementioned developmental defect, the severity level of the defect is dynamically assessed using a severity calculation model; The complete alarm information, including the severity level of the defect and the development defect marker, will be sent to the monitoring system through the network communication interface according to the priority corresponding to its severity level.

6. The online monitoring method for wind turbine blades as described in claim 5, characterized in that, The method of dynamically assessing the severity level of a defect based on its dimensional change rate using a severity calculation model specifically includes: Query the historical defect database for the historical dimensions and timestamps of the current defect at the same location; The size change rate of the defect is calculated based on the historical size and the current defect size, as well as the historical timestamp and the current timestamp. Determine the corresponding deterioration rate coefficient based on the numerical range of the dimensional change rate; The severity index is calculated by multiplying the current defect size by the deterioration rate coefficient. The severity level of the developmental defect is dynamically assessed and output based on the preset threshold range in which the severity index falls.

7. The online monitoring method for wind turbine blades as described in claim 1, characterized in that, The process of transmitting the acquired image data to the ground data processing server in real time also includes: The drone has a pre-set data cache area for temporarily storing the collected image data and the corresponding drone GPS coordinates and waypoints at the time of collection; It continuously monitors the signal strength of the wireless communication link, and automatically switches the transmission mode from real-time transmission to local buffering when the signal strength is lower than the preset threshold. When the signal strength is not lower than a preset threshold, the breakpoint resume process is initiated: the flight control system on the UAV communicates with the data buffer and determines whether the waypoint corresponding to the cached image data is located in the preset key area of ​​the wind turbine blade according to the waypoint identifier; the preset key area includes at least the leading edge and root of the blade; image data marked as corresponding to the preset key area is retrieved and transmitted first, and then the remaining cached image data are transmitted in sequence.

8. An online monitoring system for wind turbine blades based on unmanned aerial vehicle (UAV) inspection, characterized in that, include: The path planning module calculates the UAV's flight path based on the 3D digital model of the wind turbine blades using a path planning algorithm. The flight path consists of multiple waypoints arranged continuously, with horizontal spacing between waypoints ranging from 0.5 to 2 meters and a flight altitude ranging from 10 to 30 meters. The flight path covers the entire outer surface of the wind turbine blades. The flight control and image acquisition module controls the UAV equipped with a multispectral camera to fly along the flight path at a speed of 1 to 5 m / s. During the flight, it triggers the multispectral camera to acquire image data of the wind turbine blades. The multispectral camera simultaneously acquires images in the visible light band and infrared band. The exposure time for image acquisition is between 1 / 2000 and 1 / 500 s. The data communication module, integrated on the UAV, transmits the collected image data to the ground data processing server in real time via wireless communication. The image processing and defect recognition module, which is deployed in the ground data processing server, is used to receive the image data and process the image data using a convolutional neural network model, wherein the convolutional neural network model is trained to identify crack features and corrosion features on the surface of the wind turbine blades. The defect alarm and reporting module is used to generate defect alarm information when the convolutional neural network model identifies crack or corrosion features. The defect alarm information includes the defect location coordinates and defect type, and is sent to the monitoring system through the network communication interface.

Images