A mobile vision-based method and system for monitoring the structural health of a wind turbine blade in service

Abstract

The application discloses a kind of in-service fan blade structure health monitoring method and system based on mobile vision, belong to fan blade monitoring technical field.The application maintains fan rotating state, collects multi-source time series information and unifies time stamp;Through multi-view calibration, coordinate conversion matrix is solved;Develop three-dimensional feature point registration and filter effective matching point pair;Through rotation angle compensation, correct image distortion and separate vibration pseudo deformation, construct integrated dataset;Solve full-field displacement field and strain field, generate space-time deformation evolution dataset;Through low-rank decomposition and mechanics constraint, deduce damage trend;Through mechanism and data hybrid drive multidimensional discrimination damage state.The application can solve the problem that contact sensing coverage is limited, fixed vision is difficult to adapt to rotating fan, unmanned aerial vehicle vision is susceptible to vibration interference and single method generalization is poor, and can realize blade full-surface high-precision deformation measurement and early damage accurate identification under the condition that fan does not stop.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade monitoring technology, and more specifically to a method and system for monitoring the structural health of in-service wind turbine blades based on mobile vision. Background Technology

[0002] Currently, wind turbine blades are the core components of wind turbine generators for capturing wind energy. They are characterized by their lightweight, high flexibility, and low damping. During their service life, they are subjected to alternating loads and harsh environmental corrosion, making them highly susceptible to localized damage such as cracks, delamination, and leading-edge corrosion. Failure to detect these damages in time can lead to blade breakage or even the destruction of the entire turbine. Existing technologies for monitoring the structural health of wind turbine blades mainly employ two methods: contact sensing and fixed visual monitoring. Contact sensing collects response signals by installing vibration accelerometers or fiber optic sensors at key locations on the blade, offering high accuracy. Fixed visual monitoring utilizes high-speed cameras mounted on the ground, based on digital image correlation principles, to acquire full-field deformation information of the blade, offering advantages such as non-contact operation and high spatial resolution.

[0003] However, both of these sensing methods face significant technical hurdles. Contact sensing, limited by the aerodynamic shape and manufacturing processes of the blades, can only be deployed in localized areas like the blade root, failing to cover the blade tip and leading and trailing edges. Furthermore, the number of sensors deployed is limited, resulting in poor technical and economic efficiency. Fixed visual monitoring heavily relies on fixed reference coordinates, but in actual operation, wind turbine blades rotate continuously, and the nacelle yaws with the wind direction, keeping the optical axis in a non-orthogonal state. Moreover, mountainous or deep-sea wind farms lack stable platforms for mounting fixed cameras, leading to significant field-of-view obstruction. While some scholars have attempted to combine drones with cameras to construct mobile visual monitoring systems in recent years, the mechanical vibrations and real-time drift of the drones cause constant changes in the camera's extrinsic parameter matrix, rendering existing joint calibration methods, image registration criteria, and stereo vision reconstruction algorithms based on fixed references unsuitable for direct application.

[0004] Furthermore, existing methods also have significant shortcomings in damage identification. One type is the inversion method based on the finite element model, which uses iterative correction of model parameters to make the calculated response approximate the measured data, thereby inferring damage information. However, the nonlinear vibration characteristics of large flexible blades make it extremely difficult to establish a high-precision mechanical model, and model errors often mask subtle feature changes caused by damage. Another type is the data-driven identification method, which directly performs statistical analysis or deep learning on monitoring data to identify damage. Although this avoids the modeling difficulties, as a black box model, it heavily relies on the completeness of the training data, has insufficient generalization ability in variable environments, and lacks interpretability of identification results due to the lack of mechanical mechanism constraints, making it difficult to trace the cause of damage. Both types of methods fail to fully utilize the structural response information contained in the spatiotemporal evolution of blade geometry, and cannot effectively establish a deterministic correlation between "morphological changes" and "local damage".

[0005] Therefore, how to propose a method and system for monitoring the structural health of in-service wind turbine blades based on mobile vision, so as to achieve high-fidelity acquisition of the geometric morphology of the entire blade surface and accurate identification of potential damage, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for monitoring the structural health of in-service wind turbine blades based on mobile vision, which realizes high-precision measurement of three-dimensional deformation of the entire surface of the blades under the condition of grid-connected rotation without stopping the wind turbine. It integrates photogrammetry, thin shell mechanics, time series decomposition, and physical information deep learning to accurately identify the location, type and severity of blade damage, and predict the long-term evolution trend of damage.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: On the one hand, this invention proposes a method for monitoring the structural health of in-service wind turbine blades based on mobile vision, comprising: S1. Maintain the wind turbine in grid-connected rotation state, collect multi-source time-series information including blade movement visual spatiotemporal sequence data and operating condition time-series information, and unify the timestamp of the multi-source time-series information; S2. Solve the coordinate transformation matrix between multiple coordinate systems by performing multi-view calibration calculations using a standard template; S3. Combine coordinate transformation matrix, spatiotemporal sequence data with unified timestamp and working condition time series information to carry out three-dimensional feature point registration of rotating blades, and evaluate and screen effective feature matching point pairs based on the arrangement constraints of the annular template; S4. Calculate the rotation offset of each frame based on the selected valid point pairs, iteratively correct image distortion through rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporal continuous correlation features of blade geometry. S5. Based on the integrated dataset, calculate the full-field displacement field and strain field of the blade, realize the three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade; S6. Perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to deduce the blade damage development trend; S7. By integrating spatiotemporal deformation evolution data with damage evolution trends, mechanical constraints are constructed through a hybrid mechanism and data-driven approach to determine the local damage state of the blade from multiple dimensions.

[0008] Preferably, S2 includes: Using multiple non-collinear reflective markers pre-set at the leaf root as standard templates, the spatial coordinates of each marker in the world coordinate system are determined, and a local coordinate system of the leaf with the center of the leaf root as the origin is established. Collect the image coordinates of the marker points in multi-view images, and solve the extrinsic parameter matrix of each camera by combining the UAV pose data; With the goal of minimizing the reprojection error between the spatial coordinates of the marker points and the image coordinates, global optimization is performed on the extrinsic parameter matrix and the camera intrinsic parameters to obtain the coordinate transformation matrix between multiple coordinate systems.

[0009] Preferably, S3 includes: By combining the coordinate transformation matrix, multi-view images are mapped to the local coordinate system of the blade, and feature points on the blade surface are extracted and the correspondence between feature points in adjacent frames is established. The screening threshold is dynamically constructed using the statistical mean and standard deviation of the spatial coordinate deviation of each feature point pair within a preset annular area on the blade surface between adjacent frames. Feature point pairs with spatial coordinate deviations less than the screening threshold are retained as valid feature matching point pairs.

[0010] Preferably, S4 includes: Using effective feature matching point pairs as input, the offset of each frame image sequence is calculated by the spatial coordinate deviation of the same feature points between adjacent frames, and the relative rotation offset of the blade surface between adjacent frames is solved based on the offset. The rotation angle compensation parameters are iteratively optimized by using the spatial coordinate consistency deviation of effective feature point pairs within the annular template region before and after compensation as the loss function. The vibration displacement component of the UAV is obtained by integrating the vibration data collected synchronously by the UAV. The vibration displacement component is then subtracted from the image sequence after rotation angle compensation and correction to separate the imaging pseudo-deformation component introduced by mechanical vibration. By aligning the multi-view images after separating the pseudo-deformation, the UAV pose data, and the operating condition time series information, an integrated dataset is constructed.

[0011] Preferably, S5 includes: Based on the three-dimensional spatial coordinates of the corresponding feature points in each frame of the integrated dataset and the initial reference configuration of the blade, the full-field displacement field of each measuring point on the blade surface is calculated. Based on the relative rotation offset of each frame, a rigid body rotation displacement field of the blade is constructed. The rigid body rotation displacement component is subtracted from the total displacement field to obtain the pure deformation displacement field. Spatial numerical differentiation is performed on the pure deformation displacement field to calculate the full-field strain distribution on the blade surface; By combining the full-field displacement field and full-field strain distribution at various times, a spatiotemporal deformation evolution dataset of the blade is constructed.

[0012] Preferably, S6 includes: A design matrix was constructed using the displacement field time series data in the blade spatiotemporal deformation evolution dataset, and decomposed into low-rank components and sparse components. Using the low-rank component as the health reference benchmark, and combining the thin shell vibration equation, the components in the sparse component that belong to the physically permissible damage evolution are identified. The spatiotemporal evolution characteristics of damage evolution components in sparse components are extracted to infer the development trend of blade damage.

[0013] Preferably, S7 includes: Using the spatiotemporal deformation evolution dataset of blades and the development trend of blade damage as inputs, and the type, location and severity of blade damage as outputs, a hybrid-driven damage recognition network consisting of a physical information constraint layer and a data-driven learning layer is constructed. The physical information constraint layer constructs the physical constraint loss function constraint terms based on the thin shell vibration equation. : ; in, The total number of samples; For the first Network prediction of displacement field for each sample; The strain tensor field is calculated using geometric equations; The reference strain tensor field is obtained by averaging the displacement field data of two other blades on the same wind turbine at the same rotation azimuth angle as the target blade after calculation by geometric equations. The stress tensor field is calculated from the constitutive equation; For divergence operators; Density of the blade material; This is the second-order time derivative of the displacement field; External force term; and These are dynamic weighting coefficients; The data-driven loss in the data-driven learning layer includes damage type classification loss, damage location regression loss, and damage severity regression loss; Based on physical constraint loss Constructing the network total loss function using data-driven loss ; Using the total network loss function The network parameters are trained with the goal of minimizing the damage. The trained hybrid-driven damage recognition network is then used to infer the spatiotemporal deformation data of the blade and output multi-dimensional discrimination results of the blade damage type, location and severity.

[0014] On the other hand, the present invention also proposes a mobile vision-based in-service wind turbine blade structural health monitoring system, comprising: The data acquisition module is used to maintain the wind turbine's grid-connected rotation state, collect multi-source time-series information including blade movement visual spatiotemporal sequence data and operating condition time-series information, and unify the timestamps of the multi-source time-series information; The calibration and solution module is used to perform multi-view calibration calculations and solve coordinate transformation matrices between multiple coordinate systems using standard templates. The registration and filtering module is used to register the three-dimensional feature points of the rotating blade by combining the coordinate transformation matrix, the spatiotemporal sequence data with the unified timestamp and the working condition time series information, and to evaluate and filter the effective feature matching point pairs based on the arrangement constraints of the annular template. The iterative correction module is used to calculate the rotation offset of each frame based on the selected valid point pairs, iteratively correct image distortion through rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporal continuous correlation features of blade geometry. The deformation calculation module is used to calculate the full-field displacement and strain fields of the blade based on the integrated dataset, realize the three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade. The trend projection module is used to perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to project the development trend of blade damage. The damage discrimination module is used to integrate spatiotemporal deformation evolution data and damage evolution trends, and construct mechanical constraints through a hybrid mechanism and data-driven approach to discriminate the local damage state of the blade from multiple dimensions.

[0015] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects: 1. It adopts a mobile vision non-contact measurement method using drones, eliminating the need to pre-embed sensors inside the blades and avoiding spatial limitations at the blade roots, tips, and leading and trailing edges. During monitoring, the wind turbine maintains normal grid-connected power generation without needing to shut down or reduce power, significantly reducing power generation losses caused by monitoring operations. It also overcomes the limitations of ground-based fixed cameras, such as limited field of view and non-orthogonal light axis during nacelle yaw, making it suitable for various wind farm environments, including mountainous areas, nearshore areas, and deep-sea areas.

[0016] 2. By using a dynamic thresholding method with a ring template to filter effective feature matching point pairs, and combining a dual correction strategy of rotation angle iterative compensation and UAV vibration component separation, the geometric imaging distortion caused by high-speed blade rotation and the pseudo-deformation components introduced by the UAV's own mechanical vibration are effectively eliminated. This ensures that the calculation results of the three-dimensional displacement field and strain field truly reflect the stress and deformation response of the blade structure, and effectively preserves the local deformation characteristics caused by minor damage.

[0017] 3. By using low-rank matrix decomposition, the spatiotemporal deformation data of the blade is split into low-rank components reflecting healthy periodic vibrations and sparse components characterizing abnormal deformations caused by damage. This avoids the masking of weak damage characteristics by the vibration of normal alternating loads of the wind turbine. Combined with the thin-shell vibration equation, the physical feasibility of the sparse components is verified, effectively eliminating non-physical random measurement noise. This allows for accurate prediction of the long-term expansion trend of damage and enables early warning of early damage.

[0018] 4. A hybrid-driven damage identification network is constructed by integrating mechanical mechanism constraints with data-driven deep learning. This overcomes the problem of insufficient modeling accuracy caused by the nonlinear vibration characteristics of large flexible blades in pure finite element models, and also makes up for the shortcomings of pure data-driven models in lacking physical interpretation and poor environmental generalization ability. The thin shell vibration equation is embedded as a prior physical constraint into the loss function, so that the network prediction results conform to the real mechanical laws of composite blades, and the cause of damage can be traced and analyzed. Finally, the network outputs three-dimensional quantitative discrimination results of damage type, damage location and severity, which can directly serve operation and maintenance decision-making. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart of the method provided by the present invention; Figure 2 The system architecture diagram provided for this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] On the one hand, such as Figure 1 As shown in the figure, this invention proposes a method for monitoring the structural health of in-service wind turbine blades based on mobile vision, including: S1. Maintain the wind turbine in grid-connected rotation state, collect multi-source time series information including blade movement visual spatiotemporal sequence data and operating condition time series information, and unify the timestamps of the multi-source time series information.

[0023] Under the operating conditions of stable grid-connected power generation and continuous uniform rotation of the rotor, the acquisition of multi-source time-series information across the entire domain is carried out in parallel through two data channels: mobile visual spatiotemporal sequence and unit operating condition time-series data. A UAV equipped with a multi-view global exposure industrial camera cruises at a constant speed along a preset orbital path. The onboard imaging equipment continuously captures complete images of the rotating wind turbine blades at a constant frame rate. The generated mobile visual spatiotemporal sequence data, in addition to the original blade images of each frame, simultaneously bundles the six-degree-of-freedom pose output from the onboard GNSS inertial navigation system, the original sampled values ​​of three-axis vibration acceleration, and the camera shutter trigger timing markers. All visually relevant data is cached in real-time to the onboard edge storage unit and transmitted back to the ground workstation in batches via a wireless image transmission link. The unit operating condition time-series data is pulled in real-time via the wind farm SCADA industrial bus interface. The data stream encapsulates five core operating parameters: real-time rotor speed, real-time blade pitch control angle, nacelle yaw angle, real-time active power output of the unit, and rotor azimuth encoder angle. Stable data transmission is achieved through the on-site fiber optic ring network. The two sets of acquisition links are independent of each other. The vision device and the main control SCADA system are each equipped with an independent crystal oscillator. There are natural differences in sampling period and data transmission delay. If the raw data is directly mixed, it will cause the timing of blade imaging and corresponding load conditions to be misaligned. The three-dimensional reconstruction and deformation calculation will introduce unavoidable system errors. Unifying the global timestamp benchmark is a necessary prerequisite for realizing multi-source data fusion.

[0024] The entire timestamp calibration system uses Coordinated Universal Time (UTC) as the unified absolute time standard, completing time synchronization traceability for both acquisition channels at the hardware level. The UAV's onboard computing unit integrates a BeiDou / GNSS timing module; each camera shutter trigger synchronously reads the current UTC standard time, which is directly embedded into the corresponding image frame file header for storage. The wind turbine main control cabinet connects to the unified on-site timing device, and the SCADA system's local system clock continuously synchronizes with the UTC reference, synchronously recording a UTC time stamp each time a sampling action for operating parameters is triggered. Although the two data streams differ in sampling frequency and storage format, all original time stamps are traceable to the same standard time standard, providing a unified metrological scale for subsequent high-precision time series interpolation and registration.

[0025] We define the normalized mathematical definition for the two acquired data streams, and denote the complete visual sampling sequence as a set. In the formula This represents the i-th frame of the leaf image. The UTC timestamp bound to this frame image. This represents the total number of image frames acquired within a single inspection cycle. The complete sampling sequence of unit operating conditions is denoted as set [set name missing]. parameter vector The impeller speed, pitch angle, nacelle wind direction angle, active power, and impeller azimuth angle correspond sequentially to the j-th sampling point. Here, N represents the UTC timestamp corresponding to the operational condition sampling, and N is the total number of operational condition sampling points. The sampling frequency range for airborne vision is set to 25~60Hz, while the sampling frequency for SCADA operational condition data is only 1~10Hz. The visual sampling density is much higher than the operational condition sampling density. The two types of data are discretely and interleaved on the time axis, and cannot be directly matched one by one. Interpolation is required to complete the time alignment.

[0026] The temporal registration operation uses the sampling time of the visual image as a reference and employs a piecewise linear interpolation algorithm to complete the operating parameters corresponding to each image time. For any given frame image, the corresponding UTC time is... Traverse the time series dataset of working conditions to locate the conditions that meet the requirements. The expression for interpolating and calculating the synchronous operating condition parameter vector from two adjacent sets of operating condition sampling samples is as follows: ; The calculation process performs independent linear interpolation on each component of the operating condition parameter vector, such as rotational speed and pitch angle, to obtain the equivalent operating condition parameters matched to the current image time. If the image sampling time falls outside the entire operating condition sampling time interval, no interpolation operation is performed. Instead, the most recent set of operating condition parameters at the time endpoint is directly taken for constant value extension and filling, avoiding the problem of operating condition numerical distortion caused by extrapolation.

[0027] After interpolation, each blade image can be matched with a synchronized UTC timestamp and the equivalent operating parameters obtained through interpolation, ultimately forming a standardized multi-source time-series dataset with complete time alignment. This dataset fully preserves the visual geometric information of the blades, the disturbance information of the UAV, and the real-time load condition information of the unit. There is no matching deviation caused by time series offset, so it can be directly used as a standardized input data source for multi-view calibration and blade stereo feature registration.

[0028] S2. Solve the coordinate transformation matrix between multiple coordinate systems by performing multi-view calibration calculations using a standard template, including: S21. Using multiple non-collinear reflective markers pre-set at the root of the leaf as a standard template, determine the spatial coordinates of each marker in the world coordinate system, and establish a local coordinate system for the leaf with the center of the root of the leaf as the origin.

[0029] A standard calibration template is pre-laid in a predetermined area near the flange at the blade root. The template consists of no fewer than six non-collinear, highly reflective markers, all evenly distributed along the circumference of the blade root. This ensures that the UAV can capture a sufficient number of effective markers from any inspection angle, completely avoiding calibration failures caused by single-view marker occlusion or insufficient markers. A high-precision total station or laser tracker is used to measure the spatial coordinates of each reflective marker. All coordinates are traced back to the wind farm geodetic benchmark, constructing a unified global coordinate system spatial benchmark. A dedicated local coordinate system for the blade is established with the geometric center of the blade root section as the origin. This coordinate system follows industry-standard structural mechanics definitions: the X-axis points along the blade span towards the blade tip, the Y-axis points along the blade chord towards the trailing edge, and the Z-axis is automatically determined by the right-hand screw rule, providing a fixed structural benchmark for subsequent blade deformation measurement, feature registration, and deformation field calculation.

[0030] Perform normalized mathematical modeling on the spatial coordinates of the marked point, let the first... The three-dimensional spatial coordinates of the reflective markers in the world coordinate system are: The three-dimensional spatial coordinates of the same marker point in the local coordinate system of the blade are: The blade root is rigidly connected to the hub, exhibiting no significant elastic deformation during operation. Only rigid rotation and translation transformations exist between the two coordinate systems, with no scaling or deformation errors. The rigid body transformation constraint is strictly satisfied, as expressed in the following expression: ; In the formula, Let be the third-order rotation matrix from the local coordinate system of the blade to the world coordinate system. This corresponds to the three-dimensional translation vector. The data was obtained through on-site measurements using high-precision measuring equipment. Based on the parameters of the blade root flange design drawings or precise on-site measurements, and using a singular value decomposition algorithm, a unique rotation matrix and translation vector can be robustly obtained from the dual coordinate system samples of multiple marked points, thus completing the benchmark binding between the world coordinate system and the blade local coordinate system.

[0031] This calibration reference layout is highly engineering-rational. The blade root, as the load-bearing constraint end of the entire aircraft, exhibits minimal structural deformation and highly stable geometric attitude during operation, making it the only area on the entire blade suitable as a long-term fixed reference. The non-collinear circumferential arrangement of multiple marker points effectively optimizes the matrix condition number for solving the spatial equations, completely eliminating ill-conditioned problems caused by collinear measurement points and ensuring the stability and uniqueness of the coordinate system transformation parameters. The fixed local coordinate system of the blade unifies the blade point coordinate reference across all time series and perspectives, enabling lateral comparability of blade geometric data collected at different times and from different shooting angles, providing core support for subsequent time-series deformation evolution analysis.

[0032] S22. Collect the image coordinates of the marker points in the multi-view images, and solve the extrinsic parameter matrix of each camera by combining the UAV pose data.

[0033] After establishing the coordinate system benchmark and determining the coordinates of the marker points, multi-view image acquisition and sub-pixel level image point coordinate extraction were carried out. A UAV equipped with a multi-view camera system completed fixed-point shooting at multiple preset hovering positions. Each hovering position corresponded to an independent set of camera views, and all captured images covered the entire reflective marker area at the leaf root. For each acquired image frame, a sub-pixel positioning algorithm combining grayscale centroid and ellipse fitting was used to accurately extract the two-dimensional image coordinates of each reflective marker point, effectively suppressing point extraction errors caused by image noise. The image coordinates of the m-th marker point under the i-th camera view are defined as follows: At the same time as the camera captures the image, the UAV's onboard integrated navigation system outputs the camera's spatial position in the world coordinate system in real time. With attitude quaternions The attitude quaternion can be solved in real time and converted into a third-order rotation matrix. This provides prior attitude information for solving camera extrinsic parameters.

[0034] The intrinsic parameter matrix of the i-th camera is assumed to be... The intrinsic parameter matrix integrates inherent imaging parameters such as camera focal length, principal point coordinates, and lens distortion coefficients. Based on the pinhole camera imaging model, the three-dimensional coordinates of the marker point in the world coordinate system can be mapped to the two-dimensional image plane after camera extrinsic parameter transformation and intrinsic parameter projection. The complete projection imaging constraint formula is: ; In the formula, For imaging depth scale factor, , Let be the rotation matrix and translation vector corresponding to the i-th camera viewpoint, respectively, jointly representing the complete spatial transformation relationship from the world coordinate system to the camera coordinate system. Given the camera intrinsic parameters, the 3D spatial coordinates of the marker points, and the 2D image coordinates of the marker points, this formula can construct the geometric constraint equations for solving the camera extrinsic parameters. The PnP perspective point algorithm is used to iteratively solve the constraint equations, utilizing the geometric constraints of multiple sets of 3D-2D corresponding point pairs to minimize the initial projection error and obtain the initial solutions for each camera extrinsic parameter. The pose data output by the UAV integrated navigation system can be directly converted into initial values ​​for extrinsic parameter iteration, significantly narrowing the parameter search interval and accelerating the iteration convergence speed. Simultaneously, it effectively avoids the local optimum problem that easily occurs during nonlinear optimization. Even with a small number of marker points partially occluded, the effectiveness of the solution can still be guaranteed by relying on prior pose information.

[0035] The calibration architecture based on multi-view redundant observation fundamentally solves the technical shortcomings of single-view depth ambiguity and non-unique pose solutions. The intersection constraint of the projected rays from marker points at different views in three-dimensional space significantly improves the robustness of spatial pose solutions. Due to the superposition of image acquisition noise, sub-pixel extraction errors, and UAV pose measurement errors, the intrinsic and extrinsic parameters obtained independently from each view are inconsistent and cannot be directly used for unified registration of multi-view images. Therefore, global joint optimization calibration is required to achieve unified correction of parameters across all views.

[0036] S23. With the goal of minimizing the reprojection error between the spatial coordinates of the marker points and the image coordinates, global optimization is performed on the extrinsic parameter matrix and the camera intrinsic parameters to obtain the coordinate transformation matrix between multiple coordinate systems.

[0037] With minimizing the reprojection error of all marked points as the core optimization objective, and treating the intrinsic and extrinsic parameters of all cameras as optimizable variables, a multi-view joint nonlinear least squares objective function is constructed. To improve optimization robustness and suppress the interference of mismatched outliers, a Huber robust kernel function is introduced to optimize residual constraints. The modified global optimization objective function expression is as follows: ; In the formula, N represents the total number of camera viewpoints, and M represents the total number of calibration markers. This is a camera projection function used to calculate the mapping from 3D spatial points to 2D image coordinates. This is the Huber robust kernel function. It employs a quadratic penalty mechanism for small-amplitude normal residuals to ensure optimization convergence accuracy; and a linear penalty mechanism for large-amplitude abnormal residuals to mitigate the interference of mismatches and noise outliers, significantly improving the calibration algorithm's anti-interference capability.

[0038] The Levenberg-Marquardt algorithm is employed to solve the aforementioned large-scale nonlinear optimization problem, with the parameter sparsity structure used to accelerate iterative computation. The optimization convergence is determined using a double-threshold criterion: when the parameter increment norm of two consecutive iterations is less than a preset convergence threshold, or when the relative decrease in the objective function is less than a preset threshold, the optimization is considered terminated, and the optimized optimal camera intrinsic parameters are output. extrinsic rotation matrix Translation vector with extrinsic parameters .

[0039] Based on optimized and precise calibration parameters, accurate transformations of any spatial point between multiple coordinate systems can be achieved. For any spatial point in the world coordinate system... Combining the coordinate system rigid body transformation matrix obtained earlier, its corresponding coordinates in the blade's local coordinate system can be calculated inversely. The inverse coordinate transformation formula is: ; The global joint optimization calibration fully utilizes redundant observation information from all perspectives. Multi-dimensional error weighted averaging effectively suppresses single-view random and systematic errors. The optimization criterion of minimizing reprojection error ensures calibration accuracy at the pixel level, highly adaptable to the accuracy requirements of subsequent image feature matching and 3D reconstruction. The final obtained multiple sets of coordinate transformation relationships fully cover the bidirectional transformation mapping between the world coordinate system, the blade local coordinate system, and the coordinate systems of each camera. This enables the unified mapping of blade surface feature points extracted from all multi-view images to the blade's local reference coordinate system, providing a precise and unified spatial transformation foundation for subsequent rotating blade stereo feature registration, distortion correction, and 3D deformation calculation.

[0040] S3. Based on the coordinate transformation matrix, spatiotemporal sequence data with unified timestamps, and operating condition time-series information, perform three-dimensional feature point registration for the rotating blades. Evaluate and select effective feature matching point pairs based on the arrangement constraints of the annular template, including: S31. Combine the coordinate transformation matrix to map the multi-view images to the local coordinate system of the blade, extract the feature points on the blade surface and establish the correspondence between feature points in adjacent frames.

[0041] After accurately solving the multi-coordinate system transformation matrix, to eliminate the differences in imaging reference caused by different camera viewpoints and shooting poses, all leaf images acquired from different viewpoints are uniformly mapped to the local coordinate system of the leaf for subsequent feature analysis and matching calculations. For two-dimensional feature points extracted from any multi-view image, a hierarchical spatial transformation logic is followed, from the image plane to the camera normalized plane, then to the world coordinate system, and finally mapped to the local coordinate system of the leaf, to achieve accurate transformation from pixel coordinates to structural three-dimensional coordinates. The complete multi-view unified mapping chain transformation expression is as follows: ; In the formula The homogeneous coordinates of feature points in a two-dimensional image. To achieve the inverse mapping of image pixel coordinates to the camera's normalized plane spatial ray direction, The direction vector in the camera coordinate system is transformed to the world coordinate system. Combined with preset rigid body transformation parameters, the unique three-dimensional spatial coordinates of the feature point in the local coordinate system of the blade are finally calculated. For imaging observation of the same physical feature point from multiple perspectives, a multi-view triangulation mechanism is adopted to fuse the projection ray constraints of each perspective. With the goal of minimizing the global reprojection error, iterative optimization is performed to obtain the optimal estimate of the three-dimensional coordinates of the feature point, effectively avoiding the problems of depth ambiguity and insufficient accuracy of point location calculation in single-view observation.

[0042] After completing the unified spatial coordinate mapping of all image feature points, stable feature point extraction of the blade surface is carried out to provide a basic feature source for inter-frame matching and deformation calculation. Local feature detection algorithms with scale invariance and rotation invariance are selected for feature extraction. SIFT scale-invariant feature transformation or SURF accelerated robust feature algorithm can be used. By constructing a multi-scale pyramid from the image grayscale information, solving for gradient direction and magnitude, and selecting extreme points, highly stable feature points are extracted in areas with rich blade texture, abrupt changes in leading and trailing edge contours, and transitional areas of surface damage. For some blades with a single original factory coating and sparse surface texture, high-contrast random speckle patterns can be pre-sprayed during unit maintenance or factory delivery, or coded marker dot arrays can be deployed to artificially increase the overall texture density of the blade, completely solving the problem of sparse feature points and missing matching on smooth blades, ensuring that sufficient effective features can be extracted from the entire blade area from root to tip.

[0043] After extracting feature points from a single frame, feature point matching and association between adjacent temporal images are achieved based on feature descriptor similarity, thus constructing inter-frame feature correspondences. For any feature point in the t-th frame image... Candidate feature points of the (t+1)th frame image The Euclidean distance between feature descriptors is used as a metric for matching similarity. The smaller the distance, the higher the feature similarity. The formula for calculating the matching distance is: ; In the formula , These are the high-dimensional descriptor vectors of corresponding feature points in two adjacent frames. To avoid mismatches caused by accidental similarity in the high-dimensional descriptor space, a nearest neighbor-second nearest neighbor ratio filtering strategy is used for initial matching. The process iterates through the frames to find the candidate point with the smallest matching distance for the current feature point in subsequent frames. and the second smallest candidate point The corresponding matching distances are respectively , Preset ratio threshold Maximum allowable matching distance Only if both conditions are met and When both constraints are applied, the matching pair is determined to be a valid initial matching point pair. Based on this, a secondary filtering is performed in conjunction with the epipolar geometric constraints. By utilizing the epipolar geometric relationship of multi-view imaging, the epipolar distance between the matching point pairs is calculated, and abnormal point pairs that deviate from the epipolar constraints or do not meet the imaging geometric rules are eliminated, further improving the matching reliability from a geometric perspective.

[0044] This matching mechanism possesses strong engineering and mechanical rationality. The time interval between adjacent frames of wind turbine blades is extremely short, and the blade motion exhibits continuous characteristics. The overall motion is primarily rigid body rotation, superimposed with small-amplitude elastic vibrations and deformations. Feature point descriptors at the same physical location exhibit high repeatability, providing a fundamental prerequisite for accurate inter-frame matching. By uniformly mapping all images to the blade's local coordinate system before feature matching, the problems of feature appearance distortion and descriptor failure caused by drastic changes in different camera perspectives are avoided, significantly improving the stability of cross-frame and cross-view feature matching. Ratio checks ensure the uniqueness of features in the descriptor space, while epipolar geometric constraints ensure the physical feasibility of the matching results. The synergistic effect of these dual constraints effectively balances matching recall and precision, keeping the false matching rate at an extremely low level.

[0045] S32. Dynamically construct a screening threshold based on the statistical mean and standard deviation of the spatial coordinate deviations of each feature point pair within a preset annular region on the blade surface between adjacent frames.

[0046] After completing the inter-frame feature matching and association, the short-term motion and deformation characteristics of the blade are characterized based on the three-dimensional coordinate difference of the matched point pairs. Let the three-dimensional coordinates of the nth effective feature point in the t-th frame in the local coordinate system of the blade be... The three-dimensional coordinates of the corresponding matching point in frame t+1 are: Define the spatial coordinate deviation vector of feature points between adjacent frames as: ; The three components of the deviation vector correspond to the displacement changes in the spanwise, chordwise, and normal directions of the blade's local coordinate system, respectively. Under normal operating conditions, the inter-frame displacement deviation of a healthy blade is mainly dominated by the rotation of the impeller rigid body and minor elastic vibrations, and the displacement changes are continuous and regular. However, mismatched point pairs will produce irregular, jump-like abnormal displacement deviations, which are significantly inconsistent with the movement trend of neighboring feature points. Mismatched noise point pairs need to be removed through a regional statistical screening mechanism.

[0047] Based on the spanwise distribution characteristics of blade structural stiffness, the blade surface is divided into annular regions. Along the spanwise direction from the blade root to the tip, several annular template regions are divided at equal or adaptively non-equal intervals according to the variation law of blade cross-sectional curvature and stiffness. Each annular region corresponds to a circumferential ring region at a fixed spanwise position of the blade. Within the same region, the blade's stress-deformation mode and displacement variation law exhibit high consistency. Let the r-th annular region contain... Group feature matching of point pairs; the displacement deviation of all point pairs within a region constitutes the dataset. ,in This is the set of indices for feature point pairs within the annular region.

[0048] For the displacement deviation data of each annular region, the statistical mean and standard deviation of the spanwise, chordwise, and normal components are calculated separately. Taking the spanwise component as an example, the formulas for calculating the mean and standard deviation are as follows: ; Similarly, the chord component statistics can be obtained iteratively. , With normal component statistics , A regional dynamic threshold vector is constructed based on the statistical parameters of each component. The complete expression is as follows: ; In the formula This is the mean vector of the three components of the region. The standard deviation vector of the three components of the region. The adaptive threshold coefficient has an engineering value range of 2.5 to 3.5. This dynamic threshold can be adaptively adjusted according to different spanwise positions of the blade and different operating conditions. When the blade root has high stiffness and small deformation, the threshold is automatically lowered to improve screening accuracy; when the blade tip has low stiffness and large deformation, the threshold is adaptively amplified to avoid the actual structural deformation being misjudged as abnormal noise.

[0049] S33. Retain feature point pairs whose spatial coordinate deviation is less than the screening threshold as valid feature matching point pairs.

[0050] Based on statistics 3 The criteria complete the outlier identification and screening. For any pair of feature points within the r-th annular region, a pair of points is determined to be a valid matching point pair and retained only if the three components of the displacement deviation simultaneously satisfy the following constraints: ; If any single directional component exceeds the corresponding threshold range, the point pair is determined to be a mismatched abnormal point pair and is directly removed. The three-component joint discrimination mechanism can effectively avoid the limitations of single-dimensional screening, accurately identify local abnormal jump deviations, and thoroughly filter matching noise.

[0051] This dynamic screening mechanism aligns with the structural mechanics and statistical laws of blades. Within the same annular region, the blade deformation field exhibits spatial continuity and correlation. The displacement deviation of the true matching points follows a normal distribution, and outliers deviating from the statistical interval are spurious displacements caused by matching errors. Thresholds are calculated independently for each annular region, perfectly adapting to the non-uniform distribution of stiffness and deformation gradient from the blade root to the tip, thus overcoming the technical limitation of traditional fixed thresholds that cannot accommodate global deformation differences. After multi-level matching filtering and dynamic statistical screening, the final retained effective feature matching point pairs exhibit stable deviation distribution patterns and are free of random outlier noise, providing a clean and reliable feature data foundation for subsequent blade rotational distortion correction, spurious deformation separation, and high-precision three-dimensional deformation field calculation.

[0052] S4. Based on the filtered valid point pairs, calculate the rotation offset of each frame, iteratively correct image distortion through a rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporally continuous correlation features of blade geometry, including: S41. Using effective feature matching point pairs as input, calculate the offset of each frame image sequence through the spatial coordinate deviation of the same feature points between adjacent frames, and solve the relative rotation offset of the blade surface between adjacent frames based on the offset.

[0053] After completing the dynamic filtering and noise removal of effective feature matching point pairs in S3, high-precision three-dimensional spatial coordinates can be obtained for all corresponding feature points between adjacent frames in a unified blade local coordinate system, providing clean data support for the calculation of rigid body motion parameters between frames. The local three-dimensional coordinates of the nth effective feature point in frame t are defined as follows: The coordinates of the corresponding feature points in adjacent frames t+1 are: Under the ideal condition of pure rigid body rotation with no elastic deformation, the inter-frame motion of any feature point on the surface of the rotating blade is dominated only by the overall rotation of the impeller, strictly satisfying the standard rigid body rotation mapping relationship: ; In the formula This represents the instantaneous rigid body rotation matrix of the blades around the impeller axis within the time interval between adjacent frames. During actual grid-connected wind turbine operation, the blades continuously undergo minute elastic deformation under aerodynamic and wind disturbance loads. The measured inter-frame displacement of feature points is a coupled result of the impeller's rigid body rotation displacement and the blade's own elastic deformation displacement; the displacement of a single feature point cannot accurately represent the overall rotational behavior of the impeller. To effectively isolate local elastic deformation interference and accurately solve for the overall rigid body rotation offset between frames, a global least-squares fitting model is constructed based on all effective feature point pairs between frames. This model utilizes the multi-point global statistical optimality to suppress interference from single-point elastic deformation and residual noise. Let the total number of effective feature point pairs between the current frames be... Construct a joint error minimization objective function that includes the rotation matrix and the translation vector: ; In the formula This is the rigid body translation vector between frames, used to compensate for system translation deviations caused by small offsets in the imaging system and fine-tuning of the reference. This optimization model employs a singular value decomposition algorithm to achieve a non-iterative closed-form analytical solution. Compared to iterative optimization algorithms, it has advantages such as high computational efficiency, global optima, and no local extremum convergence problems, making it suitable for the batch real-time processing requirements of continuous image frame sequences. The specific solution process is as follows: First, the spatial centroids of all effective feature points in the two frames are calculated to eliminate the interference of the overall translation component on the rotation solution. The centroid calculation formula is: ; Based on decentralized feature point coordinates, an inter-frame feature point cross-correlation matrix H is constructed to characterize the spatial mapping and correlation properties of feature points between two frames. The matrix construction formula is as follows: ; Perform standard singular value decomposition on the cross-correlation matrix The initial solution yields the optimal inter-frame rotation matrix and translation vector: ; To strictly satisfy the orthogonality property of rotation matrices To ensure that the rotation transformation conforms to physical laws, the determinant of the matrix must be constrained to always be +1. If the calculation yields... If reflection distortion is found in the matrix, the last column vector of matrix V is inverted and recalculated to obtain the standard optimal rotation matrix that meets the physical constraints of rigid body rotation.

[0054] S42. Using the spatial coordinate consistency deviation of effective feature point pairs within the annular template region before and after compensation as the loss function, the rotation angle compensation parameters are iteratively optimized.

[0055] Based on the obtained optimal rigid body rotation matrix, the true relative rotation angle of the impeller between adjacent frames is quantitatively extracted, achieving accurate quantitative characterization of blade rotational distortion. Let the unit direction vector of the impeller rotation axis in the blade's local coordinate system be... Based on the inherent mapping relationship between the trace of the rotation matrix and the rotation angle, the precise calculation formula for the inter-frame relative rotation offset is derived: ; To avoid distortion in the rotation parameter solution caused by residual mismatch points and minor deformation interference, physical constraints of unit operating conditions are introduced to complete consistency verification. The inter-frame relative rotation angles solved by the algorithm are then used for this purpose. Compared with the theoretical rotation angle of the unit A comparison was made, among which This represents the average impeller speed during the current inter-frame period. A fixed image sampling time interval is set for the camera. An angular deviation tolerance threshold is pre-set. If the deviation exceeds this threshold, it is determined that abnormal noise still exists in the current effective feature point set. The process then backtracks to the S3 dynamic threshold filtering step, fine-tunes the statistical threshold coefficient for the annular region, and re-completes the feature point filtering, rigid body fitting, and rotation angle calculation until the calculated angle closely matches the theoretical angle under operating conditions. This verification mechanism constrains the accuracy of rotation parameter calculation from the perspective of the wind turbine's operating mechanism, completely eliminating rotation distortion calculation errors caused by residual feature noise and ensuring the physical authenticity of the rotation parameters in each frame.

[0056] S43. Obtain the vibration displacement component of the UAV by integrating the vibration data synchronously collected by the UAV, subtract the vibration displacement component from the image sequence after rotation angle compensation and correction, and separate the imaging pseudo-deformation component introduced by mechanical vibration.

[0057] After obtaining reliable inter-frame relative rotation angles through frame-by-frame verification, the absolute cumulative rotation angle of each frame relative to the initial reference frame is obtained through time-series cumulative calculation. Based on this, global compensation and correction of blade rotation imaging distortion is achieved. Due to random errors in feature extraction, minor environmental disturbances, and discrete sampling errors, the rotation parameters obtained by directly fitting frame by frame exhibit slight temporal fluctuations. Single-frame fitting results cannot meet the requirements for high-precision distortion correction across the entire blade domain. Therefore, combining the blade spanwise annular partitioning structure established in S3, an iterative optimization loss function with multi-region feature consistency constraints is constructed. Redundant feature information across the entire blade domain is used to refine and accumulate rotation compensation parameters, eliminating single-frame solution fluctuation errors.

[0058] Based on the spatial location of feature points in the initial reference frame, and with the goal of achieving optimal spatial consistency of feature points in each annular region after compensation, a global loss function is constructed: ; In the formula, R represents the total number of annular templates along the blade span. Let r be the set of valid feature point indices within the r-th annular region. The three-dimensional coordinates of the reference feature points under the initial reference frame. Let the blade be in the local coordinate system and rotate about the impeller axis, with a rotation angle of . The standard rotation matrix. The physical core of this loss function is: the precise cumulative rotation compensation angle can offset the spatial offset of feature points caused by the rigid body rotation of the blade to the greatest extent, so that the position of the compensated feature points is highly coincident with the initial reference position, and the remaining deviation only retains the true elastic deformation component of the blade, without rotational distortion interference.

[0059] The gradient descent iterative algorithm is used to perform high-precision iterative optimization of the cumulative rotation angle, which converges quickly to obtain the optimal compensation parameters. The iterative update formula is as follows: ; In the formula The learning rate is used to balance the convergence speed and solution accuracy. The gradient term is precisely solved through chain rule differentiation of the rotation matrix parameters. The iterative process continuously updates the cumulative rotation angle parameters. When both the parameter iteration increment and the decrease in the loss function converge to a preset minimum threshold, the iteration is considered convergent, and the optimal cumulative rotation compensation parameters are output. .

[0060] The loss function is constructed based on the global annular partitioning, which can fully adapt to the non-uniform deformation characteristics from the leaf root to the leaf tip. It utilizes redundant observation information from multiple regions to suppress the optimization bias caused by local feature anomalies, and achieves high-precision, dead-angle-free correction of the distortion of the global rotational imaging of the leaf.

[0061] S44. Align the multi-view images after separating the pseudo-deformation, the UAV pose data, and the working condition time series information to construct an integrated dataset.

[0062] After completing the precise compensation and correction of blade rotational distortion, the image sequence still contains high-frequency pseudo-deformation errors caused by UAV hovering / flight vibrations. These errors are independent of the actual structural deformation of the blades and are a core system interference in mobile vision monitoring. From the frequency domain characteristics analysis, the two types of motion signals are naturally separable: blade rotation and elastic deformation are low-frequency signals, with a dominant frequency range of 0.5~5Hz; camera vibrations caused by UAV rotor vibrations and airflow disturbances are high-frequency signals, with a dominant frequency range of 50~500Hz. The frequency bands do not overlap, and the pseudo-deformation components can be accurately separated using airborne high-frequency inertial measurement data.

[0063] The UAV's onboard inertial measurement unit synchronously acquires triaxial acceleration and triaxial angular velocity data at a high frequency of 200-1000Hz, fully covering high-frequency vibration components. The original time-series observation data is... The raw acceleration data contains systematic errors such as DC bias and baseline drift, which need to be preprocessed to remove interference components. The preprocessing formula is as follows: ; The preprocessed data retains only the pure vibration acceleration components. Two integration operations are performed in the discrete time domain using a trapezoidal integral algorithm, simultaneously coupled with high-pass filtering. The filter cutoff frequency is set to 1.5 times the impeller rotation frequency to completely eliminate low-frequency integration drift, accurately solving for the UAV's vibration velocity and displacement. ; The vibration displacement obtained by integral solution is the displacement component in the UAV body coordinate system. To match the blade deformation measurement benchmark, it relies on the UAV's real-time attitude quaternion. Solve the rotation matrix from the body coordinate system to the local blade coordinate system. To achieve coordinate datum unification for displacement components: ; After correcting for rotational distortion, the 3D coordinates of the feature points are frame-by-frame deducted from the high-frequency vibration displacement components of the UAV in the local coordinate system of the blade at the corresponding moment. This completely eliminates pseudo-deformation interference, ultimately yielding pure 3D coordinates of the feature points that contain only the true elastic deformation of the blade. ; After completing all distortion correction and pseudo-deformation separation processing, multi-source heterogeneous data time-series alignment and fusion were carried out to construct a standardized, high-fidelity, and time-series unified integrated dataset. Using the visual image UTC timestamp as the sole time-series reference, piecewise linear interpolation registration was performed on the unit's SCADA operating parameters (impeller speed, blade pitch angle, nacelle wind direction angle, active power, etc.) to solve the time-series misalignment problem between high-frequency visual sampling and low-frequency operating condition sampling, achieving accurate matching between each frame of image and synchronous operating parameters.

[0064] The final integrated dataset combines high-quality data from multiple dimensions, and its unified data structure expression is as follows: ; In the formula The total number of valid image frames. The total number of valid feature points in the i-th frame. This represents the number of multi-view images in the corresponding frame.

[0065] This dataset completely eliminates systematic errors such as rotational geometric distortion, UAV vibration pseudo-deformation, temporal misalignment, and inconsistent benchmarks. It achieves deep fusion of visual geometric information, unit operating condition information, and motion compensation information. It is time-continuous, has a unified benchmark, and a high signal-to-noise ratio. It can provide standardized and high-quality basic data support for subsequent blade full-field displacement strain field calculation, vibration characteristic analysis, damage trend inference, and multi-dimensional intelligent discrimination model training.

[0066] S5. Based on the integrated dataset, calculate the full-field displacement and strain fields of the blade to achieve three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade, including: S51. Based on the three-dimensional spatial coordinates of the same feature points in each frame of the integrated dataset and the initial reference configuration of the blade, calculate the full-field displacement field of each measuring point on the blade surface.

[0067] The integrated dataset internally stores the three-dimensional coordinates of effective feature points on the blade surface after rotational distortion correction and separation of UAV vibration pseudo-deformation. The subscript represents the unique physical measurement point number on the blade surface, and the superscript corresponds to the time sequence identifier of the image acquisition frame. The entire matching process is executed frame by frame between consecutive adjacent images. Feature points extracted from the same physical location can maintain a unique tracking trajectory across time sequences, and the standardized local three-dimensional coordinates corresponding to the measurement point can be retrieved at any acquisition time. .

[0068] The initial reference configuration of the blades is selected based on the blade geometry corresponding to the unit's stationary state under no wind and extremely low wind speed conditions. The reference coordinates of all feature points under the stationary condition are fully collected using the S2 multi-view global calibration scheme. This baseline configuration was determined in one go during the unit's factory commissioning and annual maintenance phases, and all subsequent deformation calculations under all operating conditions are based on it as the absolute reference standard for zero deformation.

[0069] Take the valid measurement point numbered in the i-th frame, and define the absolute displacement vector as the difference between the current correction coordinates and the stationary reference coordinates: .

[0070] Displacement vectors are decomposed into three-dimensional components These correspond to the spanwise, chordwise, and normal deformation components of the blade, respectively. The reference coordinates and displacement vectors of all valid measurement points in the current frame are paired and integrated to form the discrete full-field displacement field at that moment: ; In the formula This represents the total number of valid feature points in this frame. During monitoring, if a measurement point is partially obscured or its surface texture degrades due to partial occlusion of the blade, this will affect its performance. Frame loss, Frame re-detection, directly using and The displacement vector is recalculated without relying on interpolation to fill missing time periods, thus avoiding the problem of error accumulation caused by temporal interpolation.

[0071] The calculation logic of this layer conforms to the fundamental definition of deformation measurement in solid mechanics. The displacement is uniformly solved relative to an undeformed reference, and the displacement calculations at each measuring point are independent of each other. A single-point failure will not affect the overall measurement accuracy. The density of the blade surface features directly determines the spatial resolution of the displacement field, and high-density speckle markings can capture subtle local deformation gradients.

[0072] S52. Construct the rigid body rotation displacement field of the blade based on the relative rotation offset of each frame, and subtract the rigid body rotation displacement component from the total displacement field to obtain the pure deformation displacement field.

[0073] The absolute displacement obtained by S51 solution includes both the overall rigid body rotational motion of the impeller and the elastic deformation of the blades induced by aerodynamic loads. The amplitude of the impeller rotational displacement is much larger than the micro-deformation of the structure, so it is necessary to separate the rigid body component to extract pure elastic deformation data that can reflect blade damage. The cumulative rotation angles relative to the reference in each frame are also included. The solution has been completed in S41. This parameter characterizes the total rotation of the blade from the initial stationary reference to the current frame.

[0074] Let the unit vector of the impeller rotation axis in the local coordinate system of the blade be... Construct the corresponding rotation matrix using the Rodriguez rotation equation: ; It is a third-order identity matrix. Representative vector The corresponding antisymmetric matrix, numbered The expression for the rigid body displacement components caused by the rotation of the measuring point is: .

[0075] By removing the rigid body displacement from the total absolute displacement, we obtain the purely elastic deformation displacement caused only by the bending, stretching, and torsion of the blade itself: ; By summing all the pure deformation data from all measurement points in this frame, the pure deformation displacement field is obtained: .

[0076] Under the premise of small deformation theory, blade motion can be strictly decomposed into rigid body rotation superimposed with elastic deformation. Rigid body motion does not carry structural damage information, and the data after removal can be used to assess blade stress damage. This scheme does not rely on encoder or transmission chain speed acquisition values; all rotational parameters are obtained by fitting visual feature points, and are not affected by transmission torsional vibration or speed acquisition errors.

[0077] S53. Perform spatial numerical differentiation on the pure deformation displacement field to calculate the full-field strain distribution on the blade surface.

[0078] Pure deformation displacement only characterizes the spatial offset of the measuring point. The determination of local damage to the blade requires the relative stretching and shearing degree of the strain quantification element. The strain is essentially the spatial gradient of the displacement field, which can only be solved based on the numerical differentiation of discrete measuring points.

[0079] Using arbitrary measurement point reference coordinates Centered on the blade, a set of local measurement points is constructed by retrieving nearby features along the spanwise and chordwise directions. The relative position vector of neighboring points under the baseline configuration: ; Relative position vectors of neighboring points after deformation: ; Solving the deformable gradient tensor using a least squares model based on multiple neighborhood points: ; , All are three-dimensional column vectors, and matrix multiplication yields a third-order deformable gradient tensor.

[0080] Calculation of Green-Lagrange strain tensor based on deformation gradient: For a symmetric third-order tensor, the diagonal , , The elements represent the spanwise, chordwise, and normal strains, respectively, while the off-diagonal elements represent the shear strain. Wind turbine blades are thin-shell components, and the normal strain is extremely small; therefore, engineering analysis focuses only on the spanwise and chordwise in-plane normal strains and in-plane shear strains.

[0081] right Eigenvalues ​​are obtained by performing eigenvalue decomposition. Maximum and minimum principal strains: ; Mises equivalent strain calculation formula: ; After solving the strain tensor at each measurement point, the reference coordinates are bound to the corresponding strain parameters to form the discrete strain field of the whole field in this frame.

[0082] This differential fitting method relies on the assumption of deformation continuity and uses the relative displacement gradient of local measurement points to solve for strain. It does not require the division of a finite element mesh, adapts to the non-uniform discrete feature point distribution on the blade surface, and avoids the mesh sensitivity defects of traditional mesh methods.

[0083] S54. By combining the full-field displacement field and full-field strain distribution at each moment, a spatiotemporal deformation evolution dataset of the blade is constructed.

[0084] Solving the pure deformation displacement field frame by frame With strain field Subsequently, time-series timestamps, synchronous unit operating parameters, and spatial deformation data are correlated and integrated to build a unified spatiotemporal storage architecture. The number of feature points detected varies across different image frames; cross-frame measurement point tracking is used to maintain the continuity of time-series data, and deformation data from multiple frames at the same measurement point constitutes a complete time series.

[0085] Construction format of global feature-standardized spatiotemporal dataset: ; This represents the total number of all feature points that appeared during the entire inspection cycle. This is the set of valid acquisition frame indices for this measurement point. Displacement and strain values ​​corresponding to single-frame measurement point occlusion or texture loss are marked as missing values, which can be automatically filled in or directly ignored by subsequent low-rank decomposition algorithms.

[0086] The unmarked blank areas of the blade are reconstructed using radial basis function interpolation to achieve continuous deformation field reconstruction, with the reference coordinates of all measuring points as interpolation nodes, and any position on the blade... Interpolation expression for the deformation physical quantity: ; Choose thin-plate splines or Gaussian radial basis functions, with weights The solution is obtained through nodal deformation constraints. After interpolation, the discrete measurement point data is expanded into a complete and continuous displacement and strain field of the blade, which is suitable for subsequent low-rank temporal decomposition and physical constraint neural network input requirements.

[0087] This dataset simultaneously carries the spatial deformation distribution and long-term deformation evolution time series of the blade. The time series of a single measurement point can be used for S6 damage trend decomposition. The full-field synchronous spatial deformation data supports mechanical constraint determination and damage network inference. It fully covers the entire monitoring area from the leaf root to the leaf tip, and has both spatial and temporal resolution capabilities. It completes the full-link standardization processing from the original image to the mechanical deformation physical quantity, and provides a unified input data source for damage inference and multi-dimensional state discrimination.

[0088] S6. Perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to deduce the blade damage development trend, including: S61. Construct a design matrix using the displacement field time series data in the blade spatiotemporal deformation evolution dataset, and decompose it into low-rank components and sparse components.

[0089] Based on the blade spatiotemporal deformation evolution dataset, the pure elastic deformation displacement sequence at each measuring point under continuous time can be obtained. Let the total number of effective measuring points on the blade be N, the total number of acquisition frames in the entire process be T, and the number of effective observation frames for the nth measuring point be... All time-series displacement data from measurement points are arranged according to the spatial measurement point dimension and extended according to the time frame dimension to construct a high-dimensional time-series observation matrix. Matrix elements The single-direction displacement component or three-dimensional displacement vector norm of the i-th measuring point in the j-th frame is used. To fully characterize the three-dimensional deformation characteristics of the blade, three independent displacement matrices can be constructed separately in the spanwise, chordwise, and normal directions. Alternatively, the three-dimensional displacement vectors of the measuring points can be flattened and integrated into a unified observation matrix by row.

[0090] Under healthy operating conditions, wind turbine blade deformation exhibits strong periodicity and spatial correlation: the displacement response is dominated by the impeller's fundamental frequency and harmonics, the deformation at all measuring points follows a unified structural mechanics law, and the time-series displacement matrix displays significant low-rank characteristics. When local structural damage occurs on the blade, the displacement of the measuring points in the damaged area deviates from the standard periodic deformation pattern, producing irregular abnormal fluctuations, which manifest as sparse outliers in the observation matrix. Based on this mechanical prior, the observation matrix can be decoupled into low-rank components characterizing the healthy baseline and sparse components characterizing the damage anomaly, achieving precise separation between the deformation background and the damage signal.

[0091] This embodiment uses Robust Principal Component Analysis (RPCA) to achieve matrix decomposition, and the core optimization model is a constrained convex optimization problem: ; In the formula: For low-rank health deformation matrices, It is a sparse anomalous deformation matrix; The nuclear norm is equal to the sum of all singular values ​​of the matrix. It is a convex relaxation measure of the matrix rank and is used to constrain the low-rank properties of components. It is the L1 norm, which is equal to the sum of the absolute values ​​of all elements of the matrix. It is a convex relaxation measure of matrix sparsity and is used to constrain the sparsity of outlier components. The regularization balance parameter is used to coordinate the dual objectives of low-rank fitting and sparse constraints. The model has rigorous theoretical guarantees and can accurately decompose the true low-rank background and sparse outlier components when the conditions of singular value incoherence and outlier sparsity are met.

[0092] The above optimization problem is solved iteratively using the alternating direction multiplier method (ADMM). First, the augmented Lagrangian function is constructed: ; In the formula: For Lagrange multiplier matrices, For the iterative penalty parameters, This refers to the matrix inner product operation. It is the Frobenius norm. Iterative solution follows a step-by-step update rule: fixed... Update the low-rank matrix using the singular value threshold shrinkage operator. ;fixed , Update the sparse matrix using the soft threshold shrinkage operator. The final iteration updates the multiplier matrix. With penalty parameters Continue until the objective function converges, then output the optimal decomposition result. , .

[0093] The regularization parameter adopts an adaptive value selection strategy, which, combined with the matrix dimension, enables parameter configuration without experience. ; This value can effectively avoid the problem of parameter overfitting and prevent... Excessive suppression of weak damage abnormalities The drawback of excessively magnifying the measurement noise.

[0094] Compared to traditional frequency domain filtering methods, this low-rank decomposition method does not require a preset deformation period, can adaptively extract healthy deformation principal components with spatiotemporal correlation across the entire domain, fully preserve displacement temporal phase information, make full use of the spatial correlation of blade measurement points, and accurately separate the periodic rigid and elastic deformation background.

[0095] S62. Using the low-rank component as the health reference benchmark, and combining the thin shell vibration equation, identify the components in the sparse component that belong to the physically permissible damage evolution.

[0096] The low-rank matrix obtained by decomposition The standard elastic deformation response corresponding to the blade under undamaged conditions is the benchmark for normal structural deformation under aerodynamic loads; sparse matrix It includes all displacement anomalies that deviate from the healthy baseline. However, the sparse components contain non-physical anomalies such as transient responses to turbulent gusts, measurement noise, and feature matching errors, which cannot be directly identified as structural damage. It is necessary to conduct physical feasibility screening in conjunction with the vibration dynamics of the thin-shell blade to eliminate false anomalies and retain the damage evolution components corresponding to the actual stiffness degradation.

[0097] The wind turbine blades are typical thin-walled shell structures, and their dynamic deformation strictly satisfies the thin-shell vibration control equation: ; In the formula: The structural displacement vector. The density of the blade composite material, For the shell's equivalent thickness, It is a spatial differential operator that incorporates bending stiffness, membrane stiffness, and centrifugal stiffening effect. The aerodynamic external force vector, This represents the residual of the vibration equation. An ideal healthy deformation displacement field can make the residual approach zero. Displacement anomalies caused by actual structural damage must satisfy the constraints of this mechanical equation, while non-physical noise will produce a large residual.

[0098] Spatiotemporal decoupling is performed on the sparse matrix, and the sparse anomaly displacement field of the j-th frame is extracted frame by frame. Substitute the values ​​into the thin-shell vibration equation to calculate the mechanical residuals, and after removing the external steady-state terms, construct a physical residual evaluation index: ; In the formula, the second derivative of the time series A high-precision solution is obtained using the center difference method of adjacent three frames, avoiding interference from temporal differential noise. The smaller the residual, the more consistent the sparse anomalous displacement of that frame is with the vibration law of the blade structure, and the higher the probability that it belongs to the anomaly of the real structural response; if the residual is significantly large, it is judged as a non-physical false anomaly.

[0099] To achieve quantitative discrimination, an adaptive reference scale is constructed based on the low-rank component of health, and the physical allowable probability is defined: ; Reference scale parameter The standard deviation of the global time-series residuals of the low-rank components is adaptively determined: ; In the formula A low-rank matrix The corresponding healthy displacement field of the j-th frame. This parameter uses the mechanical residual fluctuation of healthy deformation as a benchmark, making the discrimination standard physically adaptive and freeing it from dependence on manual experience thresholds.

[0100] Set a fixed discrimination threshold. Perform tiered filtering: If The sparse component of the frame is determined to be a candidate component for physically permissible damage evolution, and is retained to construct a pure damage sparse matrix. Conversely, if the data is deemed to be non-physical anomalies such as noise or matching errors, it will be directly rejected. This screening mechanism strictly follows the basic laws of structural mechanics, ensuring the authenticity and reliability of damage data from the source.

[0101] S63. Extract the spatiotemporal evolution characteristics of the damage evolution components in the sparse component and deduce the development trend of blade damage.

[0102] After the physical tolerance judgment in step S62, the sparse components that are marked as candidate components for damage evolution are... This data represents the anomalous response of the blade structure that is continuous in time, concentrated in space, and conforms to mechanical laws. It contains rich information on damage evolution, and quantitative spatiotemporal evolution characteristics need to be extracted to infer the development trend of the damage.

[0103] In the spatial dimension, For each frame, calculate the distribution of its non-zero elements. For the sparse displacement field of the j-th frame... Calculate its spatial energy concentration. First, calculate the displacement anomaly amplitude at each measuring point n. ,in Let be the abnormal displacement vector at the nth measuring point. The abnormal region is defined as the region where the amplitude exceeds a threshold. A continuous spatially connected domain, wherein This is the area threshold coefficient, typically ranging from 0.2 to 0.3. It is used to statistically determine the damaged area index for this frame. This refers to the number of measuring points within the abnormal area or the percentage of that area relative to the total surface area of ​​the blade; calculating the location of the damage center. This involves calculating the weighted average of the spatial coordinates of all measuring points within the abnormal region; and calculating anisotropy indices of the damaged region, such as the ratio of the longitudinal to chordal extension lengths of the abnormal region. These spatial features collectively constitute the spatial state description of the damage in each frame.

[0104] In the time dimension, The non-zero elements of each frame are arranged in chronological order, forming a time series of damage evolution. For the nth measuring point, the sequence of its abnormal displacement amplitude over time is as follows: The damage evolution rate at each measuring point is extracted using the sliding window method. ; in The window width is typically set to half the impeller rotation period, usually 2 to 5 times, to eliminate the effects of periodic rotation modulation. When When the value remains positive and continues to increase, it indicates that the damage at the measuring point is accelerating; when Approaching zero and When the damage stabilizes at a certain non-zero level, it indicates that the damage has reached a stable state.

[0105] The overall damage severity index is calculated by combining the evolution rates of all measuring points. : ; in and These are the mean and standard deviation of the abnormal displacement amplitude of the nth measuring point during the historical healthy period, respectively. It characterizes the cumulative degree of deviation of the leaf from the healthy baseline, and its change over time reflects the overall trend of damage development.

[0106] Time series extrapolation is used to predict the future development trend of the damage. The most recent... Within a time window Data is used to construct evolutionary models, with commonly used models including linear trend models. and exponential growth model ,in and The model parameters were obtained through least squares fitting. The Akaike Information Criterion was used to compare the goodness of fit of the two models, and the better model was selected for trend extrapolation. The fitted model parameters were then combined with the location of the damage center. Based on the migration patterns over time, a comprehensive prediction of the damage propagation direction and rate is provided: ; in, The initial damage center coordinates, The velocity vector of the damage center is obtained by linear regression of the damage center position over time.

[0107] Damage evolution is a continuous process coupled with spatiotemporal coupling. Spatial feature description ensures that the extent and location of the damaged area can be accurately tracked, while temporal feature extraction ensures that the rate and acceleration of damage development can be quantified. By combining multiple feature indicators for trend extrapolation, the limitations of single indicators being susceptible to noise interference are avoided. The extrapolation results of damage development trends provide important prior information for the multi-dimensional damage assessment in the subsequent step S7: whether the damage is in an accelerated expansion phase, whether the damage location has migrated, and the spatial distribution pattern of the damaged area are all key bases for judging the severity and urgency of the damage. At the same time, the extrapolation results extend the early warning capability of the monitoring system in the temporal dimension, enabling maintenance personnel not only to understand the current damage status but also to predict the future damage expansion path and the remaining time to reach a critical risk state.

[0108] S7. Integrating spatiotemporal deformation evolution data with damage evolution trends, mechanical constraints are constructed through a hybrid mechanism- and data-driven approach to determine the local damage state of the blade from multiple dimensions, including: Using the spatiotemporal deformation evolution dataset of blades and the development trend of blade damage as inputs, and the type, location and severity of blade damage as outputs, a hybrid-driven damage recognition network consisting of a physical information constraint layer and a data-driven learning layer is constructed.

[0109] The hybrid-driven damage recognition network in this embodiment adopts a multi-task learning architecture with shared hard parameters. The overall structure consists of three modules: a shared backbone network, a task-specific output head, and a physical information constraint layer.

[0110] The shared backbone network is responsible for extracting general feature representations from the input data. Its structure is a five-layer fully connected neural network, with the number of neurons in each layer set to 256, 128, 64, 32, and 16 respectively. Each fully connected layer is followed by a batch normalization layer and a ReLU activation function to accelerate training convergence and suppress overfitting. The input to the shared backbone network is a concatenated comprehensive feature vector, which consists of two parts: the first part is a displacement feature vector formed by flattening the temporal displacement field data of each measurement point in the blade spatiotemporal deformation evolution dataset, with a dimension of [missing information]. (N is the number of effective measurement points); the second part is the damage development trend characteristics derived from step S63, including the damage area index of the current frame. Location of the damage center Overall injury severity index Damage evolution rate at each measuring point The aforementioned trend features are compressed into a trend feature vector with a dimension of 32 after passing through independent fully connected encoding branches (two fully connected layers with 64 and 32 neurons respectively). The displacement feature vector and the trend feature vector are concatenated at the input layer of the shared backbone network to form a vector with a dimension of [missing information]. The comprehensive input feature vector.

[0111] The shared backbone network outputs a shared feature vector with a dimension of 16. This vector integrates blade spatial deformation information and temporal evolution trend information, and serves as the common input to the specific output heads of the subsequent three tasks.

[0112] The three task-specific output heads are a damage type classification head, a damage location regression head, and a damage severity regression head. They have the same structure but their parameters are independent and not shared. Each output head consists of two fully connected neural network layers. The first layer has 32 neurons, and the number of neurons in the second layer is determined by the task: the classification head's output layer has C neurons (the total number of damage categories), followed by a Softmax activation function, outputting the probability distribution of damage type C; the location regression head's output layer has 3 neurons, no activation function, and directly outputs the three-dimensional coordinate vector of the damage center in the local coordinate system of the blade. The output layer of the severity regression head has only 1 neuron, followed by a sigmoid activation function to compress the output. Interval, representing a normalized damage severity scalar .

[0113] The physical information constraint layer has no trainable parameters and does not participate in the network's forward feature mapping transformation. It only applies thin-shell structure dynamic constraints to the network's predicted displacement field during the model training stage, ensuring that the model output strictly satisfies the continuous medium mechanics and vibration control equations, effectively suppressing the non-physical understanding and overfitting problems of pure data-driven models.

[0114] Physical constraint loss as follows: ; In the formula, The total number of samples in the training set; Predict the displacement field for the three-dimensional measuring point of the blade corresponding to the k-th sample. This represents the number of valid measurement points per sample. The second-order Green-Lagrange strain tensor obtained from the predicted displacement field has a geometric constitutive relation that strictly follows the definition in continuum mechanics: ; The strain field is solved with high precision by automatic differentiation via the network, without human parameter intervention. The healthy reference strain tensor field is constructed by taking the average value of the measured displacement fields of the other two blades of the same wind turbine under the same rotation azimuth angle and the same working conditions after solving the geometric equation, and serves as the mechanical benchmark for the blade's damage-free deformation.

[0115] The second-order Cauchy stress tensor is solved using the linear elastic constitutive equations for composite materials: In the formula The fourth-order composite material elastic stiffness tensor is obtained from blade material testing and design parameter calibration. The stress tensor divergence operator characterizes the equivalent transformation of surface forces into volume forces in a micro-element. To obtain the second-order time-matter derivative of the displacement field, this paper employs a second-order precision central difference scheme for numerical solution: ; The equivalent density of the blade composite material; This represents the vector field of unsteady aerodynamic forces on the blade surface. This is an adaptive dynamic weighting coefficient that can be adjusted in real time according to the operating status of the wind turbine. When the unit's operating conditions are stable and the speed fluctuation is less than a preset threshold, the weighting is increased. The value is selected to strengthen the dominant guiding role of mechanical constraints; under unsteady conditions such as strong turbulence and gusts, Adaptive decay avoids training bias caused by the mismatch between the fixed physical model and extreme operating conditions.

[0116] The physical constraint loss comprises two complementary mechanical criteria, achieving a balance between global conservation and local sensitivity. The first criterion... To constrain strain consistency, based on the principle of unique strain distribution in healthy blades under the same operating conditions, it can highly sensitively capture local strain redistribution caused by cracks and delamination, and has a strong ability to identify weak stiffness degradation characteristics. (Second item) To ensure dynamic equilibrium of the blade micro-element and eliminate non-physical spurious deformations in the network output, the two constraints work together to effectively overcome the shortcomings of poor generalization and weak interpretability of purely data-driven models, while also compensating for the insufficient local damage identification capability of traditional mechanistic models.

[0117] During training, for each input sample k, the network outputs a predicted displacement field. Then, the physical information constraint layer performs the following six calculations: The first step is to automatically solve for the spatial partial derivatives using a deep learning framework, and then substitute them into the geometric equations to calculate the components of the Green-Lagrange strain tensor: ; Automatic differentiation is a standardized operator built into PyTorch and TensorFlow, eliminating the need to manually write differentiation code and directly applying it to the network output. Regarding spatial coordinates Find the gradient.

[0118] The second step involves extracting the displacement fields of the other two blades of the same wind turbine in the same rotational orientation from the integrated dataset. After solving for the strain using the aforementioned geometric equations, the average value is taken to obtain the reference strain tensor field. .

[0119] The third step is to solve for the stress tensor based on the contraction relationship of the linear elastic constitutive tensor of composite materials: ; It is a fourth-order elastic stiffness tensor, with parameters taken from Young's modulus, Poisson's ratio, and shear modulus from blade material test reports or design drawings.

[0120] Step 4: Calculate the second-order time derivative of the displacement field using a second-order central difference scheme: ; The sampling time interval between adjacent frames of the camera.

[0121] The fifth step is to perform divergence calculations on the stress tensor. It also relies on automatic differentiation within the framework.

[0122] Step 6: Substitute all field quantities into The formula calculates the physical residual loss, which is used as a weighted component of the total loss. It constrains the optimization direction of network parameters through backpropagation, but does not update the parameters separately.

[0123] Working condition adaptive weights Quantification rule: Set a threshold for rotational speed fluctuation rate The ratio of the standard deviation to the mean of the rotation speed over 10 consecutive frames in the sliding window is less than hour, Volatility exceeding the limit will decay exponentially. ;in The standard deviation of the sliding window rotation speed. This represents the average rotational speed within the window.

[0124] The data-driven learning layer employs a multi-task supervised learning paradigm, simultaneously achieving damage classification, spatial localization, and severity quantification. The network output includes three dimensions: discrete damage category, 3D damage center coordinates, and normalized damage severity. Damage categories include C classes: healthy and undamaged, surface cracks, composite material delamination, leading-edge corrosion, and trailing-edge cracking. The network's endpoints are activated via Softmax to output a probability vector. , of which elements This represents the predicted probability that the k-th sample belongs to the c-th type of damage. The damage location is a continuous vector output, representing the three-dimensional spatial coordinates of the damage center in the local coordinate system of the blade. The damage severity is a continuous scalar output, with values ​​normalized to between 0 and 1, where 0 represents no damage and 1 represents a critical failure state. Mean squared error loss is used for predicting the damage location, while mean squared error or mean absolute error loss is used for predicting the damage severity.

[0125] The damage classification task uses the standard multi-class cross-entropy loss function, with the following rigorous probabilistic statistical form: ; In the formula The system encodes the true labels using one-hot encoding and implements classification supervision by measuring the cross-entropy distance between the predicted probability distribution and the true label distribution.

[0126] The damage localization task employs Euclidean space norm 2 mean square error loss: ; The damage severity regression task uses a scalar mean squared error loss, in the following form: ; In the formula , These are the three-dimensional coordinates of the predicted and actual damage centers, respectively. These are the predicted and actual damage severity scalars, respectively.

[0127] Network total loss function for: ; in This is the loss balance weighting coefficient. The value is dynamically adjusted according to the training phase. In the first phase (pure data-driven training), the value is set to... =0, in the second stage (joint fine-tuning) take =0.05; and The value is fixed at 1.0 throughout the entire training process.

[0128] The training dataset was constructed as follows: At least 20,000 sets of blade spatiotemporal deformation evolution data samples were collected. Each set of samples contained 32 consecutive frames of temporal displacement field data and corresponding damage state labels. All samples were randomly divided into training, validation, and test sets in a ratio of 7:1.5:1.5. Before training, the input data was standardized so that the mean of each feature dimension was 0 and the standard deviation was 1.

[0129] The training employs a two-stage progressive strategy: The first stage is purely data-driven pre-training, using only... , and Constituting the total loss (i.e. =0), update all network parameters with the Adam optimizer, initial learning rate is 1×10. -4 The batch size is 128, and the training run is 200 epochs. The evaluation is performed on the validation set every 10 epochs. If the validation loss does not decrease for 20 consecutive epochs, the training is terminated early.

[0130] The second stage is joint fine-tuning with physical constraints, which introduces physical constraint loss on top of the pre-trained model. (Right now =0.05), with a low learning rate of 1×10 -5 The network was trained for another 50 epochs to gradually satisfy the mechanical constraints of the thin-shell vibration equation while maintaining its discrimination accuracy. L2 regularization was used in both stages to suppress overfitting, with a regularization coefficient of 5 × 10⁻⁶. -4 .

[0131] After training, all network parameters are frozen, and the network enters inference mode. During inference, the spatiotemporal deformation data and damage trend characteristics of a single sample of blade are input. The data flows sequentially through a shared backbone network to extract shared features, and is decoded by three task-specific output heads. A single forward propagation is sufficient to obtain the discrimination results in three dimensions.

[0132] The final output of the damage category is determined by the probability distribution output by the classification head and processed by the maximum probability discrimination criterion. The final determination is made by selecting the category with the highest predicted probability among the C-type lesions. The final output of the lesion location is a three-dimensional coordinate vector directly output by the location regression head. This coordinate marks the spatial location of the damage center in the local coordinate system of the blade. The final output of the damage severity is a normalized scalar output from the severity regression head after Sigmoid activation. The closer the value is to 1, the more severe the damage.

[0133] To enhance output reliability under extreme conditions, a physical residual confidence verification mechanism is added: the physical constraint loss value of the current input sample is calculated synchronously during the inference phase. If this value exceeds the training set If the sample's judgment result is more than three standard deviations from the historical mean, a credibility warning is triggered, indicating that the credibility of the judgment result may be reduced due to extreme working conditions or data anomalies, and manual review is recommended.

[0134] The final output is a discrimination result that integrates three dimensions: damage type label, three-dimensional spatial location coordinates, and normalized severity score, which can directly serve the wind farm operation and maintenance decision-making.

[0135] On the other hand, reference Figure 2 The present invention also proposes a mobile vision-based in-service wind turbine blade structural health monitoring system, comprising: The data acquisition module is used to maintain the wind turbine's grid-connected rotation state, collect multi-source time-series information including blade movement visual spatiotemporal sequence data and operating condition time-series information, and unify the timestamps of the multi-source time-series information; The calibration and solution module is used to perform multi-view calibration calculations and solve coordinate transformation matrices between multiple coordinate systems using standard templates. The registration and filtering module is used to register the three-dimensional feature points of the rotating blade by combining the coordinate transformation matrix, the spatiotemporal sequence data with the unified timestamp and the working condition time series information, and to evaluate and filter the effective feature matching point pairs based on the arrangement constraints of the annular template. The iterative correction module is used to calculate the rotation offset of each frame based on the selected valid point pairs, iteratively correct image distortion through rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporal continuous correlation features of blade geometry. The deformation calculation module is used to calculate the full-field displacement and strain fields of the blade based on the integrated dataset, realize the three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade. The trend projection module is used to perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to project the development trend of blade damage. The damage discrimination module is used to integrate spatiotemporal deformation evolution data and damage evolution trends, and construct mechanical constraints through a hybrid mechanism and data-driven approach to discriminate the local damage state of the blade from multiple dimensions.

[0136] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0137] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring the structural health of in-service wind turbine blades based on mobile vision, characterized in that, include: S1. Maintain the wind turbine in grid-connected rotation state, collect multi-source time-series information including blade movement visual spatiotemporal sequence data and operating condition time-series information, and unify the timestamp of the multi-source time-series information; S2. Solve the coordinate transformation matrix between multiple coordinate systems by performing multi-view calibration calculations using a standard template; S3. Combine coordinate transformation matrix, spatiotemporal sequence data with unified timestamp and working condition time series information to carry out three-dimensional feature point registration of rotating blades, and evaluate and screen effective feature matching point pairs based on the arrangement constraints of the annular template; S4. Calculate the rotation offset of each frame based on the selected valid point pairs, iteratively correct image distortion through rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporal continuous correlation features of blade geometry. S5. Based on the integrated dataset, calculate the full-field displacement field and strain field of the blade, realize the three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade; S6. Perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to deduce the blade damage development trend; S7. By integrating spatiotemporal deformation evolution data with damage evolution trends, mechanical constraints are constructed through a hybrid mechanism and data-driven approach to determine the local damage state of the blade from multiple dimensions.

2. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 1, characterized in that, S2 include: Using multiple non-collinear reflective markers pre-set at the leaf root as standard templates, the spatial coordinates of each marker in the world coordinate system are determined, and a local coordinate system of the leaf with the center of the leaf root as the origin is established. Collect the image coordinates of the marker points in multi-view images, and solve the extrinsic parameter matrix of each camera by combining the UAV pose data; With the goal of minimizing the reprojection error between the spatial coordinates of the marker points and the image coordinates, global optimization is performed on the extrinsic parameter matrix and the camera intrinsic parameters to obtain the coordinate transformation matrix between multiple coordinate systems.

3. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 1, characterized in that, S3 includes: By combining the coordinate transformation matrix, multi-view images are mapped to the local coordinate system of the blade, and feature points on the blade surface are extracted and the correspondence between feature points in adjacent frames is established. The screening threshold is dynamically constructed using the statistical mean and standard deviation of the spatial coordinate deviation of each feature point pair within a preset annular area on the blade surface between adjacent frames. Feature point pairs with spatial coordinate deviations less than the screening threshold are retained as valid feature matching point pairs.

4. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 3, characterized in that, S4 include: Using effective feature matching point pairs as input, the offset of each frame image sequence is calculated by the spatial coordinate deviation of the same feature points between adjacent frames, and the relative rotation offset of the blade surface between adjacent frames is solved based on the offset. The rotation angle compensation parameters are iteratively optimized by using the spatial coordinate consistency deviation of effective feature point pairs within the annular template region before and after compensation as the loss function. The vibration displacement component of the UAV is obtained by integrating the vibration data collected synchronously by the UAV. The vibration displacement component is then subtracted from the image sequence after rotation angle compensation and correction to separate the imaging pseudo-deformation component introduced by mechanical vibration. By aligning the multi-view images after separating the pseudo-deformation, the UAV pose data, and the operating condition time series information, an integrated dataset is constructed.

5. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 4, characterized in that, S5 include: Based on the three-dimensional spatial coordinates of the corresponding feature points in each frame of the integrated dataset and the initial reference configuration of the blade, the full-field displacement field of each measuring point on the blade surface is calculated. Based on the relative rotation offset of each frame, a rigid body rotation displacement field of the blade is constructed. The rigid body rotation displacement component is subtracted from the total displacement field to obtain the pure deformation displacement field. Spatial numerical differentiation is performed on the pure deformation displacement field to calculate the full-field strain distribution on the blade surface; By combining the full-field displacement field and full-field strain distribution at various times, a spatiotemporal deformation evolution dataset of the blade is constructed.

6. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 5, characterized in that, S6 include: A design matrix was constructed using the displacement field time series data in the blade spatiotemporal deformation evolution dataset, and decomposed into low-rank components and sparse components. Using the low-rank component as the health reference benchmark, and combining the thin shell vibration equation, the components in the sparse component that belong to the physically permissible damage evolution are identified. The spatiotemporal evolution characteristics of damage evolution components in sparse components are extracted to infer the development trend of blade damage.

7. The method for monitoring the structural health of in-service wind turbine blades based on mobile vision according to claim 1, characterized in that, S7 includes: Using the spatiotemporal deformation evolution dataset of blades and the development trend of blade damage as inputs, and the type, location and severity of blade damage as outputs, a hybrid-driven damage recognition network consisting of a physical information constraint layer and a data-driven learning layer is constructed. The physical information constraint layer constructs the physical constraint loss using the thin-shell vibration equation. : ； in, The total number of samples; For the first Network prediction of displacement field for each sample; The strain tensor field is calculated using geometric equations; The reference strain tensor field is obtained by averaging the displacement field data of two other blades on the same wind turbine at the same rotation azimuth angle as the target blade after calculation by geometric equations. For divergence operators; The stress tensor field is calculated from the constitutive equation; Density of the blade material; This is the second-order time derivative of the displacement field; External force term; and These are dynamic weighting coefficients; The data-driven loss in the data-driven learning layer includes damage type classification loss, damage location regression loss, and damage severity regression loss; Based on physical constraint loss Constructing the network total loss function using data-driven loss ; Using the total network loss function The network parameters are trained with the goal of minimizing the damage. The trained hybrid-driven damage recognition network is then used to infer the spatiotemporal deformation data of the blade and output multi-dimensional discrimination results of the blade damage type, location and severity.

8. A mobile vision-based in-service wind turbine blade structural health monitoring system, characterized in that, include: The data acquisition module is used to maintain the grid-connected rotation of the wind turbine, collect multi-source time-series information including blade movement visual spatiotemporal sequence data and operating condition time-series information, and unify the timestamps of the multi-source time-series information; The calibration and solution module is used to perform multi-view calibration calculations and solve coordinate transformation matrices between multiple coordinate systems using standard templates. The registration and filtering module is used to register the three-dimensional feature points of the rotating blade by combining the coordinate transformation matrix, the spatiotemporal sequence data with the unified timestamp and the working condition time series information, and to evaluate and filter the effective feature matching point pairs based on the arrangement constraints of the annular template. The iterative correction module is used to calculate the rotation offset of each frame based on the selected valid point pairs, iteratively correct image distortion through rotation angle compensation strategy, separate the imaging pseudo-deformation component introduced by mechanical vibration, align multi-source temporal information, and construct an integrated dataset containing spatiotemporal continuous correlation features of blade geometry. The deformation calculation module is used to calculate the full-field displacement field and strain field of the blade based on the integrated dataset, realize the three-dimensional deformation field estimation of the blade, and generate a spatiotemporal deformation evolution dataset of the blade. The trend projection module is used to perform low-rank matrix decomposition on the spatiotemporal deformation evolution dataset and combine it with mechanical constraints to project the development trend of blade damage. The damage discrimination module is used to integrate spatiotemporal deformation evolution data and damage evolution trends, and construct mechanical constraints through a hybrid mechanism and data-driven approach to discriminate the local damage state of the blade from multiple dimensions.

Images