A fan blade dynamic tracking shooting and detection method, system and electronic device

Abstract

The application discloses a kind of fan blade dynamic tracking shooting and detection method, system and electronic equipment, comprising: obtaining fan operating parameter, shooting platform state parameter, environmental parameter and blade image data, and according to uniform to fan SCADA clock benchmark;With state vector describes blade phase, angular velocity and blade tip three-dimensional position, uses extended Kalman filter fusion mechanism to the working condition input and visual observation are combined and fused, establishes blade target prediction model;Based on the model and with the blade speed, wind speed and phase dynamic change safety envelope generates dynamic shooting track, gimbals pointing parameter and camera acquisition parameter, and according to adaptive setting exposure time;Control shooting platform and gimbals are tracked to target blade and perform synchronous shooting;Again carry out space-time registration and image enhancement;Identify blade defect, then defect is mapped to blade spanwise / chordwise coordinate system and outputs detection report.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for wind power equipment, and in particular to a method, system and electronic equipment for dynamic tracking, imaging and detection of wind turbine blades. Background Technology

[0002] As a key component of wind turbine generators that directly interacts with wind energy, wind turbine blades are subjected to complex environments such as high wind speeds, rain erosion, sand erosion, salt spray, lightning strikes, and alternating temperature differences. This makes them prone to defects such as cracks, leading-edge erosion, coating peeling, adhesive layer delamination, and lightning strike damage. If these blade defects are not detected and addressed in a timely manner, they can lead to decreased power generation efficiency, increased vibration, and in severe cases, even blade structural failure, thereby affecting the overall operational safety of the turbine.

[0003] Existing blade inspection methods mainly include manual rope inspection, ground telescope inspection after shutdown, and static photography inspection based on drones. Manual inspection has problems such as high operational risks, low efficiency, and strong subjectivity; although shutdown inspection is convenient for taking pictures, it will cause power generation loss; static photography inspection usually requires the wind turbine to be shut down or the blade speed to be low, which is difficult to meet the dynamic blade inspection needs of wind turbines in operation.

[0004] For operating wind turbine blades, due to continuous blade rotation and the influence of factors such as yaw, pitch changes, wind disturbance, and blade elastic deformation, the target blade exhibits obvious periodic motion characteristics and a high relative velocity. For example, a 50-meter blade operating at 20 revolutions per minute has a tip linear velocity of approximately 105 meters per second; a 100-meter blade operating at 20 revolutions per minute has a tip linear velocity of approximately 209 meters per second. This magnitude of relative motion means that when a conventional camera captures a shot at a shutter speed of 1 / 2000 second, the blade tip displacement during the exposure period can reach several centimeters, far exceeding the resolution requirements for sub-centimeter level defects. This results in significant motion blur and target loss.

[0005] Traditional shooting methods based on single-frame recognition or simple target tracking are prone to problems such as target loss, unstable perspective, blurred images, repeated shooting or missed shots.

[0006] In the area of ​​dynamic inspection by unmanned aerial vehicles (UAVs), existing technologies have proposed some solutions. For example, Chinese patent application CN116480536A discloses a method for autonomous UAV tracking and inspection of wind turbine blades. This method establishes a cylindrical enclosure model of the wind turbine and dynamically tracks the blades based on visual recognition and simple spatial relationships, while performing collision detection in real time. However, this method relies solely on the UAV's own camera for visual recognition and motion control, without integrating operational data from the wind turbine's SCADA system. When strong wind disturbances cause image quality degradation or when the blades are obstructed by the tower / nacelle, visual tracking is prone to failure, making it impossible to actively predict and maintain the status using operating parameters such as wind turbine rotor speed, yaw angle, and pitch angle.

[0007] Chinese patent application CN115951718A discloses a method for unmanned aerial vehicle (UAV) inspection of wind turbine blades. The method achieves blade inspection through local dynamic path planning. However, the safe distance is set in a fixed or graded manner and fails to be continuously and dynamically adjusted according to the real-time changes in blade rotation speed, wind speed and phase. This results in two types of problems: "too conservative, resulting in excessive shooting distance and insufficient imaging resolution" or "too aggressive, resulting in high collision risk".

[0008] In addition, Chinese patent CN110261394B discloses a method for inspecting wind turbine blades during peak wind season. It adopts different fixed inspection intervals (first interval / second interval) for different wind conditions. Although it takes wind conditions into account, it is still a discrete, preset safety distance classification strategy, rather than a safety envelope constraint that changes continuously and dynamically with the real-time phase, rotation speed and platform speed of the blades. Therefore, it cannot maximize close-range shooting opportunities while ensuring safety.

[0009] In terms of image acquisition, existing non-stop inspection solutions mostly use high-pixel high-speed cameras (such as 1 / 16000 second global shutter) to suppress motion blur, but fail to establish a quantitative adaptive constraint relationship for exposure time based on the real-time linear velocity of the leaf tip and the camera ground sampling distance (GSD), making it difficult to accurately control motion blur at the pixel level under different camera models, different rotation speeds and different shooting distances.

[0010] In terms of defect localization, existing technologies often only provide defect detection results at the image pixel level, lacking a mechanism to accurately map image coordinates to the blade's physical coordinate system (spanwise / chordwise coordinates), which makes subsequent maintenance decisions, multi-period data comparison, and cross-platform data alignment difficult.

[0011] Therefore, there is an urgent need to propose a technical solution that can achieve multi-source data fusion dynamic tracking and defect detection for wind turbine blades in operation, so as to improve inspection efficiency, image quality and detection accuracy under the condition of no or minimal downtime, and to solve the shortcomings of existing technologies in terms of visual occlusion robustness, dynamic adaptability of safe distance, precise control of exposure time and physical coordinate positioning of defects. Summary of the Invention

[0012] The present invention aims to overcome at least one of the defects of the prior art and provide a method for dynamic tracking, shooting and detection of wind turbine blades, applicable to shooting platforms such as drones, ground mobile robots, tower-mounted shooting devices or nacelle-mounted shooting devices, so as to improve the stability and continuity of target tracking.

[0013] This invention provides a method for dynamic tracking, imaging, and detection of wind turbine blades, comprising the following steps:

[0014] S1. Multi-source data acquisition: Acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters, and blade image data;

[0015] S2. Establish a target blade prediction model: Based on the wind turbine operating parameters and blade image data obtained in step S1, perform blade phase recognition and motion state estimation to obtain the phase information, angular velocity information and spatial position prediction information of the target blade, and establish a target blade prediction model.

[0016] S3. Dynamic shooting trajectory generation: Based on the target leaf prediction model and the preset safety envelope, the dynamic shooting trajectory of the shooting platform, the gimbal pointing parameters and the camera acquisition parameters are generated to form the dynamic shooting window corresponding to the target leaf.

[0017] S4. Predictive tracking and synchronous shooting: Based on the dynamic shooting trajectory, gimbal pointing parameters and camera acquisition parameters, control the shooting platform and / or gimbal to predictively track the target blade, and perform synchronous shooting when the target blade enters the dynamic shooting window.

[0018] S5. Image enhancement and spatiotemporal registration: Deblurring, distortion correction, brightness compensation, and spatiotemporal registration are performed on the captured image or video data to obtain the image data of the blade to be inspected.

[0019] S6. Defect Detection: Input the image data of the blade to be inspected into the defect detection model to obtain detection data on the type, location, size and / or severity of blade defects;

[0020] S7. Defect Mapping and Structured Output: Based on the phase information, shooting platform state parameters, gimbal pointing parameters, and wind turbine geometric parameters, the blade defects are mapped to the blade coordinate system, and the detection results are output.

[0021] Furthermore, in step S1, the wind turbine operating parameters include one or more of the following: rotor speed, yaw angle, pitch angle, nacelle azimuth, power generation, and operating status information; the imaging platform status parameters include one or more of the following: platform position, speed, attitude, and altitude information; the environmental parameters include one or more of the following: wind speed, wind direction, turbulence intensity, and illumination change information; the time synchronization alignment includes using the SCADA system clock as the reference clock, according to the expression... Local timestamps of each source data Corrected to the reference clock, where This is the calibration delay of each source sensor relative to the reference clock.

[0022] Furthermore, in step S2, the method for blade phase identification and motion state estimation is as follows:

[0023] S21. Perform time synchronization processing on the fan operating parameters;

[0024] S22. Extract leaf edge, leaf tip, marker point or texture features from leaf image data;

[0025] S23. Based on the wind turbine operating parameters and blade image data, multi-source heterogeneous data fusion is performed using Kalman filtering, extended Kalman filtering, particle filtering, or deep temporal network to obtain the real-time phase and motion state estimation results of the target blade.

[0026] Furthermore, in step S23, the blade phase identification and motion state estimation employ an extended Kalman filter fusion mechanism, using a state vector... Describe the phase θ, angular velocity ω of the target blade, and the three-dimensional spatial position of the blade tip in the wind turbine coordinate system. The state prediction equation is: The observation equation is The state transition matrix F is determined by the blade rotation dynamics relationship, and the control input is... Constructed from rotor speed, pitch angle, and yaw angle, B is the corresponding control input matrix, and H is the visual observation mapping function. This refers to visual observations such as leaf tip pixel coordinates, leaf angle observations, or marker point positions extracted by the vision system. , These are process noise and observation noise, respectively, and are calculated according to a preset covariance matrix. , Weighted fusion is then performed. The extended Kalman filter operates as follows: first, based on the wind turbine operating conditions input... Perform a prediction step, then utilize visual observation. Measurement updates are performed to obtain the fused phase and motion state estimation results. When visual observation is missing due to tower / nacelle obstruction or insufficient image quality, the extended Kalman filter completes the prediction calculation based solely on the operating condition input, thereby maintaining the state continuity of the target blade when it is outside the field of view or when the field of view quality deteriorates.

[0027] Furthermore, the target blade prediction model is also used to predict the phase, angular velocity, spatial position, and time window for entering the optimal shooting field of view of the target blade at future moments, so as to generate the motion trajectory of the shooting platform and the shooting parameters of the camera in advance.

[0028] Furthermore, in step S3, the preset safety envelope dynamically changes with blade rotation speed, wind speed, blade phase, and the speed of the imaging platform, constraining the minimum safety distance and satisfying the expression. ,in This is the static minimum safe distance. ω is the ambient wind speed, r is the blade angular velocity, θ is the blade sweep radius, and θ is the real-time blade phase. As the reference phase, For the shooting platform speed, k1, k2, and k3 are preset weighting coefficients. k1 reflects the coupled effect of wind speed on the instantaneous sway amplitude of the blades and the wind resistance margin of the shooting platform; k2 reflects the instantaneous danger level near the swept surface of the blades at different phases; and k3 reflects the braking distance margin caused by the inertia of the shooting platform itself. Under the action of the safety envelope, the minimum safe distance when the target blade is in the phase interval of the incoming flow direction is automatically amplified, while it is tightened when it is in the leeward phase interval, thus making close-range shooting feasible while ensuring operational safety.

[0029] The generation of the dynamic shooting trajectory includes the following steps:

[0030] A safety envelope constraint is constructed based on the rotor radius, blade sweep area, tower position, and minimum safe distance of the shooting platform. The dynamic part of the safety envelope constraint is updated in real time according to the blade rotation speed, wind speed, and blade phase, so that the minimum safe distance is automatically increased when the target blade is in the incoming flow phase interval and tightened when it is in the leeward phase interval, in order to adapt to the close-range shooting requirements under different working conditions. Based on the predicted trajectory of the target blade and the safety envelope constraint, model predictive control, trajectory optimization algorithm, or visual servo control algorithm is used to generate the flight trajectory or motion trajectory of the shooting platform.

[0031] Furthermore, the camera acquisition parameters include at least one of exposure time, gain, shutter trigger time, focal length, and frame rate, and are adaptively adjusted according to the target leaf's movement speed and lighting conditions to reduce motion blur and improve image clarity in key areas. In one embodiment, the upper limit of the exposure time is expressed as: , ;

[0032] Constraints are applied, where p is the maximum allowed pixel displacement threshold (typically 1 to 2 pixels), and GSD is the ground sampling distance (cm / pixel). This represents the tip linear velocity (m / s). For example, when GSD = 0.5 cm / pixel, p = 1, At that time, it can be deduced that Considering system margin and actual achievable shutter speed, the following settings can be configured: This adaptive exposure control mechanism can significantly reduce tip motion blur.

[0033] In step S4, when controlling the shooting platform and / or gimbal to predict and track the target blade, at least one of the following methods is adopted: feedforward control based on the target prediction model; closed-loop correction based on the position error of the target blade in the image; wind disturbance compensation control based on environmental parameters; and when the covariance of the extended Kalman filter output is greater than a preset threshold, the feedforward control weight is automatically reduced while the visual closed-loop correction weight is increased.

[0034] Furthermore, step S4 also includes: when the target blade is detected to be obstructed by the tower or nacelle or the image quality is lower than a set threshold, switching to prediction hold mode, and the target blade prediction model continues to calculate the phase and spatial position of the target blade; after the target blade re-enters the field of view, re-identification and re-locking are performed based on phase information and image features.

[0035] Furthermore, in step S5, the deblurring is achieved using a motion deblurring algorithm; the distortion correction is achieved by correcting lens distortion based on camera calibration parameters; the brightness compensation is achieved by compensating for image brightness according to different lighting conditions, and improving image details through noise reduction and contrast enhancement; the spatiotemporal registration processing method is to extract the stable region of the target leaf at continuous time points, and achieve this through key frame screening, multi-frame fusion, or super-resolution enhancement.

[0036] Furthermore, in step S6, the defect detection model is used to identify at least one of the following: cracks, leading-edge erosion, trailing-edge cracking, lightning damage, surface paint peeling, adhesive peeling, contamination, icing, and foreign matter adhesion, and outputs the defect category, location, size, confidence level, and / or severity. The defect detection model includes one or more of the following: target detection model, semantic segmentation model, instance segmentation model, or multi-model fusion structure; the output of the defect detection model includes the defect category, bounding box, segmented region, size, confidence level, and severity level; the defect mapping and structured output in step S7 includes the wind turbine number, blade number, defect type, defect location, defect size, severity level, corresponding image evidence, shooting time, and corresponding operating condition information.

[0037] This invention also protects a dynamic tracking, imaging, and detection system for wind turbine blades, comprising,

[0038] The data acquisition module is used to acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters, and blade image data; and to synchronize and align the data from each source under a unified clock reference.

[0039] The state estimation module is used to perform blade phase recognition and motion state estimation based on wind turbine operating parameters and blade image data, and to establish a blade target prediction model using an extended Kalman filter fusion mechanism.

[0040] The trajectory planning module is used to generate dynamic shooting trajectories, gimbal pointing parameters, and camera acquisition parameters based on the blade target prediction model and the safety envelope that dynamically changes with blade rotation speed, wind speed, and blade phase.

[0041] The tracking control module is used to control the shooting platform and / or gimbal to predictively track and synchronously shoot the target blade;

[0042] The image processing module is used to perform deblurring, distortion correction, brightness compensation, and spatiotemporal registration on the acquired images or videos;

[0043] The defect detection module is used to identify the type, location, size, and / or severity of blade defects.

[0044] The results output module is used to map defects to the blade coordinate system and output the detection results.

[0045] The shooting platform can be any one of a drone, a ground mobile robot, a tower-mounted shooting device, or a cabin-mounted shooting device.

[0046] The present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, it implements the aforementioned method for dynamic tracking, imaging, and detection of wind turbine blades.

[0047] Some embodiments of the present invention also protect a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the method for dynamic tracking, imaging and detection of wind turbine blades.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] 1. This invention synchronizes the wind turbine SCADA operating parameters, inertial measurement data from the imaging platform, environmental parameters, and visual observation data under a unified clock reference, and uses an explicit state vector. By combining the extended Kalman filter fusion mechanism with the operating condition input and visual observation, this invention can accurately predict the future phase, angular velocity, and spatial position of the target blade while the wind turbine is in operation, improving the stability and continuity of target tracking. Compared with single-source prediction methods that rely solely on visual information or SCADA data, this invention can continue to maintain prediction based on the operating condition input even when visual observation is temporarily obstructed or unreliable, significantly reducing the target loss rate.

[0050] 2. This invention introduces a safety envelope formula that dynamically varies with blade rotation speed, wind speed, and blade phase. This allows the shooting platform to adaptively shrink or expand the safe distance from the blade sweeping surface under different working conditions, thereby achieving stable close-range shooting of the target blade while ensuring operational safety. This avoids the problem of excessive shooting distance or insufficient collision avoidance margin caused by the "fixed safety distance constant" in existing solutions.

[0051] 3. This invention utilizes the linear velocity of the blade tip. Adaptive setting of camera exposure time limit It can control the blade tip motion blur to within 1 to 2 pixels within the operating range of 50 to 100 meters blades and 10 to 20 revolutions per minute. Taking GSD=0.5 cm / pixel as an example, the blade tip motion blur in a single frame can be reduced from the level of 5 to 10 cm under the traditional 1 / 2000 second shutter speed to less than 1 cm, which significantly improves image clarity and defect identifiability.

[0052] 4. This invention employs a three-layer tracking control strategy consisting of feedforward predictive control, visual closed-loop correction control, and wind disturbance compensation control. Based on the state covariance output of the extended Kalman filter, the feedforward and closed-loop weights are adaptively adjusted. This effectively addresses issues such as wind disturbance, sensor measurement delay, and target occlusion during operation, achieving more stable and accurate dynamic tracking.

[0053] 5. This invention improves image quality in complex environments and increases the accuracy of defect detection by performing deblurring, distortion correction, brightness compensation, and spatiotemporal registration on captured images.

[0054] 6. By mapping the defect detection results to the blade coordinate system, this invention can realize the spanwise and chordwise location, quantitative expression and structured output of defects, which facilitates subsequent maintenance decisions, historical comparison and operation and maintenance management.

[0055] 7. This invention is not only applicable to UAV inspection platforms, but can also be extended to tower-mounted shooting devices, nacelle-mounted shooting devices and other mobile platforms, and has good versatility and engineering application value. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the overall process of the dynamic tracking, imaging and detection method for wind turbine blades provided in an embodiment of the present invention.

[0057] Figure 2 This is a schematic diagram of the module structure of the wind turbine blade dynamic tracking, imaging and detection system provided in an embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention.

[0059] Figure 4 The following is a reference schematic diagram of the dynamic shooting sub-process based on tracking compensation and the windward side (PS side) / leeward side (SS side) of the blade, provided for embodiments of the present invention.

[0060] Figure 5 This is a schematic diagram of the blade numbering and phase estimation process provided in an embodiment of the present invention.

[0061] Figure 6 This is a schematic diagram comparing the imaging effects of hovering capture (without tracking) and dynamic tracking capture methods on the same wind turbine blade tip area, as provided in this embodiment of the invention. Detailed Implementation

[0062] The accompanying drawings illustrate the technical solutions of the embodiments of the present invention in more detail. Throughout the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The described embodiments are some, but not all, embodiments of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0063] Example 1

[0064] This embodiment provides a method for dynamic tracking, imaging, and detection of wind turbine blades.

[0065] like Figure 1 As shown, this embodiment provides a method for dynamic tracking, shooting, and detection of wind turbine blades, applicable to shooting platforms such as drones, ground mobile robots, tower-mounted shooting devices, or nacelle-mounted shooting devices. The method includes the following steps.

[0066] Step S1: Multi-source data acquisition and time synchronization. Acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters, and blade image data. The wind turbine operating parameters can be provided by the wind turbine control system, SCADA system, or unit sensors, specifically including but not limited to rotor speed, yaw angle, pitch angle, nacelle orientation, power generation, and operating status information. The imaging platform status parameters can be provided by GNSS, RTK, IMU, barometer, visual odometry, or integrated navigation devices, specifically including but not limited to platform position, speed, attitude, and altitude information. The environmental parameters can be provided by anemometers, wind vanes, light sensors, or image-based environmental estimation results, specifically including but not limited to wind speed, wind direction, turbulence intensity, and light intensity variation information. The blade image data can be acquired by a visible light camera, zoom camera, or other imaging sensors. To ensure the accuracy of subsequent state estimation and control, the data from the above different sources are processed according to an expression. Synchronization alignment: Use the wind turbine SCADA system clock as the reference clock. For each source sensor's local timestamp, The calibration delay of each source sensor relative to the reference clock is measured once by the system during the initialization phase by comparing with the synchronization signal and stored in the delay calibration table. In this embodiment, the acquisition timestamp of the camera, the attitude timestamp of the IMU, the rotation speed timestamp of the SCADA system, and the timestamp of the environmental sensor are all unified to the reference clock according to this rule, so that various types of data can be fused under the same timing reference.

[0067] Step S2: Blade Phase Recognition and Motion State Estimation. Based on wind turbine operating parameters and blade image data, blade phase recognition and motion state estimation are performed to obtain the target blade's phase information, angular velocity information, and spatial position prediction information, and a target blade prediction model is established. Specifically, blade edge, blade tip feature points, blade outline, surface texture features, or preset marker features are first extracted from the blade image data as visual observations. The control input is constructed by combining operating condition information such as wind turbine rotor speed n, yaw angle ψ, and pitch angle β. Using state vectors

[0068]

[0069] Describe the phase θ, angular velocity ω of the target blade, and the three-dimensional spatial position of the blade tip in the wind turbine coordinate system. The state is updated using an extended Kalman filter fusion mechanism, and its state prediction equation is as follows:

[0070]

[0071] The observation equation is:

[0072]

[0073] Where F is the state transition matrix, derived from the blade rotation dynamics relationship (e.g., ... , The discretized form of the control input matrix is ​​determined, where B is the control input matrix constructed based on the wind turbine operating conditions, and H is the nonlinear function that maps the state to the visual observation space. , Let process noise and observation noise be represented respectively, and let their covariance matrices be respectively. and The fusion process includes two phases: prediction and updating. In the prediction phase, only the operating condition input is considered. A one-step extrapolation of the state and its covariance yields the following results: and During the update phase, visual observation will be used. The difference from the predicted value is taken as the innovation, calculated using the Kalman gain. The state is corrected. When visual observations are of insufficient quality due to tower / nacelle obstruction, strong backlighting, or motion blur, the update phase is automatically skipped, and the state is continuously calculated based on the operating condition input, ensuring that the target blade phase estimation remains continuous even with short-term visual gaps. To further improve noise resistance and prediction accuracy, particle filtering or deep temporal networks can be used instead of extended Kalman filtering, but the definitions of their state variables and observations remain consistent. In some implementations, the target blade prediction model also outputs the time window when the target blade enters the optimal shooting field of view in the future, which is used for subsequent trajectory planning and triggering shooting.

[0074] In an implementation that uses a deep temporal network to replace or assist the extended Kalman filter, one specific implementation is as follows: A multi-source feature sequence within a sliding time window of length N (e.g., N=20, corresponding to approximately 1 second of historical data) is used as the network input. The input vector at each moment is composed of wind turbine operating parameters, visual observations, and the pose of the imaging platform. The wind turbine operating parameters include rotor speed n, blade pitch angle β, and yaw angle ψ. The visual observations include blade tip pixel coordinates, blade profile principal axis angles, and their observation confidence levels. The imaging platform pose includes platform position, velocity, and attitude. The main network uses a two-layer Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), or Temporal Convolutional Network (TCN) with a hidden layer dimension of 64–128. A fully connected output layer is then connected to output the phase θ, angular velocity ω, and three-dimensional spatial position of the target blade within the next M control cycles (e.g., M=10, corresponding to approximately 500 milliseconds of prediction time). The predicted sequence.

[0075] During the training phase of the deep temporal network, blade phase and tip position obtained from historical inspection data using high-precision encoders, lidar, or manual annotation are used as labels. The loss function is the mean square error between the predicted sequence and the label sequence, with higher loss weights applied to samples with missing visual observations or low observation confidence, prompting the network to extrapolate based on the operating condition sequence during periods of visual observation interruption. Training samples can be constructed from multiple inspection data collected under different wind speeds, rotational speeds, and occlusion conditions, and expanded using a time window sliding method. During the inference phase, the deep temporal network and the extended Kalman filter run in parallel: when the visual observation confidence is higher than a set threshold, the fusion result of the extended Kalman filter is used as the state estimation output; when the visual observation confidence is lower than the set threshold or visual observations are missing, the prediction output of the deep temporal network replaces the filter's observation update result, or the two are weighted and fused according to their respective confidence levels, thereby further improving the prediction accuracy of phase and motion state under long-term occlusion and strongly nonlinear operating conditions. The state variables of the deep temporal network are defined similarly to the aforementioned state vector. and observations This ensures consistency and allows for direct connection with subsequent steps S3 to S7.

[0076] Step S3: Dynamic Shooting Trajectory Generation and Safety Envelope Constraint. Based on the target blade prediction model and the preset safety envelope, the dynamic shooting trajectory of the shooting platform, gimbal pointing parameters, and camera acquisition parameters are generated to form the dynamic shooting window corresponding to the target blade. The dynamic minimum safety distance in the safety envelope is updated in real time according to the following formula:

[0077]

[0078] in This is the minimum static safety distance (2 to 5 meters can be used as the default value). ω is the ambient wind speed (m / s), r is the blade angular velocity (radians / s), and θ is the real-time blade phase. The reference phase is the phase value taken when the blade is in a horizontal position. The shooting platform speed is given in meters per second (m / s), and k1, k2, and k3 are preset weighting coefficients (k1≈0.3s, k2≈0.05s², and k3≈0.5s can be used as example values). Under the influence of this formula, when the blade is in the phase interval of the incoming flow direction (i.e., Approaching 1), Automatic zoom forces the shooting platform to maintain a greater collision avoidance margin; when the blades are in the leeward phase range... Automatic tightening makes close-up shooting feasible. The safety envelope also includes minimum static safety distance constraints between the shooting platform and the tower and nacelle, as well as constraints such as the platform's maximum speed, maximum acceleration, flight altitude, and prohibited areas. When generating dynamic shooting trajectories, the motion trajectory of the shooting platform and the gimbal angle change sequence can be obtained through model predictive control algorithms, trajectory optimization algorithms, or visual servo control algorithms, based on the future position sequence of the target blades and the shooting angle requirements. Furthermore, by combining the blade linear velocity, lighting conditions, and target distance, the exposure time, gain, focal length, frame rate, and shutter trigger time are coordinated to form a dynamic shooting window adapted to the target's motion characteristics.

[0079] In one specific implementation, the model predictive control predicts the time domain within each control cycle. and control time domain The objective function for optimizing the motion of the shooting platform is:

[0080]

[0081] Where k is the current control period. The predicted position of the shooting platform in the (k+i)th control cycle. The desired shooting position given by the target blade prediction model is the position where the target blade is located at the center of the camera's field of view and at the optimal shooting distance, under the premise of satisfying the safety envelope constraint. and These are the predicted pointing angle and the expected pointing angle of the gimbal, respectively. For controlling the speed or acceleration of the shooting platform, This represents the control increment between adjacent control cycles; , , , Let be the corresponding positive definite weight matrix, where and Used to ensure position tracking accuracy and pointing tracking accuracy Used to suppress and control energy consumption Used to suppress control quantity mutations to obtain a smooth shooting trajectory.

[0082] The objective function is solved under the following constraints: the distance between the imaging platform and the blade sweep surface satisfies the aforementioned safety envelope constraint, i.e. The velocity and acceleration of the shooting platform satisfy the platform's kinematic constraints. and Flight altitude meets The imaging platform's position does not enter the prohibited areas corresponding to the tower and nacelle. After solving the above optimization problem in each control cycle, only the first control variable in the obtained control sequence is executed, and the problem is resolved in the next control cycle based on the updated blade prediction state, thereby achieving predictive tracking control in the rolling time domain. When using trajectory optimization algorithms or visual servo control algorithms, the position tracking term and pointing tracking term in the above formula can be used as the corresponding cost terms, while retaining the same safety envelope constraints and platform kinematic constraints.

[0083] Step S4: Predictive tracking and synchronized shooting. For example... Figure 4 As shown, based on the dynamic shooting trajectory, gimbal pointing parameters, and camera acquisition parameters, the shooting platform and / or gimbal are controlled to predictively track the target blade, and synchronous shooting is performed when the target blade enters the dynamic shooting window; the shooting target includes key areas such as the windward side (PS side) and leeward side (SS side) of the blade, as well as the leading and trailing edges. Through target tracking compensation control, the blurring problem of blade imaging under operating conditions can be improved, and the image clarity of the target blade can be enhanced. In a preferred embodiment, the tracking control adopts a dual-loop control mechanism of feedforward prediction plus visual closed-loop correction, and the weights of the two are adaptively adjusted according to the state covariance of the extended Kalman filter output: when the covariance is small and the state estimation is reliable, the weight of feedforward control is increased; otherwise, the weight of visual closed-loop correction is increased. Among them, the feedforward prediction control is used to adjust the platform attitude and gimbal pointing in advance based on the future spatial position output by the target blade prediction model; the visual closed-loop correction control compensates the platform and gimbal in real time based on the actual offset error of the target blade in the image, so as to reduce the field of view deviation and improve the target holding stability. To address the impact of wind disturbance during operation, anti-wind disturbance compensation control can be introduced based on changes in wind speed, wind direction, and platform attitude to improve the stability of the shooting process. To reduce motion blur, control is implemented based on the blade tip linear velocity. Adaptively adjust the exposure time to meet the requirements. (p is the maximum allowed pixel displacement threshold, typically 1-2 pixels; GSD is the ground sampling distance). For example, for a 50-meter blade operating at 20 revolutions per minute... When GSD = 0.5 cm / pixel and p = 1, it can be deduced that Considering the available shutter speeds and system margin, the exposure time can be set to approximately 1 / 12000s. In low-light environments, the gain can be appropriately increased and multi-frame fusion technology can be used to improve the imaging effect. When the target blade is temporarily obscured by the tower and / or nacelle, or when the image quality is below a preset threshold, the system switches to prediction hold mode and continues tracking based on the state result calculated by the extended Kalman filter using only the operating condition input. After the target blade re-enters the field of view, it is re-identified and re-locked by combining image features and phase information.

[0084] Figure 4 The enlarged view on the right further illustrates the spatial relationships in the aforementioned predictive tracking and synchronous shooting process, consisting of two parts. The upper part is a top-down view of the shooting routes and a schematic diagram of the wind turbine attitude: In the top-down view, four shooting routes are symmetrically planned along the wind turbine orientation (the direction of incoming flow indicated by the arrows in the figure), centered on the wind turbine nacelle and hub. These are the leading edge shooting route, the windward side (PS surface) shooting route, the leeward side (SS surface) shooting route, and the trailing edge shooting route. The shooting platform approaches the target blade sequentially along these four routes, thereby completing the coverage of the four key surfaces of the blade in one flight. In the schematic diagram of the wind turbine attitude on the right, the solid and dashed arrows represent the round-trip sweeping segments of the shooting platform along the blade span, from the blade root to the blade tip and from the blade tip to the blade root, respectively. The vertical arrows represent the direction of incoming flow corresponding to the wind turbine orientation, thereby determining the spatial position of each shooting route relative to the wind turbine rotation plane and the switching time between adjacent segments. The lower part is a schematic diagram of the airfoil profile of the blade, which marks the leading edge, trailing edge, windward side (PS surface), and leeward side (SS surface), as well as the main spars and shear web structure inside the profile. This illustrates the division of the blade surface area corresponding to the four shooting paths in the upper part. Specifically, the leading edge shooting path and the trailing edge shooting path cover the curvature transition areas of the leading edge and trailing edge of the airfoil profile, respectively, while the windward side shooting path and the leeward side shooting path cover the windward side and leeward side of the airfoil profile, respectively. The execution order, segment length, and corresponding gimbal pointing angle of the four shooting paths are all determined by the dynamic shooting trajectory and gimbal pointing parameters generated in step S3, and synchronous shooting is triggered when the target blade enters the corresponding dynamic shooting window.

[0085] Step S5: Image Enhancement and Spatiotemporal Registration. The captured image or video data undergoes deblurring, distortion correction, brightness compensation, and spatiotemporal registration to obtain the image data of the blade to be inspected. Specifically, firstly, lens distortion is corrected based on camera calibration parameters; secondly, motion blur generated during dynamic shooting is repaired using motion deblurring algorithms; thirdly, image brightness is compensated according to different lighting conditions, and image detail is improved through noise reduction and contrast enhancement. For continuous video data, spatiotemporal registration can be performed on multiple frames to extract stable regions of the target blade at continuous time points, and high-quality images to be inspected are generated through keyframe filtering, multi-frame fusion, or super-resolution enhancement.

[0086] As a specific implementation, the motion deblurring is achieved as follows. Since the motion state of the target blade has been given by the target blade prediction model, the blur kernel can be constructed analytically from the motion parameters, without relying entirely on blind estimation based on a single frame image. Specifically, the motion of the target blade relative to the camera image plane during the exposure period is approximated as uniform linear motion, and the length of the motion blur kernel is calculated using the following formula:

[0087]

[0088] Where L is the length of the blur kernel in pixels. Let be the equivalent linear velocity of the blade relative to the camera in this image region, and take for the blade tip region. , The exposure time is given by θ, and GSD is the ground sampling distance. The direction φ of the blur kernel is the projection direction of the phase tangent of the point at the exposure time onto the camera image plane, determined by the phase information θ, the wind turbine geometric model, and the camera's intrinsic and extrinsic parameters. Based on this, a linear motion blur kernel h(x,y) of length L and direction φ is constructed, and the blurred image is restored in the frequency domain using Wiener filtering.

[0089]

[0090] Where G(u,v) is the spectrum of the blurred image, and H(u,v) is the spectrum of the blur kernel h(x,y). Let H(u,v) be the conjugate of H(u,v), and K be a regularization constant related to the image signal-to-noise ratio. To obtain the spectrum of the restored image, an inverse Fourier transform is performed to obtain the deblurred image. For cases with strong noise or biased blur kernels, Richardson-Lucy iterative deconvolution can be used, with 10-30 iterations, combined with a total variational regularization term to suppress ringing effects. In another specific implementation, a deep deblurring network (e.g., a multi-scale encoding / decoding structure or a generative adversarial structure) trained with blurred-sharp image pairs can directly output the restored image, with the blur kernel length L and direction φ calculated by the target blade prediction model as additional input conditions to improve the stability of the restoration result. Both methods aim to restore the edge details of defects such as surface cracks and coating peeling on the blade, providing clear image data of the blade to be inspected for subsequent defect detection.

[0091] Step S6: Defect Detection. The image data of the blade to be inspected is input into the defect detection model to identify surface defects. The defect detection model can be a target detection model, a semantic segmentation model, an instance segmentation model, or a multi-model fusion structure, used to identify defect types such as cracks, leading-edge erosion, trailing-edge cracking, lightning damage, paint peeling, adhesive peeling, contamination, icing, and foreign matter adhesion. The model output may include information such as defect category, bounding box, segmented region, size, confidence level, and severity level.

[0092] Step S7: Defect Mapping and Structured Output. Based on the shooting platform position, attitude, gimbal angle, camera intrinsic and extrinsic parameters, target blade phase information, and wind turbine geometric model, establish a mapping relationship from image pixel coordinates to blade entity coordinates. Calculate the positional parameters of the defect in the spanwise and chordwise directions of the blade, and associate them with the blade number, shooting time, and operating condition parameters to form a structured inspection result. For example... Figure 5 As shown, the blade number can be estimated and relocked by normalizing the phase difference of each blade and combining it with the operating condition switching time. In this embodiment, the detection result may include at least: wind turbine number, blade number, defect type, defect location, defect size, severity level, corresponding image evidence, shooting time, and corresponding operating condition information.

[0093] Specifically, the mapping from image pixel coordinates to blade entity coordinates is achieved through the following coordinate transformation. First, the pixel coordinates (u,v) of the defect in the image are corrected for distortion based on the lens distortion coefficient. Then, the defect is back-projected into a direction vector in the camera coordinate system based on the camera intrinsic parameter matrix K:

[0094]

[0095] Where K is the equivalent focal length , and principal point coordinates , The camera intrinsic parameter matrix is ​​constructed. Secondly, a rotation matrix is ​​constructed based on the shooting platform attitude and gimbal angle. and the position of the shooting platform in the wind turbine coordinate system The direction vector is then transformed into a wind turbine coordinate system with the tower center as the origin and the rotor rotation axis as the X-axis:

[0096]

[0097] in Here, λ represents the three-dimensional coordinates of the defect point in the wind turbine coordinate system, and λ is the scale factor, determined by the intersection of the line of sight and the blade surface in the wind turbine geometric model, or directly given by ranging results from laser ranging, binocular vision, etc. Furthermore, based on the real-time phase θ of the target blade and the installation phase difference between this blade and the reference blade... (For three-bladed turbines, take values ​​of 0, 2π / 3, and 4π / 3 respectively) and the rotor cone angle γ and main shaft tilt angle τ, transform the coordinates from the turbine coordinate system to the blade coordinate system:

[0098]

[0099] in This represents the position of the hub center in the wind turbine coordinate system. For the blade phase θ+ A rotation matrix composed of the rotor cone angle γ and the main shaft tilt angle τ; the blade coordinate system has the hub center as the origin and the blade span as the axis of rotation. The axis is based on the chord direction of the blade. The axis. Finally, the location parameters of the defects in the blade spanwise and chordwise directions are respectively... Projections onto the two axes mentioned above:

[0100]

[0101] Where s is the spanwise position of the defect, and c is the chordwise position of the defect. and In the blade coordinate system shaft and The unit vector of the axis. To facilitate comparison between different models, the above position parameters can also be normalized and expressed as... and ,in For the hub radius, c is the blade length, and c(s) is the chord length at position s in the spanwise direction; the defect size is also converted from pixel size to physical size according to the above proportional relationship.

[0102] Figure 5 The four color views on the right are examples of lidar point cloud processing results during blade number estimation, used to illustrate... Figure 5 The actual execution effect of the process on the left. In each view, the measurement points belonging to different blades after being segmented by multiple blade point clouds are displayed in red, green and blue respectively. The colors are only used to distinguish different blades and do not represent other physical quantities. The spatial distribution of each colored point cloud reflects the spanwise orientation of the corresponding blade relative to the wind turbine coordinate system at that moment. Based on this, the angle of each blade can be obtained by estimating and statistically analyzing the point cloud angle. The text above each view represents the estimated parameters corresponding to the point cloud in that frame. Here, `roll` is the target blade phase angle obtained from the point cloud angle statistics, in degrees; `speed_f` and `speed_c` are the forward and current estimates of the blade rotational angular velocity obtained from the blade angle difference, respectively. Values ​​of -10.0 and -1.0 in the image correspond to the default values ​​when no effective angular velocity estimate has been obtained for that frame; `std` is the standard deviation of the point cloud angle estimate, used to characterize the reliability of the angle estimate for that frame; `lidar_time` is the acquisition timestamp of the point cloud in that frame; `delta_time` is the time interval between this frame and the previous frame of the point cloud, with a value of -1.0000 indicating that there is no effective adjacent frame interval for this frame; `start` and `end` are the start and end flags of the flight path segment, and their combinations are used to determine whether the current segment is in... Figure 5The "Is the flight path switching process in progress?" branch in the left-hand flow; id is the blade number finally locked in this frame; lidar_cnt is the number of effective point clouds participating in the angle estimation of this frame. In the four views, id takes values ​​of 2, 1, 0, and 1 respectively, illustrating the process of the target blade number being estimated, switched, and relocked sequentially as the blade rotates and the flight path switches: for frames where delta_time is positive, according to... Figure 5 In the left-hand flow, the "predict blade ID by combining blade rotation speed and interval time" branch determines the blade number; for frames where delta_time is −1.0000, the "switching blade ID by combining blade angle switching threshold" branch determines the blade number.

[0103] To verify the technical effect of the present invention, this embodiment is compared and simulated with the following parameters: the wind turbine model is a 2.5MW onshore unit, the rotor radius is 50 meters, the blade length is 48.8 meters, and the rotor speed is 20 revolutions per minute; the shooting platform is a hexacopter UAV with a maximum flight speed of 15 m / s and a maximum acceleration of 3 m / s²; the camera uses a 1-inch CMOS sensor with a focal length of 35mm and a resolution of 3840×2160, corresponding to a GSD of 0.5 cm / pixel; the control cycle is 50ms; the process noise covariance of the extended Kalman filter is taken as... The observed noise covariance is taken as .

[0104] To illustrate the technical advantages of this invention over existing dynamic focusing shooting schemes, under the same working conditions of a wind turbine radius of 50 meters, a rotor speed of 20 revolutions per minute, and a camera GSD of 0.5 cm / pixel, a comparative simulation was conducted between a control scheme using a fixed safety distance and a fixed exposure time (which maintains the same control structure as known dynamic focusing inspection schemes in the art) and this embodiment. The results are as follows: (1) The number of instantaneous loss of target blades decreased from an average of about 4.2 times per flight in the control scheme to about 0.8 times in this embodiment, a reduction of about 80%; (2) The number of motion blur pixels at the blade tip in a single frame decreased from about 8 to 10 pixels in the control scheme to about 1 to 2 pixels, a reduction of about 80%; (3) The safety margin of the closest distance between the shooting platform and the blade tip was dynamically adjusted from a fixed value of 2 meters to between 1.5 and 4.5 meters, while retaining close-range shooting opportunities in the leeward phase interval and significantly improving the collision avoidance margin in the inflow phase interval; (4) The average error of defect spanwise positioning decreased from about 30 cm in the control scheme to about 10 cm, a reduction of about 67%. The above data are typical simulation examples. In practical applications, they can be reproduced and verified according to specific models, camera configurations, and algorithm parameters. Figure 6The comparison shows the imaging effect of the same wind turbine blade tip area using the hovering capture (no tracking) method and the dynamic tracking capture method of this embodiment in the above comparison. The blade tip outline and surface texture are relatively blurry under the hovering capture method without tracking, while the blade tip outline is clear and the identifiability of surface defects is significantly improved after using the dynamic tracking capture method of this invention, which intuitively verifies the effect of this invention in suppressing blade tip motion blur.

[0105] Example 2

[0106] This embodiment provides a dynamic tracking, imaging, and detection system for wind turbine blades.

[0107] like Figure 2 As shown, this embodiment provides a dynamic tracking, imaging, and detection system for wind turbine blades, including:

[0108] The data acquisition module 201 is used to acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters and blade image data, and to perform time synchronization and alignment of the data from each source under a unified clock reference.

[0109] The state estimation module 202 is used to perform blade phase recognition and motion state estimation based on wind turbine operating parameters and blade image data, and to establish a target blade prediction model using an extended Kalman filter fusion mechanism.

[0110] The trajectory planning module 203 is used to generate dynamic shooting trajectory, gimbal pointing parameters and camera acquisition parameters based on the target blade prediction model and the safety envelope that is dynamically adjusted according to the blade rotation speed, wind speed and blade phase.

[0111] The tracking control module 204 is used to control the shooting platform and / or gimbal to predictively track and synchronously shoot the target blade;

[0112] Image processing module 205 is used to perform deblurring, distortion correction, brightness compensation and spatiotemporal registration on the acquired image or video data;

[0113] The defect detection module 206 is used to identify the type, location, size and / or severity of blade defects;

[0114] The result output module 207 is used to map defects to the blade coordinate system and output the detection results.

[0115] In this embodiment, the system can be deployed in an onboard computing unit of an unmanned aerial vehicle, an edge computing device, a ground workstation, or a cloud processing platform. It can run in real time or use a front-end acquisition and back-end analysis approach to achieve some offline processing.

[0116] Example 3

[0117] This embodiment provides an electronic device.

[0118] like Figure 3 As shown, this embodiment provides an electronic device including a processing device 1001, a read-only memory 1002, a random access memory 1004, a storage device 1003, an input device 1007, an output device 1008, a communication device 1009, and an input / output interface 1006. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected via a bus 1005 and coupled to the storage device 1003, the input device 1007, the output device 1008, and the communication device 1009 via the input / output interface 1006. When the processing device 1001 executes a computer program stored in the storage device 1003 or the read-only memory 1002, it implements the various steps in the wind turbine blade dynamic tracking, shooting, and detection method described in Embodiment 1. The electronic device can be an airborne processing device for a UAV, an edge computing gateway, an inspection workstation, a server device, or other terminal devices with computing and control capabilities.

[0119] Example 4

[0120] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various steps of the dynamic tracking, imaging, and detection method for wind turbine blades described in Embodiment 1. The computer-readable storage medium can be a read-only memory, flash memory, solid-state drive, mobile memory card, or other storage medium capable of storing program instructions.

[0121] The above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments above, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention should not depart from the spirit and scope of the present invention. Those skilled in the art can also make other changes within the spirit of the present invention and use them in the design of the present invention, as long as they do not deviate from the technical effects of the present invention. These changes made according to the spirit of the present invention should all be included within the scope of protection claimed by the present invention.

Claims

1. A method for dynamic tracking, imaging, and detection of wind turbine blades, characterized in that, Includes the following steps: S1. Multi-source data acquisition: Acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters, and blade image data; S2. Establish a target blade prediction model: Based on the wind turbine operating parameters and blade image data obtained in step S1, perform blade phase recognition and motion state estimation to obtain the phase information, angular velocity information and spatial position prediction information of the target blade, and establish a target blade prediction model. S3. Dynamic shooting trajectory generation: Based on the target leaf prediction model and the preset safety envelope, the dynamic shooting trajectory of the shooting platform, the gimbal pointing parameters and the camera acquisition parameters are generated to form the dynamic shooting window corresponding to the target leaf. S4. Predictive tracking and synchronous shooting: Based on the dynamic shooting trajectory, gimbal pointing parameters and camera acquisition parameters, control the shooting platform and / or gimbal to predictively track the target blade, and perform synchronous shooting when the target blade enters the dynamic shooting window. S5. Image enhancement and spatiotemporal registration: Deblurring, distortion correction, brightness compensation and spatiotemporal registration are performed on the captured image or video data to obtain the image data of the blade to be inspected. S6. Defect Detection: Input the image data of the blade to be inspected into the defect detection model to obtain detection data on the type, location, size and / or severity of blade defects; S7. Defect Mapping and Structured Output: Based on the phase information, shooting platform state parameters, gimbal pointing parameters, and wind turbine geometric parameters, the blade defects are mapped to the blade coordinate system, and the detection results are output.

2. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 1, characterized in that, In step S1, the wind turbine operating parameters include one or more of the following: rotor speed, yaw angle, pitch angle, nacelle azimuth, power generation, and operating status information; the imaging platform status parameters include one or more of the following: platform position, speed, attitude, and altitude information; the environmental parameters include one or more of the following: wind speed, wind direction, turbulence intensity, and light intensity variation information; the time synchronization alignment includes using the SCADA system clock as the reference clock, according to the expression... Local timestamps of each source data Corrected to the reference clock, where This is the calibration delay of each source sensor relative to the reference clock.

3. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 1, characterized in that, In step S2, the method for blade phase identification and motion state estimation is as follows: S21. Perform time synchronization processing on the fan operating parameters; S22. Extract leaf edge, leaf tip, marker point or texture features from leaf image data; S23. Based on the wind turbine operating parameters and blade image data, multi-source heterogeneous data fusion is performed using Kalman filtering, extended Kalman filtering, particle filtering, or deep temporal network to obtain the real-time phase and motion state estimation results of the target blade.

4. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 3, characterized in that, In step S23, the state vector The phase, angular velocity, and spatial position of the target blade are described using the following extended Kalman filter fusion mechanism: State prediction equation Observation equation Where F is the state transition matrix and B is the control input matrix constructed based on the wind turbine operating parameters. The input parameters consist of rotor speed, pitch angle, and yaw angle; H is the visual observation mapping function. For visual observation, , These are process noise and observation noise, respectively. By weighted fusion of the covariance of the operating condition input and visual observations, the real-time phase and motion state estimation results of the target blade are obtained. When visual observation is missing due to occlusion, the extended Kalman filter makes predictions based solely on the operating condition input.

5. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 1, characterized in that, In step S3, the preset safety envelope is a minimum safety distance constraint that dynamically changes with blade rotation speed, wind speed, blade phase, and imaging platform speed, and satisfies the expression... ,in This is the static minimum safe distance. ω is the ambient wind speed, r is the blade angular velocity, θ is the blade sweep radius, and θ is the real-time blade phase. As the reference phase, For the shooting platform speed, k1, k2, and k3 are preset weighting coefficients; The generation of the dynamic shooting trajectory includes the following steps: A safety envelope constraint is constructed based on the rotor radius, blade sweep area, tower position, and minimum safe distance of the shooting platform. The dynamic part of the safety envelope constraint is updated in real time according to the blade rotation speed, wind speed, and blade phase, so that the minimum safe distance is automatically increased when the target blade is in the incoming flow phase interval and tightened when it is in the leeward phase interval, in order to adapt to the close-range shooting requirements under different working conditions. Based on the predicted trajectory of the target blade and the safety envelope constraint, model predictive control, trajectory optimization algorithm, or visual servo control algorithm is used to generate the flight trajectory or motion trajectory of the shooting platform.

6. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 1, characterized in that, The camera acquisition parameters include at least one of exposure time, gain, shutter trigger time, focal length and frame rate. In step S4, the camera acquisition parameters are adaptively adjusted according to the movement speed of the target blade. The upper limit of the exposure time is given by the expression Determine that p is the maximum allowed pixel displacement threshold, and GSD is the ground sampling distance. Let be the linear velocity of the leaf tip, so that the motion blur of the leaf tip during the exposure time does not exceed p pixels.

7. The method for dynamic tracking, imaging, and detection of wind turbine blades according to claim 1, characterized in that, In step S4, when controlling the shooting platform and / or gimbal to predict and track the target blade, at least one of the following methods is adopted: feedforward control based on the target prediction model; closed-loop correction based on the position error of the target blade in the image; wind disturbance compensation control based on environmental parameters; and when the covariance of the extended Kalman filter output is greater than a preset threshold, the feedforward control weight is automatically reduced while the visual closed-loop correction weight is increased.

8. The method according to claim 1, characterized in that, Step S4 further includes: when the target blade is detected to be obstructed by the tower or nacelle or the image quality is lower than a set threshold, switching to prediction hold mode, and the target blade prediction model continues to calculate the phase and spatial position of the target blade; after the target blade re-enters the field of view, re-identification and re-locking are performed based on phase information and image features. In step S5, the deblurring is achieved using a motion deblurring algorithm; the distortion correction is achieved by correcting lens distortion based on camera calibration parameters; the brightness compensation is achieved by compensating for image brightness according to different lighting conditions, and improving image details through noise reduction and contrast enhancement; the spatiotemporal registration processing method is to extract the stable region of the target leaf at continuous time points, and achieve this through key frame screening, multi-frame fusion, or super-resolution enhancement. In step S6, the defect detection model includes one or more of the following: target detection model, semantic segmentation model, instance segmentation model, or multi-model fusion structure; the output of the defect detection model includes defect category, bounding box, segmented region, size, confidence level, and severity level; in step S7, the defect mapping and structured output includes wind turbine number, blade number, defect type, defect location, defect size, severity level, corresponding image evidence, shooting time, and corresponding operating condition information.

9. A dynamic tracking, imaging, and detection system for wind turbine blades, characterized in that, include, The data acquisition module is used to acquire wind turbine operating parameters, imaging platform status parameters, environmental parameters, and blade image data; And perform time synchronization and alignment of all source data under a unified clock reference; The state estimation module is used to perform blade phase recognition and motion state estimation based on wind turbine operating parameters and blade image data, and to establish a blade target prediction model using an extended Kalman filter fusion mechanism. The trajectory planning module is used to generate dynamic shooting trajectories, gimbal pointing parameters, and camera acquisition parameters based on the blade target prediction model and the safety envelope that dynamically changes with blade rotation speed, wind speed, and blade phase. The tracking control module is used to control the shooting platform and / or gimbal to predictively track and synchronously shoot the target blade; The image processing module is used to perform deblurring, distortion correction, brightness compensation, and spatiotemporal registration on the acquired images or videos; The defect detection module is used to identify the type, location, size, and / or severity of blade defects. The results output module is used to map defects to the blade coordinate system and output the detection results; The shooting platform can be any one of a drone, a ground mobile robot, a tower-mounted shooting device, or a cabin-mounted shooting device.

10. An electronic device, characterized in that, include: Memory, processor, and computer program stored in said memory and executable on said processor; When the processor executes the computer program, it implements the wind turbine blade dynamic tracking, shooting and detection method according to any one of claims 1 to 10.

Images