Blade dynamic three-dimensional trajectory reconstruction and clearance analysis system based on monocular vision

Abstract

The present application relates to the field of wind power generation technology, especially to a kind of blade dynamic three-dimensional trajectory reconstruction and clearance analysis system based on monocular vision, disclose a kind of blade dynamic three-dimensional trajectory reconstruction and clearance analysis system based on monocular vision, comprising: the system is by one-time self-calibration to construct geometric fingerprint model, and according to the model, after the dynamic compensation of blade elastic deformation and tower vibration, predict and only in the key event time point of minimum clearance discrete image measurement is executed, to solve clearance distance, the present application replaces continuous tracking with event-driven discrete measurement, avoids the inherent contradiction of prior art in cost computing power and reliability from the working mode;Its double compensation mechanism to blade deformation and tower vibration, also improve the measurement objectivity and adaptability of system in real physical environment.

Description

TECHNICAL FIELD

[0001] The application relates to a monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system and belongs to the technical field of wind power generation. BACKGROUND

[0002] Currently, for large equipment such as wind turbine generators, the operation safety thereof needs to be ensured, and real-time monitoring of dynamic clearance between a blade and a tower is a key link for preventing disastrous accidents. In the direction of visual monitoring technology based on image data processing, a mainstream technical method is to continuously and uninterruptedly track the motion trajectory of a blade by using a high-frame-rate camera, so as to cover all moments of possible dangerous approach.

[0003] However, when the large-scale wind farm has dual constraints of low cost and high reliability, the technical cost of the continuous tracking mode for pursuing full-time domain coverage is revealed, that is, a higher-performance hardware or computing unit needs to be configured to process redundant image data generated in most safe periods, and when the external environment has light mutation or rain and fog interference, any algorithm depending on the continuity of the tracking process may fail due to feature loss at the critical moment. Linear improvement by simply improving the hardware specifications or algorithm complexity cannot solve the structural contradiction between performance cost and robustness, but instead, it will aggravate the burden of the system. The core of the problem is that the technical method is based on the premise that the safety of the final result must be ensured by monitoring the complete process, which makes the whole monitoring system fall into the dilemma of coexistence of high consumption and high vulnerability.

[0004] Specifically, the prior art mainly has the following deficiencies: 1. There is an opposite relationship between the performance requirement of high-precision continuous three-dimensional tracking and the economic principle of large-scale low-cost deployment; 2. The system pays a high computing power consumption to process massive information redundancy, but at the moment of dangerous approach, information acquisition may fail due to environmental uncertainty; 3. All tracking algorithms depending on the continuity assumption of features have a theoretical upper limit of reliability when dealing with physical environment disturbance, and it is difficult to become an independent guarantee means. Therefore, how to avoid the technical path of continuous high-cost tracking of the blade, and develop a monitoring system aiming at reducing the cost and computing power demand and not depending on the continuity of the tracking process, which can accurately predict the occurrence moment of the risk event and only perform discrete measurement at the moment, and can also actively eliminate the error introduced by the real physical effects such as elastic deformation of the blade itself and tower vibration in the measurement, becomes a technical problem to be solved by the application. SUMMARY

[0005] The application provides a monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system, which mainly aims to solve the internal contradiction between cost, computing power and robustness caused by the dependence of the prior art on continuous tracking, and the problem that the elastic deformation of the blade and the vibration error of the tower cannot be effectively compensated in a low-cost monocular vision framework.

[0006] To achieve the above-mentioned purpose, the application provides a monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system, which comprises a monocular camera and a processing unit; the processing unit is configured to:

[0007] performing geometric fingerprint self-calibration, when the fan is in a uniform rotation working condition, collecting a periodical two-dimensional trajectory coordinate of a predetermined feature point on the blade, and combining the known length of the blade, constructing a system geometric fingerprint model defining the three-dimensional pose of the camera and the three-dimensional attitude of the blade rotation plane by fitting an ellipse to the trajectory coordinate;

[0008] performing dynamic compensation prediction, based on the system geometric fingerprint model and real-time fan operation parameters, and compensating the blade theoretical position according to a power-deformation prior model to predict a key event time point taking into account the aeroelastic deformation;

[0009] performing event-driven compensation measurement, collecting images only in a time window adjacent to the key event time point, measuring the two-dimensional image distance between the blade and the tower, compensating the two-dimensional image distance according to the camera motion amount calculated from the image background optical flow, and converting the compensated two-dimensional image distance into a three-dimensional physical clearance distance according to the system geometric fingerprint model.

[0010] Preferably, the processing unit, when performing the event-driven compensation measurement, is further configured to: define one or more image sub-regions in the image collected by the monocular camera according to the position information of the blade edge and the tower profile predicted by the system geometric fingerprint model; and perform an edge detection algorithm only in the one or more image sub-regions to measure the two-dimensional image distance between the blade edge and the tower profile.

[0011] Preferably, the processing unit is further configured to perform systematic physical lesion diagnosis: continuously record and accumulate the deviation vector between the actual observed two-dimensional image position of the blade at the key event time point and the position predicted by the system geometric fingerprint model during long-term operation of the system; and when the integral value or time sequence norm of the deviation vector within a predetermined time window exceeds a structural deformation diagnosis threshold, output a diagnosis signal representing that the system has a long-term structural position deviation.

[0012] Preferably, the real-time wind turbine operating parameters include real-time output power and blade pitch angle; the power-deformation prior model is a quantization mapping model taking real-time output power and blade pitch angle as inputs and outputting a three-dimensional spatial displacement compensation vector; and the processing unit obtains the three-dimensional spatial displacement compensation vector through the quantization mapping model to compensate the blade theoretical position.

[0013] Preferably, the processing unit is further configured to, in the geometric fingerprint self-calibration phase, automatically identify and store a stationary long shot with rich and stable features as a reference anchor point of the long shot background area at the image edge far away from the blade rotation area by analyzing the corner points and texture distribution of the image; and the camera self-motion amount is obtained by calculating the optical flow displacement vector of the long shot background area in the current image relative to the reference anchor point.

[0014] Preferably, the processing unit is further configured to perform blade aerodynamic instability online diagnosis: record a series of two-dimensional residual values between the predicted positions and the actual detected positions calculated by the processing unit in multiple consecutive measurements ; perform fast Fourier transform spectral analysis on a residual time sequence composed of the two-dimensional residual values ; and output a diagnosis signal representing that the blade is in an aerodynamic instability state when the following condition is met: wherein, represents a fast Fourier transform operation on the residual time sequence, is a preset characteristic frequency related to blade flutter, is an instability diagnosis energy threshold.

[0015] Preferably, the system further comprises an acoustic sensor and an acoustic event arbitration module; the acoustic sensor is configured to collect acoustic signals generated when the blade sweeps through the tower; the acoustic event arbitration module is configured to identify the energy peak of the acoustic signals to determine an acoustic event timestamp; compare the time difference between the acoustic event timestamp and the timestamp of the visual measurement event determined by the processing unit; and determine that the three-dimensional physical clearance distance calculated by the processing unit is invalid measurement when the time difference exceeds the interval bounded by a synchronization tolerance range.

[0016] Preferably, the processing unit is further configured to: perform sleep-wakeup control, put the image processing function of the processing unit and the monocular camera into a sleep or low-power standby state during the non-measurement period between two consecutive key event time points; and wake up the image processing function and the monocular camera to the working state within a predetermined time window before the next key event time point arrives.

[0017] Preferably, the processing unit, in performing the geometric fingerprint self-calibration phase, is further configured to: establish a static contour model representing the key region of the tower in the field of view of the monocular camera according to the result of the ellipse fitting and in combination with the camera imaging model; and, in performing the event-driven compensation measurement, utilize the static contour model to assist in locating the tower contour.

[0018] Preferably, the processing unit, in performing the dynamic compensation prediction, is further configured to: distinguish different blades of the wind turbine according to the phase angle signal in the real-time wind turbine operation parameters; and, for each blade, independently predict the key event time point at which the blade will reach the minimum distance to the tower in three-dimensional space.

[0019] Compared with the prior art, the present application has the beneficial effects that: the system provided by the present application has changed its working mode. Firstly, a geometric model containing the complete spatial relationship between the camera and the blade rotation plane is constructed in advance through one-time geometric fingerprint self-calibration. Subsequently, the system can be separated from continuous image tracking and instead performs key event prediction according to the wind turbine's own operation parameters. This prediction-measurement separation operation structure converts the continuous high-load visual processing task into a low-load discrete image measurement that occurs only at key moments, thus avoiding the high hardware and computing power costs paid to ensure full-time monitoring from the working principle. In addition, the system's reliability is no longer subject to the problems of feature loss or environmental interference that are difficult to avoid in long-term tracking, and a compensation mechanism is established to deal with real physical environment disturbances during the clearance distance calculation. The mechanism processes the two main factors affecting the measurement, i.e., the aerodynamic elastic deformation of the blade under load and the vibration of the tower under wind action, through a power-deformation prior model and a camera motion solving mechanism based on image long-range background optical flow, respectively. The two mechanisms work together to enable the system to distinguish whether the observed displacement is due to the morphological change of the blade itself or the motion of the observer itself, thereby obtaining a clearance measurement result that excludes multiple real disturbance factors without adding additional dedicated sensors. BRIEF DESCRIPTION OF DRAWINGS

[0020] Fig. 1 The overall logical flowchart of the system monitoring and health diagnosis of the present application;

[0021] Fig. 2 The quantitative relationship diagram of the blade elastic deformation and output power of the present application;

[0022] Fig. 3 The system hardware composition and data interaction between modules of the present application. DETAILED DESCRIPTION

[0023] In order to make the technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings, and it should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the protection scope of the present application.

[0024] A monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system, its operation process is designed as three logically connected stages: first, the system performs a one-time geometric fingerprint self-calibration to build a system geometric fingerprint model that defines the three-dimensional spatial relationship between the camera and the fan rotating plane; then, in the continuous operation of the system, a key event prediction module based on the model and combined with real-time operating parameters, dynamically compensates and predicts the key event of the blade about to reach the minimum clearance position in the non-image domain; finally, an event-driven ranging module only in the window adjacent to the predicted key event time point, performs a discrete image measurement containing multiple disturbance compensation to calculate the three-dimensional physical clearance distance; the application scenario of the system is mainly for the operation safety monitoring of large horizontal axis wind turbine generators on land or at sea, aiming to cope with how to analyze and warn the dynamic clearance between the blade and the tower in a low-cost and high-reliability manner under complex physical environment; in the grid-connected operation state, the wind turbine blade tip speed can reach dozens of meters per second, and the blade will also be elastically deformed due to aerodynamic load, and the tower carrying the camera will also vibrate at low frequency in strong wind, these common physical phenomena together pose technical challenges to the reliability of the monitoring system and the objectivity of the measurement results; in order to establish the geometric basis for all subsequent predictions and measurements, the system performs a geometric fingerprint self-calibration at power-on initialization or in the condition of low-speed uniform rotation of the fan, for example, the speed is maintained at 5 rpm; under the premise of not relying on external calibration equipment or complex manual operation, accurately solving the extrinsic parameters of the monocular camera in the tower coordinate system and the attitude of the blade rotating plane in space is the technical problem to be solved at this stage; to solve this problem, the geometric fingerprint self-calibration module configured by the present application has the following procedures: a monocular camera installed at a fixed position on the tower continuously acquires an image sequence of the blade rotating at least one complete cycle in the wind wheel plane at a preset frame rate, under this low-speed, uniform-speed rigid body motion condition, the processing unit stably identifies and records the two-dimensional coordinates of a predetermined feature point on the blade, such as the blade tip or the navigation light, in each frame of image Since the physical trajectory of the blade tip is a space circle, its projection on the camera imaging plane is an ellipse, after the processing unit acquires a predetermined number of two-dimensional coordinate points, it performs a least squares ellipse fitting algorithm to solve the five parameters of the projection ellipse,

[0025] In the specific procedure of geometric fingerprint self-calibration, the processing unit first applies the FAST algorithm, which is an algorithm for extracting the position of image corners at high speed, on the predetermined contour line segment radially outside the blade in each frame of image in parallel to obtain the position coordinates of a set of candidate feature points; then, for each candidate feature point , an LBP operator, which is an operator for describing the texture information of the neighborhood of a pixel point, is calculated for the neighborhood of the candidate feature point, and only those points whose LBP texture variance is greater than a preset texture complexity threshold are retained, and this step is used to filter out feature points that lack stable tracking information due to surface smoothness or uniform illumination; finally, the set of high-quality feature points filtered in all image frames is taken as input, and a RANSAC algorithm framework, which is an algorithm framework for estimating the parameters of a mathematical model through iterative random sampling and excluding incidental abnormal data points, is enabled to fit a projection ellipse model that can obtain the most inliers, and five ellipse parameters determined by the model are directly solidified as the core part of the geometric fingerprint model of the system regarding the camera pose and the blade attitude; since the actual length of the blade is a known fan design parameter, the processing unit takes the ellipse parameters and the length of the blade as input through a certain mathematical model to inversely solve the geometric parameters of the system at one time, and solidifies them into a system geometric fingerprint model, which at least includes: the three-dimensional position and attitude of the camera relative to the center of rotation of the blade, the three-dimensional normal vector of the blade rotation plane, and a static contour model representing the key region of the tower in the image, so that the self-calibration procedure converts the spatial calibration problem into an automatic geometric solving process based on the motion of the system itself, thereby providing a unified geometric reference for all subsequent calculations of the system.

[0026] After the system geometric fingerprint model is established, the next technical problem is that the blade, as an elastic body, will be deformed in waving and twisting under the aerodynamic load in actual high-power operation, so that the actual position of the blade deviates from the theoretical prediction of the geometric fingerprint model based on the rigid body assumption, and this deviation is particularly significant when the fan load is at the highest risk and the consequences are the most serious. The dynamic compensation prediction module of the system introduces a power-deformation prior model and performs key event prediction based on the model. The model is a quantitative model that maps the real-time operation parameters of the fan to the three-dimensional displacement compensation vector of the blade tip through offline calibration or according to the fan design data, and its carrier can be a lookup table or a polynomial function. The operation procedure is as follows: in the prediction period, the processing unit obtains the real-time output power and the blade pitch angle , the processing unit queries the power-warping prior model with these two parameters as inputs, and obtains a corresponding three-dimensional spatial displacement compensation vector , accordingly, the processing unit first calculates the theoretical position of the blade under the rigid body assumption based on the geometric fingerprint model, the rotational speed and the phase angle signal, and then superimposes the three-dimensional spatial displacement compensation vector on the theoretical position to finally predict a key event time point , i.e. the blade tip will reach the three-dimensional spatial minimum distance point from the tower at this time; in this way, the system converts a complex real-time fluid-structure coupling analysis problem into a low-computational-cost real-time position correction operation based on conventional electrical parameters, improving the accuracy of the prediction result under real load.

[0027] After predicting the key event time point , the system faces a third technical problem, i.e. under strong wind, the tower itself will produce low-frequency vortex-induced vibration, causing the camera platform serving as the measurement reference to displace, and if not compensated, this amount of camera motion will be included in the final clearance measurement result, possibly leading to misjudgment, therefore, the event-driven compensation measurement module only wakes up within the time window adjacent to the predicted key event time point , e.g. 50 milliseconds in advance, collects 10-20 frames of images, and performs a discrete measurement integrated with camera motion compensation once, the compensation procedure of which is as follows: in the self-calibration stage, the processing unit has automatically identified and stored an image sub-region representing a static landscape as a reference anchor point in the area far from the blade rotation by analyzing the image corner points and texture distribution, when constructing the landscape background reference anchor point for compensating the tower vibration, the system first enters a baseline calibration mode, which requires continuous image sequence collection for a time period not less than 300 seconds under the condition that the wind turbine is completely static, e.g. during the night period without wind; in this static baseline sequence, for the preselected landscape background area, a KLT optical flow tracker, which is an algorithm for calculating the optical flow displacement of sparse feature points in the image sequence, continuously calculates the optical flow displacement vector of all stable feature points in the area ; since any apparent motion under this working condition is only due to the electronic noise of the camera image sensor itself, the processing unit calculates the statistical mean and the standard deviation of the amplitudes of all optical flow displacement vectors, and determines a micro-motion threshold , which statistically defines the maximum apparent displacement that can be produced by non-true physical motion; in subsequent regular operation, the system only wakes up when the optical flow energy calculated for a candidate landscape area is significantly higher than the micro-motion threshold The procedure will be based on a physical and statistical foundation, when the measurement module is woken up and starts to acquire images, the processing unit will perform two tasks in parallel: one is to measure the 2D pixel distance between the blade and the tower outline within the image sub-region predicted by the geometric fingerprint model The second is to calculate the optical flow displacement vector of the locked long-range background region relative to the reference anchor point in the current image by sparse optical flow or template matching algorithm Since the physical long-range background is static, the vector represents the 2D motion of the camera itself caused by the tower vibration; before conversion, the processing unit will use the optical flow displacement vector to inversely compensate the measured 2D image coordinates of the blade and the tower to eliminate the displacement introduced by the camera motion, and then convert the compensated 2D image distance to 3D physical clearance distance according to the projection relationship in the system geometric fingerprint model This mechanism realizes adaptive optical compensation of the observer's own motion without increasing inertial sensors by using background information in the image.

[0028] In addition, the processing unit of the present application is also configured to perform system health diagnosis and online diagnosis functions, one is systematic physical lesion diagnosis, in the long-term operation of the system, the processing unit continuously records and accumulates the deviation vector between the observed 2D image position of the blade and the model predicted position at the key event time point, when the integral value of the deviation vector within a predetermined time window or its time series norm exceeds a set structural deformation diagnosis threshold, the system outputs a diagnosis signal representing the permanent structural deformation of the blade or the tilt of the tower, thereby converting the systematic deviation between the model and reality into diagnostic information for structural health; the second is the online diagnosis of blade aerodynamic instability, the processing unit records a series of 2D residual values between the predicted position and the actual detected position calculated in multiple consecutive measurements and performs fast Fourier transform spectral analysis on the residual-error time series composed of the residual values When the energy value at a certain preset characteristic frequency related to blade flutter exceeds a preset instability diagnosis energy threshold , the system outputs a diagnostic signal indicating that the blade is in an aerodynamic instability state, which utilizes residual information that is originally considered as measurement noise to provide a way to characterize the dynamic stability of the blade; to improve the reliability of system decision-making, the system can further include an acoustic event arbitration module, which includes an acoustic sensor for collecting acoustic signals generated when the blade sweeps the tower, and the acoustic event arbitration module identifies an energy peak of the signals to determine an acoustic event timestamp , and compares it with the timestamp of the visual measurement event determined by the processing unit , and when the absolute value of the time difference between the two exceeds a preset synchronization tolerance range, it is determined that the three-dimensional physical clearance distance at this time is invalid measurement, and this mechanism provides a decision arbitration way for the system to deal with rare optical interference events encountered by the visual sensor by introducing an independent source based on different physical principles for cross-validation; in terms of operation, the processing unit also performs sleep-wake control, and during the non-measurement period between two consecutive key event time points, the image processing function and the monocular camera are in a sleep or low-power state, and are only awakened to the working state within a predetermined time window before the next key event time point arrives, which reduces the redundant data processing and computing power consumption of the system during the non-key period.

[0029] Embodiment 1: In a specific application of a wind turbine generator set located in a high-altitude coastal area that is being subjected to strong winter storms and turbulent flow head-on impact, the set is running at near full power, accompanied by irregular gusts caused by atmospheric turbulence; under such a combination of high load and strong turbulence, monitoring the running clearance between the blade and the tower faces two technical problems: first, the aerodynamic load causes the blade to produce considerable elastic deformation in the flapwise direction, causing its actual motion trajectory to deviate from the prediction based on the ideal rigid body model; second, the turbulent wind field causes the tower structure to produce low-frequency vibration, so that the monocular camera platform installed thereon is in a state of continuous motion, and any visual measurement result without compensation is the coupling of the actual motion of the blade and the motion of the camera itself; during operation, the image processing function of the system is in a sleep state most of the time; the key event prediction module in the processing unit continuously obtains real-time operating parameters from the wind turbine SCADA system, and when the obtained real-time output power and the blade pitch angle are in a high position, the module calculates the theoretical position of the blade according to the system geometric fingerprint model and the real-time phase angle, and the built-in power-deformation prior model is called, and according to the input and values, a three-dimensional spatial displacement compensation vector for compensating the deformation of the blade under this working condition is output , which is used to correct the theoretical tip position of the blade, so as to predict the critical event time point ; this dynamic compensation based on the operating condition makes the prediction of the critical event time point reflect the actual shape change of the blade under load.

[0030] At a preset window before the predicted critical event time point , the processing unit and the monocular camera are woken up and an image sequence is collected; the event-driven compensation measurement module processes two pieces of information in parallel at this time: in the image sub-region containing the blade and the tower contour predicted according to the geometric fingerprint model, the two-dimensional pixel distance between them is measured; at the same time, in a static reference anchor region located at the far end of the field of view determined in the previous self-calibration stage, the camera's own optical flow displacement vector caused by the tower vibration is calculated ; before distance conversion, the system first uses the optical flow displacement vector to compensate the two-dimensional image coordinates of the blade and the tower, so as to separate the motion of the observation platform in calculation, and then according to the geometric projection relationship, the three-dimensional physical clearance distance is calculated from the compensated two-dimensional pixel distance ; this working mode reduces the risk of tracking interruption caused by sudden changes in light or rain and fog blocking in continuous image tracking in bad weather; the three-dimensional physical clearance distance obtained in this process is a measurement value that has taken into account both the elastic deformation of the blade and the tower vibration, the system confirms the operating condition of the wind turbine under this condition, avoids the risk of misjudgment due to uncompensated deformation, and avoids the distortion of the measurement value caused by the tower vibration; the working mode of the system converts a measurement problem coupled with multiple dynamic variables into a discrete geometric measurement problem based on model prediction and compensation at a specific time.

[0031] Example 2: To quantitatively verify the measurement accuracy of the technical solution of the present application under the working condition of combined interference including blade elastic deformation and tower vibration, a semi-physical simulation test platform is used in this example, which is composed of a monocular camera, a graphics workstation for generating high-fidelity visual scene of fan operation, and a processing unit. The graphics workstation renders real-time three-dimensional dynamic images of a standard 2.5 MW wind turbine under specific working conditions according to the dynamic model of the wind turbine, for the monocular camera to collect. The purpose of the test is to compare the deviation between the three-dimensional physical clearance distance measurement results of the technical solution of the present application and a solution that only includes the basic geometric fingerprint model without integrating dynamic compensation function, and the geometric true value provided by the simulation model when facing the same dynamic disturbance. The test working condition is set as follows: the fan in the simulation model operates at 80% rated power output state. Under this working condition, a static displacement of 1.2 m in the flapwise direction is applied to the tip of the blade to simulate aeroelastic deformation. At the same time, to simulate the vibration of the tower, a sinusoidal vibration displacement with a frequency of 0.5 Hz and an amplitude of ±0.5 m is applied to the virtual camera pose of the simulation visual scene. The vibration frequency is set to be close to the first-order resonance frequency of a typical hundred-meter tower, and the vibration amplitude is set to correspond to the engineering range of wind-induced vibration under this wind speed level. The system of the test group and the control group has completed the self-calibration of the system geometric fingerprint by pre-processing the running visual scene of the simulation model under the low-speed uniform speed working condition.

[0032] After the experiment started, both systems simultaneously processed the simulated scene and recorded their respective output three-dimensional physical clearance distance measurements at a frequency of 10Hz, along with the geometric true value provided by the simulation system as a reference. During continuous observation, the control group's measurements showed significant systematic deviations and periodic fluctuations. For example, at two consecutive key event points where the geometric true value was 5.50m, the measured values ​​were 4.31m and 4.78m, respectively. In contrast, the experimental group's measurements at the same times were 5.53m and 5.48m, showing a higher consistency with the geometric true value. Comparing the two sets of data with the geometric true value revealed that the control group's measurement results were systematically smaller by approximately 1.2m. This deviation is consistent with the blade deformation set in the simulation model. Correspondingly, its readings fluctuate by approximately ±0.25m, which is consistent with the projection of the camera displacement caused by tower vibration onto the imaging plane. The measurement results of the experimental group show that its built-in power-deformation prior model effectively compensates for the systematic measurement deviation introduced by blade deformation due to high-power operation, while its camera motion calculation mechanism based on background optical flow suppresses the measurement fluctuations introduced by tower vibration. The experimental results confirm that the system disclosed in this invention, through the synergistic effect of its built-in predictive compensation for blade aeroelastic deformation and measurement compensation for tower vibration, can provide a clearance distance measurement result that is closer to the geometric true value of the measured object under dynamic operating conditions containing the above two physical disturbances.

[0033] Example 3: This example combines Figs. 1 to 3 This section describes a system for reconstructing dynamic 3D trajectories of blades and analyzing clearance based on monocular vision. Fig. 1 As shown, the process begins by acquiring a sequence of rotating blade images under constant wind turbine speed conditions. This sequence is then used as input to perform geometric fingerprint self-calibration, thereby constructing a system geometric fingerprint model that includes key information such as camera pose, blade disk attitude, and tower profile. During normal system operation, this model, combined with real-time wind turbine operating parameters acquired from external sources, such as output power and blade pitch angle, is used to perform dynamic compensation prediction to estimate a critical event time point that takes into account blade elastic deformation. Subsequently, the system performs an event-driven compensation measurement only near the predicted time point. After compensating for the camera's own motion, this measurement calculates the three-dimensional physical clearance and ultimately outputs clearance analysis and early warning monitoring results. Simultaneously, the deviation data generated by the prediction and measurement stages are used for two parallel diagnostic branches. First, by analyzing the long-term accumulated position deviation, systemic physical lesion diagnosis is performed, and a diagnostic signal for structural deformation / tower tilt is output when the deviation exceeds a threshold. Second, by analyzing the characteristic frequency energy of the residual sequence, online diagnosis of blade aerodynamic instability is performed, and a diagnostic signal for blade aerodynamic instability is output when the energy exceeds a threshold.

[0034] likeFig. 2 As shown in the figure, the horizontal axis represents the output power (%), and the vertical axis represents the deformation (in meters). The figure contains two curves: the solid line composed of square data points represents the trend of blade deformation in the flapping direction as a function of output power, while the dashed line composed of triangular data points represents the trend of blade torsional deformation as a function of output power. This graph shows that the main elastic deformation of the blades all exhibit non-linear growth with the increase of the wind turbine's output power, thus providing the system with a basis for real-time power parameters. With blade pitch angle parameters A quantitative data model of the system is constructed by accurately and dynamically compensating for the theoretical position of the blade.

[0035] like Fig. 3 As shown, the entire system consists of wind turbine field equipment deployed on-site, a remote cloud / central server, and the wind turbine's own SCADA system. The wind turbine field equipment is the core, including a monocular camera for acquiring image signals and an acoustic sensor for acquiring acoustic signals. Data from both sensors is sent to the edge computing node processing unit. This processing unit runs core processing software embedded with geometric fingerprint and power deformation models. The edge computing node obtains the operating condition data required for dynamic compensation by querying the wind turbine SCADA system for real-time operating parameters, and uploads the calculated clearance distance and diagnostic results to the cloud / central server operation and maintenance monitoring platform for remote monitoring and analysis.

[0036] Example 4: This example discloses an engineering procedure for offline parameter calibration and model building for a specific wind turbine model before applying the system of the present invention to that model. This procedure is used to trace and quantify the data source when key models and key thresholds in the core algorithm of the system are applied to specific hardware models. In the initial stage of this procedure, technicians use the computer-aided design model of the new wind turbine and its material physical property parameters to build an aeroelasticity simulation model of the wind turbine in a finite element analysis software environment. This procedure aims to transform the discrete data representing the physical response of the blades under different operating conditions generated by the simulation model into a power-deformation prior model that can be efficiently queried in the processing unit and can cover the operating range. To build this model, technicians perform steady-state simulations covering multiple discrete operating points within the predetermined operating range of the wind turbine in a finite element analysis environment. Each set of simulation operating conditions is represented by a unique real-time output power. With blade pitch angle Defined by combination; under each simulation condition, record the three-dimensional spatial displacement compensation vector of the blade tip relative to its position under no-load condition after the simulation model stabilizes. After completing the simulation covering the predetermined operating condition matrix, a series of discrete data sets will be obtained. , for filling a two-dimensional look-up table, the input dimension of which is power and pitch angle, and the node value of which is the corresponding displacement compensation vector; for the working condition parameters encountered in online operation that are not on the nodes of the look-up table, the processing unit adopts a bilinear interpolation algorithm to calculate the compensation vector according to the data of the four adjacent nodes, thereby generating a static data model that can be called by the embedded system, which maps a specific operating condition to the corresponding displacement compensation vector.

[0037] To determine the instability diagnosis energy threshold for online diagnosis of blade aerodynamic instability , the skilled person first determines a characteristic frequency related to flutter of the blade of the type by modal analysis of the finite element model; subsequently, the system of the application is caused to process a simulation scene simulating operation of the fan in a wind field without instability phenomenon and to collect a sequence of two-dimensional residual values output continuously over a period of 300 seconds ; the sequence is subjected to fast Fourier transform, and a statistical average of the amplitude of the energy spectrum line at the characteristic frequency is calculated ; finally, the instability diagnosis energy threshold is set to , which provides a statistically based criterion for distinguishing between normal disturbance and abnormal vibration for subsequent online diagnosis; by executing the above procedure, a system software instance containing a power-deformation prior model with completed parameter calibration for a specific fan type and an instability diagnosis energy threshold is generated, which, after being deployed to physical hardware in the field, can directly execute its established dynamic compensation prediction and online diagnosis functions.

[0038] Embodiment 5: This embodiment discloses a set of initial baseline calibration and diagnosis threshold setting procedures executed by the system before it is put into formal operation after completing physical installation and self-calibration of the geometric fingerprint, for adapting the diagnosis and arbitration functions of the system to the initial physical characteristics and environmental noise characteristics of each fan as its health baseline; in this procedure, the system is placed in an online calibration mode for a period of 100 impeller rotation cycles, during which the fan is stably operated in a working condition below 50% of the rated power without turbulence; the processing unit synchronously records the visual measurement event timestamps output by the event-driven compensation measurement module and the acoustic event timestamps perceived by the acoustic sensor during this period, and calculates the statistical distribution of the time difference between them to obtain the mean and the standard deviation ; subsequently, the boundaries of the synchronization tolerance range for acoustic event arbitration are set to , and time differences exceeding this range in subsequent online measurement will be judged as invalid measurements.

[0039] During the same calibration run, the processing unit also records the sequence of deviation vectors between the actual observed 2D image position of the blade tip and the predicted position by the system geometric fingerprint model at each key event time point, and calculates the statistical baseline features of this sequence, including the mean value of the vector magnitude and the standard deviation ; correspondingly, the structural deformation diagnostic threshold for systematic physical pathology diagnosis is set as a quantitative condition, i.e. when the sliding average of the deviation vector magnitude over N measurement cycles continuously exceeds , a diagnostic signal is triggered; the structural deformation diagnostic threshold thus set, constitutes the quantitative decision criterion for distinguishing normal measurement deviation from permanent structural deformation in subsequent online diagnosis.

[0040] Embodiment 6: This embodiment discloses a set of model verification and fault-tolerant generation procedures that the system executes to construct the high-confidence geometric and compensation references that the system relies on for its subsequent online measurement and diagnostic functions; in constructing the distant background reference anchor that serves as the vibration compensation reference, the system needs to verify the physical stationary property of the selected region, especially in deployment scenarios where clear stationary distant background is lacking within the camera field of view; for this purpose, after the system performs geometric fingerprint self-calibration, it enters a background region validity self-check mode, in which it selects a candidate distant background region at the edge of the field of view and continuously calculates the dense optical flow field within this region for a pre-set time duration; if the average value of the calculated optical flow field vector magnitude exceeds a micro-motion threshold value that is associated with the camera sensor noise level, the system determines that the candidate region is not a stationary background, and automatically disables the compensation function based on background optical flow, while recording a system status log that indicates that the subsequent measurement results do not include compensation for camera platform vibration; this procedure establishes a verifiable decision basis for whether to enable vibration compensation in the future.

[0041] In constructing the geometric fingerprint model that is the core of the system, to cope with the working conditions where the pre-defined feature points are difficult to be continuously identified due to illumination or blade surface contamination, the system enables a fault-tolerant generation procedure; when acquiring each frame of image of the blade tip trajectory, the processing unit extracts multiple candidate feature point coordinates on the pre-defined contour line segment outside the blade radius, rather than relying on a single optimal feature point; in performing ellipse fitting, the system uses a random sample consensus algorithm to iteratively randomly select a subset of candidate points for ellipse fitting, and takes the ellipse parameters that obtain the most inlier support as the final geometric fingerprint model parameters; this procedure makes the generation of the geometric fingerprint model immune to the temporary loss or erroneous identification of a small number of candidate feature points, thereby providing a more robust geometric reference for the entire system.

[0042] It is apparent for a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments, but that the application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the application.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not to limit the present application, and although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the present application.

Claims

1. A monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system, characterized in that, The system comprises a monocular camera and a processing unit; the processing unit is configured to: perform geometric fingerprint self-calibration, when the fan is in a uniform rotation condition, collect a two-dimensional trajectory coordinate of a predetermined feature point on the blade in a cycle, and combine the known length of the blade, through elliptical fitting of the trajectory coordinate, to construct a system geometric fingerprint model defining the three-dimensional pose of the camera and the three-dimensional attitude of the blade rotation plane; perform dynamic compensation prediction, based on the system geometric fingerprint model and real-time fan operating parameters, and according to a power-deformation prior model, to compensate the theoretical position of the blade, so as to predict a key event time point taking into account the aeroelastic deformation; perform event-driven compensation measurement, only in a time window adjacent to the key event time point, to collect images, measure the two-dimensional image distance between the blade and the tower, and compensate the two-dimensional image distance according to the camera motion quantity calculated from the optical flow of the distant background in the image, and then convert the compensated two-dimensional image distance into a three-dimensional physical clearance distance according to the system geometric fingerprint model; the processing unit, when performing the event-driven compensation measurement, is further configured to: according to the position information of the blade edge and the tower profile predicted by the system geometric fingerprint model, define one or more image sub-regions in the image collected by the monocular camera; and perform an edge detection algorithm only in the one or more image sub-regions to measure the two-dimensional image distance between the blade edge and the tower profile; the processing unit is further configured to perform systematic physical lesion diagnosis: in the long-term operation of the system, continuously record and accumulate the deviation vector between the actual observed two-dimensional image position of the blade at the key event time point and the position predicted by the system geometric fingerprint model; and when the integral value or time sequence norm of the deviation vector within a predetermined time window exceeds a structural deformation diagnosis threshold, output a diagnosis signal representing that the system has a long-term structural position deviation; the real-time fan operating parameters include real-time output power and blade pitch angle; the power-deformation prior model is a quantitative mapping model taking the real-time output power and blade pitch angle as input and outputting a three-dimensional space displacement compensation vector; the processing unit obtains the three-dimensional space displacement compensation vector through the quantitative mapping model to compensate the theoretical position of the blade.

2. The monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system according to claim 1, characterized in that, the processing unit is further configured to: in the geometric fingerprint self-calibration stage, by analyzing the corner points and texture distribution of the image, automatically identify and store a static distant view with rich and stable features as a reference anchor point of the distant background area at the edge of the image away from the blade rotation area; and the camera motion quantity is obtained by calculating the optical flow displacement vector of the distant background area in the current image relative to the reference anchor point.

3. The monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system according to claim 1, characterized in that, The processing unit is also configured to perform online diagnosis of blade aerodynamic instability: recording a series of two-dimensional residual values ​​between predicted positions and actual detection positions calculated by the processing unit in multiple consecutive measurements. For two-dimensional residual values The resulting residual time series Perform Fast Fourier Transform (FFT) spectral analysis; and output a diagnostic signal characterizing the blade's aerodynamic instability state when the following conditions are met: ,in, This indicates that a Fast Fourier Transform operation is performed on the residual time series. The preset characteristic frequency related to blade flutter, This is a threshold energy for instability diagnosis.

4. The monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system according to claim 1, characterized in that, The system further comprises an acoustic sensor and an acoustic event arbitration module; the acoustic sensor is configured to collect acoustic signals generated when the blade sweeps the tower; the acoustic event arbitration module is configured to identify the energy peak of the acoustic signal to determine an acoustic event timestamp; comparing a time difference between the acoustic event timestamp and a timestamp of a visual measurement event determined by the processing unit; and determining the three-dimensional physical clearance distance resolved by the processing unit as an invalid measurement when the time difference exceeds an interval bounded by a synchronization tolerance range.

5. The monocular vision-based blade dynamic three-dimensional trajectory reconstruction and clearance analysis system according to claim 1, wherein, The processing unit is further configured to perform a sleep-wake control to put the image processing function of the processing unit and the monocular camera into a sleep or low-power standby state during a non-measurement period between two consecutive key event time points; and to wake up the image processing function and the monocular camera to an active state within a predetermined time window before the next key event time point arrives.

Images