A tower clearance real-time monitoring method combining millimeter wave radar and video

Abstract

The application provides a millimeter wave radar and video fusion tower clearance real-time monitoring method, which comprises a video monitoring camera system and a millimeter wave radar system; first, the two systems are installed at the bottom of the cabin and rotated to point to the blade clearance area, realizing image acquisition and radar reflection point data acquisition in the blade movement cycle; then the controller system calculates the blade tip clearance value according to the collected image data and radar reflection points, qualitatively evaluates the deviation of the two according to the spatial relationship between the blade tip trajectory line and the blade tip reflection point, and ensures the high-precision clearance monitoring of the blade tip and the tower. The clearance monitoring method of the fusion of the two sensors can realize the functions of mutual checking of the accuracy between the systems in normal weather, selection of the clearance for unit control in severe weather, stronger environmental adaptability and working condition adaptability, and avoids the problems of failure of a single sensor due to environmental influence or low clearance accuracy.

Description

Technical Field

[0001] This invention relates to a method for real-time monitoring of the blade clearance value of wind turbine generator sets, and in particular to a method for calculating the blade clearance value by fusing multiple sensors, including video surveillance cameras and millimeter-wave radar. Background Technology

[0002] During operation, large-bladed, high-tower wind turbines are prone to accidents due to the flexible tower and lightweight blade design. Under extreme conditions, the dynamic clearance between the blades and the tower may fall below the minimum design clearance, leading to a serious "blade-tower sweep" accident. This can result in anything from blade replacement and tower repair to complete turbine collapse, causing significant property damage and power loss. Therefore, real-time monitoring of the blade tip clearance between the blades and the tower is crucial for proactive control and ensuring turbine safety.

[0003] Existing airspace monitoring solutions include three-line lidar airspace monitoring, video airspace monitoring, and millimeter-wave radar airspace monitoring. Each monitoring solution is subject to environmental interference. Factors that interfere with millimeter-wave radar include rain and snow, while those that interfere with lidar include rain, snow, fog, and dust storms. Factors that interfere with video surveillance cameras include dense fog and patchy fog.

[0004] Currently, in the field of intelligent driving, there are publicly available applications proposing methods for joint calibration of millimeter-wave radar and cameras using homography matrix calculation and Zhang Zhengyou camera calibration. However, these methods assume that the millimeter-wave radar outputs three-dimensional spatial coordinate information, while the actual radar output dimensional information has a large deviation in the height dimension. Furthermore, the millimeter-wave radar used in the field of wind turbine safety control outputs two-dimensional plane coordinate information, which further limits the application of joint calibration. There are also publicly available applications proposing a scheme in which video monitoring devices and radar monitoring devices jointly participate in clearance calculation, which improves the environmental adaptability of the entire clearance monitoring system. However, subjective and objective factors have a significant impact on the clearance results, which manifests as the need to accumulate sufficient data for objective calculation of confidence level and subjective calculation of confidence level based on experience, and then combine the subjective and objective confidence levels to calculate the final clearance result. Summary of the Invention

[0005] This invention addresses the shortcomings of current airspace monitoring schemes by proposing a multi-sensor fusion method for monitoring the airspace of wind turbine generators based on video monitoring cameras and millimeter-wave radar. It also proposes a method for calculating the deviation between two monitoring systems, using both quantitative and qualitative analysis, to solve the problems of single airspace monitoring devices failing due to environmental probabilistics and the need for multiple sensors to accumulate data for subjective and objective judgments on airspace accuracy.

[0006] The technical solution adopted to achieve the above-mentioned objectives of this invention is as follows:

[0007] A method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video, comprising the following steps:

[0008] S1: Install the camera system on the left or right front side of the bottom of the wind turbine nacelle, and adjust its attitude so that its field of view (FOV) faces the area between the tower and the blades, so that the camera system can capture images of the blades and the tower at the same time.

[0009] S2: The image recognition algorithm is used to calculate the captured image information, identify the movement trajectory points of the blade tip, fit the blade tip movement trajectory points identified by the image using the least squares method to obtain the blade tip movement trajectory straight line, and further obtain the blade tip movement trajectory straight line during the low speed and stable speed period of the wind turbine.

[0010] S3: In the horizontal projection plane at the same height as the blade tip, the straight line of the blade tip motion trajectory during the low speed and stable speed period of the wind turbine is translated to be tangent to the outer circumference of the tower. The tangent is used as the baseline and the tangent point is used as the tower feature point. A two-dimensional coordinate system is constructed in the horizontal projection plane at the same height as the blade tip, with the tower feature point as the origin. The vertical distance between the tower feature point and the straight line of the blade tip motion trajectory obtained in real time is calculated. This distance is the video monitoring clearance value.

[0011] S4: Install the millimeter-wave radar system on the bottom left rear side of the wind turbine nacelle, and adjust its attitude to face the area between the tower and the blades, so that the millimeter-wave radar system can monitor the horizontal two-dimensional radar point cloud data of the blades at a fixed azimuth angle.

[0012] S5: In the two-dimensional coordinate system of S4, the two-dimensional coordinates at the blade tip are calculated by least squares curve fitting and linear integration, and the radar monitoring clearance value is calculated based on the two-dimensional coordinates at the blade tip.

[0013] S6: In the two-dimensional coordinate system of S3, the distance deviation between the video monitoring clearance value and the radar monitoring clearance value is quantitatively calculated as the system deviation. At the same time, the spatial relationship between the straight line of the blade tip motion trajectory and the two-dimensional coordinate information at the blade tip is qualitatively analyzed.

[0014] S7: Compare the distribution characteristics of motion trajectory points in S2 with the quality of fitting residuals in S5. If one of the distribution characteristics or the quality of fitting residuals is poor, select the other result for output, that is, select the video monitoring net value or the radar monitoring net value as the net monitoring result; when both have good distribution characteristics and fitting residual quality, perform inter-system mutual calibration and early warning of excessive deviation.

[0015] Furthermore, in S2, a straight line is selected for the blade tip movement trajectory when the blade rotation speed is less than 3 rpm.

[0016] Furthermore, in S4, as the blade enters and leaves the field of view of the millimeter-wave radar, 3 to 7 frames of radar point cloud are collected according to the acquisition frequency of the millimeter-wave radar. The number of blade point clouds collected by the radar conforms to the characteristic of first increasing and then decreasing. The local maximum search algorithm is used to find the peak moment of the number of point clouds in adjacent 7 frames, thereby obtaining the complete point cloud data of a blade in the radar field of view.

[0017] Furthermore, in S5, when using the least squares curve fitting method, the blade tip two-dimensional coordinates are calculated by linear integration from the blade root based on the fitted curve, and the Y value of the blade tip two-dimensional coordinates is calculated as the radar monitoring clearance value.

[0018] Furthermore, the system deviation described in S6 is affected by both the slope of the straight line of the blade tip motion trajectory and the two-dimensional coordinates of the blade tip point, and this system deviation is regarded as the inherent deviation between systems.

[0019] Furthermore, during the qualitative analysis in S6, if the straight trajectory of the blade tip passes through the blade tip point fitted by the millimeter-wave radar, it is determined that there is no deviation between the video monitoring clearance and the millimeter-wave radar monitoring clearance during the initial calibration process; otherwise, it is determined that there is a calculation deviation introduced during the initial calibration process of either the video monitoring clearance or the millimeter-wave radar monitoring clearance.

[0020] Furthermore, the distribution characteristics of motion trajectory points in S2 and the quality of fitting residuals in S5 were both good in S7 under normal weather conditions. The accuracy of the two was cross-checked, and the video monitoring airspace was selected as the final result.

[0021] Furthermore, in S7, under low visibility weather conditions such as dense fog and patchy fog, the distribution characteristics of motion trajectory points monitored by video are irregular, the blade trajectory points are discontinuous and there are abnormal points and outliers that deviate from the blade tip trajectory points, and the straight-line fitting quality of the blade tip motion trajectory is not high. Therefore, the video monitoring clearance is deemed to be invalid, and the radar monitoring clearance value is selected as the clearance monitoring result.

[0022] Furthermore, in S7, under weather conditions with heavy rain and snow, the two-dimensional radar point cloud data of the blades monitored by the millimeter-wave radar is mixed with rain and snow noise. At this time, the number of blade point clouds monitored by the millimeter-wave radar and the quality of the multinomial fitting of the blade attitude are not high. The blade attitude fitting line does not conform to the blade morphology rules, and it is determined that the radar monitoring clearance is invalid. The video monitoring clearance value is selected as the clearance monitoring result.

[0023] Furthermore, if the distribution characteristics of the motion trajectory points in S2 and the quality of the fitting residuals in S5 are both poor, then invalidity is taken as the final net result in S7.

[0024] Compared with the prior art, the technical solution provided in this application has the following advantages:

[0025] 1. This application uses a video monitoring camera to capture images of the blades, ensuring data continuity and intuitive visibility. It uses millimeter-wave radar to monitor the blade targets and calculates the blade attitude based on the monitored blade target points, ensuring data interpretability.

[0026] 2. In this application, the results of the air clearance identified by the video monitoring camera and the results of the air clearance monitored by the millimeter-wave radar are converted to a horizontal plane at the same height as the blade tip, so as to provide a standard for comprehensively evaluating the results of the two systems.

[0027] 3. This application uses quantitative and qualitative analysis to analyze the deviations between the video monitoring system and the millimeter-wave monitoring system, thereby decomposing the inherent deviations of the two systems and the deviations between the systems under normal weather conditions, and effectively evaluating the accuracy and precision of the net air value of the two systems.

[0028] 4. In this application, the effectiveness of the current air clearance can be determined by identifying the blade tip trajectory line based on the video monitoring of the blade image. If there are abnormal points or high leverage points in the blade trajectory points, the air clearance fails. The effectiveness of the current air clearance can also be determined by curve fitting based on the target reflection points of the blade monitored by millimeter wave radar. If the blade attitude fitting line does not conform to the blade morphology rules, the air clearance fails. Factors causing video monitoring equipment failure include dense fog, patchy fog, heavy rain, and heavy snow, while factors causing millimeter wave monitoring equipment failure include rain and snow. Integrating the two for air clearance calculation can improve the environmental adaptability of the entire monitoring system and enable cross-checking of air clearance accuracy between systems under normal weather conditions. The air clearance calculation scheme combining video surveillance cameras and millimeter wave radar proposed in this application can solve the problem of air clearance failure in most environments, only failing in dense fog and heavy rain / snow conditions. In these cases, a characteristic air clearance value is output, and it is recommended that the wind turbine operate at limited power. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the process of integrating video surveillance system and millimeter wave monitoring system for airspace monitoring provided in an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the camera field of view (FOV) provided in an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the image recognition blade tip motion trajectory points and the fitted blade tip motion trajectory lines in the top view of a wind turbine provided in an embodiment of the present invention;

[0032] Figure 4 This is a schematic diagram of the fitting of the blade point cloud and blade attitude curve in the side view of the wind turbine provided in this embodiment of the invention;

[0033] Figure 5 This is a schematic diagram of the millimeter-wave radar monitoring of the blade point cloud and the calculation of the distance between the blade tip and the central axis of the tower in the front view of the wind turbine provided in the embodiment of the present invention;

[0034] Figure 6 This is a schematic diagram of calculating the blade tip coordinates by fitting a millimeter-wave point cloud curve in a top view of a wind turbine, provided in an embodiment of the present invention.

[0035] Figure 7 This is a schematic diagram showing the relationship between the straight line of the blade tip movement trajectory and the coordinates of the blade tip point in the top view of the wind turbine provided in this embodiment of the invention;

[0036] In the diagram: 101-Nacelle; 102-Outer circle of the tower; 103-Tower; 104-Characteristic points of the tower; 201-Millimeter-wave radar; 202-Video surveillance camera; 301-Hub; 401-Baseline; 402-Straight line of blade tip motion trajectory; 501-Blade attitude fitting curve; 502-Fitting curve when the blade is perpendicular to the ground. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be explained in complete detail below with reference to the accompanying drawings. The explained embodiments are only a part of the embodiments of the present invention, and not all of them. Furthermore, known technologies or commonly used technical terms are not necessary for describing the present invention, and have been omitted. Throughout the embodiments, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0038] Commonly used terms understood by those skilled in the art are not described in detail here. Instead, specific embodiments are used to illustrate the technical solutions proposed in this invention. This invention relates to a video surveillance camera clearance monitoring scheme and a millimeter-wave radar clearance monitoring scheme. Their processing procedures can be combined or decoupled and used as separate monitoring schemes. Identical or similar concepts and processing procedures will not be described further in the embodiments. The embodiments of this invention will be explained below with reference to the accompanying drawings.

[0039] First, the real-time tower clearance monitoring method integrating millimeter-wave radar and video provided in this invention is based on a clearance monitoring system, comprising a controller system communicating with the wind turbine main control system, and a video surveillance camera system and a millimeter-wave radar system communicating with the control system. The camera system is installed on the left front or right front side of the bottom of the nacelle, and the millimeter-wave radar system is installed on the left rear side of the bottom of the nacelle. In this embodiment, the video surveillance camera 202 is located on the left front side of the bottom of the nacelle 101 (between the tower 103 and the hub 301), and the millimeter-wave radar 201 is located on the left rear side of the bottom of the nacelle 101 (at the rear of the nacelle). Both systems rotate to point towards the blade clearance area through a rotating adjustment structure, realizing image acquisition and radar reflection point data acquisition during the blade motion cycle. The controller system calculates the blade tip clearance value based on the acquired image data and blade motion reflection point data, quantitatively assesses the deviation between the two based on the blade tip motion trajectory line, and qualitatively assesses the deviation based on the radar fitting of the blade tip coordinates, finally determining a high-precision clearance value between the blade tip and the tower support, which is transmitted to the wind turbine main control system in real time to achieve safe control of the unit.

[0040] This invention provides a technical solution for monitoring the airspace clearance of wind turbine towers based on the fusion of video surveillance cameras and millimeter-wave radar. The overall process is as follows: Figure 1 As shown in the attached diagram, the applicant will now describe the process flow in detail. It is important to emphasize that those skilled in the art should understand the interrelationships between the steps in the process. The process is not strictly performed according to the order indicated by the arrows in the flowchart. The video surveillance system and the millimeter-wave monitoring system are two parallel subsystems. The processing rate and order of the two subsystems depend on technical parameters such as the sampling period of the video surveillance camera and the sampling period of the millimeter-wave radar. The processing order for a single blade may alternate, but the deviation in processing time for a single blade between the video surveillance camera and the millimeter-wave radar is very small.

[0041] The FOV of video surveillance camera 202 is as follows Figure 2 As shown, pointing towards the target movement area of ​​the blade, images of both the blade and the tower 103 can be captured simultaneously. When the blade enters the camera's field of view (FOV), the pixel coordinates of the blade tip in the image can be identified using an image recognition algorithm. When the blade leaves the camera's FOV, the blade outline and the blade tip disappear. Based on the set of blade tip pixel coordinates identified within the camera's FOV, a least-squares linear fitting is performed to obtain the blade tip motion trajectory line. The number of blade tip points and the pixel distance between adjacent blade tip points depend on the current rotational speed of the wind turbine and the camera's frame rate. From the top view of the wind turbine, the fitted blade tip motion trajectory line in the image can be projected onto a horizontal plane at the same height as the blade tip, thus obtaining the blade tip motion trajectory line 402 and the blade tip trajectory points on the horizontal plane at the same height as the blade tip. Figure 3 As shown.

[0042] During periods of low and stable unit speed, the slope of the blade tip trajectory remains essentially constant. The point of tangency between the parallel line of the blade tip trajectory line 402 and the outer circle 102 of the tower can be calculated at this moment. This point of tangency serves as the tower feature point 104, and the tower feature point 104 and the baseline 401 remain unchanged. The baseline 401 serves as the video monitoring clearance baseline. In this embodiment, the baseline is obtained by translating the blade tip trajectory line when the blade speed is less than 3 rpm. For each blade's periodic entry into and exit from the camera's field of view (FOV), the blade tip trajectory line 402 can be fitted and calculated. The vertical distance from the tower feature point 104 to the blade tip trajectory line 402 is then used as the video monitoring clearance value VC.

[0043] The millimeter-wave radar monitoring device selected in this invention is a radar device that outputs a two-dimensional planar point cloud based on the target reflection point within the radar cone. A radar device that can output a three-dimensional spatial coordinate point cloud was not selected because radar devices that output three-dimensional coordinate point clouds have lower precision and accuracy in the third dimension. Furthermore, they have a lower data acquisition frequency, which poses a risk of missing the detection of blades because the time it takes for the blades to enter and leave the radar FOV is longer than the radar's single sampling time.

[0044] In this invention, when the blade enters the monitoring range of the millimeter-wave radar 201, the number of blade point clouds monitored by the radar starts to increase from 0. When the blade leaves the monitoring range of the millimeter-wave radar 201, the number of blade point clouds monitored by the radar starts to decrease to 0. According to this rule, the number of blade point clouds captured by adjacent radar single sampling is plotted in time sequence. Using the local maximum search algorithm, the peak frame with the highest number of points in the adjacent 7 frames of point cloud data can be found. The point clouds of the 3 frames before and after the peak frame number are merged as the complete point cloud of the blade within the monitoring range of the millimeter-wave radar.

[0045] After segmenting the blade point cloud data into 7 frames for each blade, the point cloud distribution information in the radar coordinate system appears as follows in the wind turbine side view: Figure 4As shown. The XOY coordinate system is the radar coordinate system, and the elevation angle α is the angle between the radar's center normal and the vertical ground line. In practice, a suitable elevation angle α is determined based on the monitoring range of the millimeter-wave radar 201, the blade length, and the horizontal distance between the millimeter-wave radar 201 and the hub 301, so as to achieve a larger blade area appearing within the horizontal beam angle of the millimeter-wave radar 201, that is, the area enclosed by FOVU and FOVD in the embodiment (horizontal field of view). Those skilled in the art should understand that the blade attitude fitting curve 501 can be fitted by various polynomial fitting methods based on the radar point cloud data, and linear integration is performed starting from the junction of the hub 301 and the nacelle 101 (blade root prior point). When the integration reaches the blade length, the coordinates of point P at the blade tip (point P is the blade tip point calculated by the millimeter-wave radar fitting) are obtained. The horizontal distance WY between point P and the radar is further calculated, and the straight-line distance between the blade root prior point and point P is calculated. In practical applications, the blade tip distance (i.e., the arc length between the last radar detection point and point P on the blade attitude fitting curve 501) detected by the normal radar is within 10m, which further ensures the accuracy of the fitting.

[0046] In the front view of the fan, as shown Figure 5 In this diagram, the elevation angle β of the millimeter-wave radar 201 is the angle between the radar center normal and the central axis of the tower. It is actually determined based on the monitoring range of the millimeter-wave radar 201 and the vertical distance between the millimeter-wave radar 201 and the central axis of the nacelle 101. This ensures that the tower 103 does not appear within the area (vertical field of view) enclosed by the FOVL and FOVR of the millimeter-wave radar 201, thus preventing the radar electromagnetic beam from being blocked by the tower 103 and allowing for better monitoring of the moving blades. At this point, the straight-line distance between point P and the prior point at the blade root can be calculated using trigonometric functions to determine the vertical distance WX between point P and the central axis of the tower. Those skilled in the art should understand that the blade clearance value monitored by the millimeter-wave radar 201 is the clearance value at that moment, while the true clearance value should be the time when the blade coincides with the central axis of the tower, i.e., the moment of the fitted curve 502 in the figure when the blade is perpendicular to the ground. Due to the circular motion of the blade and the effect of blade gravity, the blade tip P will have a certain drop ΔY, and the resulting clearance deviation cannot be quantitatively calculated at this time.

[0047] At the same level as the leaf tip, such as Figure 6 As shown, the spatial relationship between point P and the radar is illustrated. The vertical distance WC between point P and the outer circle of the tower can be calculated. Since the clearance value is defined as the safe clearance between the blade tip and the tower wall, WC is the clearance value of the millimeter-wave radar 201.

[0048] The aforementioned blade tip trajectory line 402 and point P analyzed by millimeter-wave radar are summarized and displayed below. Figure 7The deviation between the video-monitored clearance VC and the radar-monitored clearance WC can be visually observed. The deviation between WC and VC consists of two parts: first, the systematic deviation caused by the slope of the blade tip trajectory line; and second, the inherent deviation in the membership relationship between point P and the blade tip trajectory line 402. Figure 7 Within the established two-dimensional coordinate system, the vertical distance between point P and the blade tip trajectory line 402 can be calculated as the inherent clearance deviation between the video monitoring system and the millimeter-wave monitoring system. When the system calibration is normal and the unit is running smoothly, this deviation should be close to 0. In practice, this is used to qualitatively evaluate the accuracy of the calibration of the two systems. The deviation between WC and VC is affected by the slope of the blade tip trajectory line, the radar pitch angle β, and the blade clearance value distribution range. Specifically, the larger the angle between the blade tip trajectory line 402 and the nacelle centerline, the greater the deviation between WC and VC. In particular, the blade tip trajectory line 402 is completely perpendicular to the nacelle centerline. Theoretically, WC and VC are equal. The larger the radar elevation angle β, the greater the deviation between WC and VC. However, the applicant found that β is approximately fixed at 6-10° based on the radar's FOV, and this value is fixed during the initial radar installation for a single wind turbine and will not be changed. Therefore, this variable can be considered to cause a fixed deviation. The blade clearance distribution range refers to the clearance area within which the turbine operates. At low turbine speeds, the distance between the blade tip and the tower wall is greater. As the turbine speed increases, this distance gradually decreases. A larger clearance area results in a greater deviation between WC and VC than a smaller clearance area. In actual use, the appropriate clearance angle should be selected... This serves as a quantitative assessment of the bias between the two systems.

[0049] In low-visibility environments such as dense fog and patchy fog, the pixel distance distribution between adjacent leaf tip trajectory points is uneven, and the standard deviation of the distance from each point to the leaf tip motion trajectory line 402 is relatively large. In extreme cases, only two leaf tip trajectory points are identified. Although the leaf tip motion trajectory line can be fitted from two leaf tip trajectory points, the clearance accuracy and precision will be low. In extreme cases, when the fog is even thicker, the camera cannot detect the blades at all. That is, when no leaf tip trajectory points are identified, the video monitoring solution is completely ineffective. In this case, the radar monitoring clearance value is selected as the clearance monitoring result.

[0050] When dealing with rain and snow, the characteristic of point cloud quantity first increasing and then decreasing no longer applies, meaning that millimeter-wave monitoring of blade point clouds fails. In extreme cases, the blade target is completely covered by rain and snow noise, resulting in a chaotic temporal sequence of point cloud quantity and disordered gradient information between adjacent point clouds. In this situation, the quality of millimeter-wave radar monitoring of blade point cloud quantity and blade attitude polynomial fitting is low, and the blade attitude fitting line does not conform to the blade morphology rules. Therefore, radar monitoring clearance is deemed ineffective, and video monitoring clearance values ​​are selected as the clearance monitoring results.

[0051] The airspace monitoring scheme provided in this embodiment only fails in extreme environments such as dense fog and heavy rain or snow. In such cases, the distribution characteristics of the motion trajectory points and the quality of the fitting residuals are both poor. The scheme is deemed invalid as the final airspace result, and the characteristic airspace value is output. It is recommended that the wind turbine units operate at limited power.

[0052] The embodiments of the present invention are merely illustrative examples for explaining the present invention. Those skilled in the art should understand that various modifications, additions, adjustments, or similar substitutions can be made to the specific embodiments described, without departing from the scope defined by the appended claims.

Claims

1. A tower clearance real-time monitoring method based on fusion of millimeter wave radar and video, characterized in that Includes the following steps: S1: Install the camera system on the left or right front side of the bottom of the wind turbine nacelle, and adjust its attitude so that its field of view (FOV) faces the area between the tower and the blades, so that the camera system can capture images of the blades and the tower at the same time. S2: The image recognition algorithm is used to calculate the captured image information, identify the movement trajectory points of the blade tip, fit the blade tip movement trajectory points identified by the image using the least squares method to obtain the blade tip movement trajectory straight line, and further obtain the blade tip movement trajectory straight line during the low speed and stable speed period of the wind turbine. S3: In the horizontal projection plane at the same height as the blade tip, the straight line of the blade tip motion trajectory during the low speed and stable speed period of the wind turbine is translated to be tangent to the outer circumference of the tower. The tangent is used as the baseline and the tangent point is used as the tower feature point. A two-dimensional coordinate system is constructed in the horizontal projection plane at the same height as the blade tip, with the tower feature point as the origin. The vertical distance between the tower feature point and the straight line of the blade tip motion trajectory obtained in real time is calculated. This distance is the video monitoring clearance value. S4: Install the millimeter-wave radar system on the bottom left rear side of the wind turbine nacelle, and adjust its attitude to face the area between the tower and the blades, so that the millimeter-wave radar system can monitor the horizontal two-dimensional radar point cloud data of the blades at a fixed azimuth angle. S5: In the two-dimensional coordinate system of S4, the two-dimensional coordinates at the blade tip are calculated by least squares curve fitting and linear integration, and the radar monitoring clearance value is calculated based on the two-dimensional coordinates at the blade tip. S6: In the two-dimensional coordinate system of S3, the distance deviation between the video monitoring clearance value and the radar monitoring clearance value is quantitatively calculated as the system deviation. At the same time, the spatial relationship between the straight line of the blade tip motion trajectory and the two-dimensional coordinate information at the blade tip is qualitatively analyzed. S7: Compare the distribution characteristics of motion trajectory points in S2 with the quality of fitting residuals in S5. If one of the distribution characteristics or the quality of fitting residuals is poor, select the other result for output, that is, select the video monitoring net value or the radar monitoring net value as the net monitoring result; when both have good distribution characteristics and fitting residual quality, perform inter-system mutual calibration and early warning of excessive deviation.

2. The millimeter wave radar and video fusion tower clearance real-time monitoring method according to claim 1, characterized in that: In S2, select a straight line for the blade tip trajectory when the blade rotation speed is less than 3 rpm.

3. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: In S4, as the blade enters and leaves the field of view of the millimeter-wave radar, 3 to 7 frames of radar point cloud data are collected according to the acquisition frequency of the millimeter-wave radar. The number of blade point clouds collected by the radar follows the characteristic of first increasing and then decreasing. The local maximum search algorithm is used to find the peak moment of the number of point clouds in adjacent 7 frames, and then obtain the complete point cloud data of a blade in the radar field of view.

4. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: In S5, when using the least squares curve fitting method, the blade tip two-dimensional coordinates are calculated by linear integration from the blade root based on the fitted curve, and the Y value of the blade tip two-dimensional coordinates is calculated as the radar monitoring clearance value.

5. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: The system deviation described in S6 is affected by both the slope of the straight line of the blade tip motion trajectory and the two-dimensional coordinates of the blade tip point. This system deviation is regarded as the inherent deviation between systems.

6. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: When performing qualitative analysis in S6, if the straight trajectory of the blade tip passes through the blade tip point fitted by the millimeter-wave radar, it is determined that there is no deviation between the video monitoring clearance and the millimeter-wave radar monitoring clearance during the initial calibration process; otherwise, it is determined that there is a calculation deviation introduced during the initial calibration process of either the video monitoring clearance or the millimeter-wave radar monitoring clearance.

7. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: In S7, the distribution characteristics of motion trajectory points in S2 and the quality of fitting residuals in S5 are both good under normal weather conditions. The accuracy of the two is cross-checked, and the video monitoring airspace is selected as the final result.

8. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: In S7, under conditions of dense fog or patchy fog with low visibility, the distribution characteristics of motion trajectory points monitored by video are irregular, the blade trajectory points are discontinuous and there are abnormal points and outliers that deviate from the blade tip trajectory points, and the straight-line fitting quality of the blade tip motion trajectory is not high. Therefore, the video monitoring clearance is deemed to be invalid, and the radar monitoring clearance value is selected as the clearance monitoring result.

9. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: In S7, under heavy rain and snow conditions, the two-dimensional radar point cloud data of the blades monitored by the millimeter-wave radar is mixed with rain and snow noise. At this time, the number of blade point clouds monitored by the millimeter-wave radar and the quality of the multinomial fitting of the blade attitude are not high. The blade attitude fitting line does not conform to the blade shape rules, and it is determined that the radar monitoring clearance is invalid. The video monitoring clearance value is selected as the clearance monitoring result.

10. The method for real-time monitoring of tower clearance by fusing millimeter-wave radar and video according to claim 1, characterized in that: If the distribution characteristics of the motion trajectory points in S2 and the quality of the fitting residuals in S5 are both poor in S7, then invalidity will be used as the final net result.

Images