A Visual Optimization Method for Blade Condition Assessment of Large Wind Turbines

Abstract

The present invention discloses a visual optimization method for blade state assessment of large wind turbines, and the method comprises the following steps: S1, a pan-tilt camera and a central controller for controlling the pan-tilt camera are configured for the large wind turbine; S2, motion path planning is performed on the pan-tilt camera; S3, a blade tracking algorithm is generated according to the planned motion path, so as to calculate the motion information of the pan-tilt camera and control the motion of the pan-tilt camera; S4, according to the motion information of the pan-tilt camera, a deep learning method is used to evaluate the icing state of the pan-tilt camera, so as to evaluate the icing state of the blades of the large wind turbine; the present invention can, under sufficient light conditions, use the tracking control of the pan-tilt camera for shooting, avoid the problem of backlight shooting, be able to clearly capture the surface texture of the blade, and when frozen by ice and snow, can combine the deep learning method to evaluate the icing state of the pan-tilt camera itself, and further evaluate the icing state of the blade.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine status assessment, and particularly to a visual optimization method for blade status assessment of large wind turbines. Background Art

[0002] As a core component of a wind turbine generator set, the operating status of the blade directly affects the availability of the unit, and further affects the investment return of the entire wind farm. For the monitoring of the blade operating status, a video monitoring scheme is usually adopted. However, when encountering snowstorm or freezing rain weather, the blade starts to ice, and in severe cases, it even freezes the pan-tilt camera responsible for video monitoring. When the pan-tilt camera is frozen, it cannot move to the designated working position for blade monitoring, and thus it is impossible to judge the blade icing. In addition, the blade operating status belongs to all-weather monitoring. If the shooting is carried out at a fixed working angle, there will always be a situation of backlight shooting, resulting in unclear shooting of the blade surface texture, thus affecting the discrimination of icing and damage status.

[0003] The current pan-tilt camera control methods for blade operating status monitoring mainly include: 1) fixed monitoring and shooting based on a single angle; 2) fixed monitoring and shooting based on multiple angles; 3) shooting based on manual observation. That is, there is currently no special scheme for analyzing the video monitoring control of the blade operating status, which will cause the following problems: 1) In the stage of ice and snow melting, the weather conditions improve and the sunlight may be strong. When the camera faces the sun directly, it is impossible to clearly shoot the blade surface texture; 2) The movement mode and route of the pan-tilt are fixed and cannot be combined with the status of the unit; 3) There is no closed-loop feedback port for developing the quantized pose of the pan-tilt on the market pan-tilt cameras, and it is impossible to transmit the quantized pitch angle and yaw angle of the pan-tilt to the upper computer, and only the control and display command values can be fed back. The pan-tilt camera cannot return the quantized pose value, and when it is frozen by ice and snow, it is impossible to obtain the real angle or report the icing status of the pan-tilt itself. Summary of the Invention

[0004] The purpose of the present invention is to provide a visual optimization method for blade status assessment of large wind turbines to solve the deficiencies in the prior art. A pan-tilt camera is adopted, the movement path of the pan-tilt is determined by offline planning, and a blade tracking algorithm is formulated to synchronize the pan-tilt camera with the blade movement to ensure that the blade is at the center of the shooting screen of the pan-tilt camera.

[0005] The present invention is realized through the following technical solutions: A visual optimization method for blade status assessment of large wind turbines includes the following steps:

[0006] S1. A large wind turbine is configured with a pan-tilt camera and a central controller for controlling the pan-tilt camera;

[0007] S2. Perform movement path planning for the pan-tilt camera;

[0008] S3. Generate a blade tracking algorithm according to the planned motion path, thereby calculating the motion information of the pan-tilt camera and controlling the motion of the pan-tilt camera;

[0009] S4. According to the motion information of the pan-tilt camera, use the deep learning method to evaluate the icing state of the pan-tilt camera, thereby evaluating the icing state of the blades of the large wind turbine.

[0010] Further, the step S1 includes the following steps:

[0011] The pan-tilt camera is installed on the top of the nacelle of the large wind turbine and is used to record the video of the operating state of the blades of the large wind turbine. The central controller includes a GPU and a neural computing unit. The central controller is communicatively connected to the pan-tilt camera and is used to control the pan-tilt camera to track the blade motion, intercept the video stream and perform detailed video analysis.

[0012] Further, the step S2 includes the following steps:

[0013] Perform a motion path planning for the pan-tilt camera, and divide the motion path of the pan-tilt camera into the following four segments:

[0014] Segment a: Scan each component on the top of the nacelle according to a preset path for offline planning;

[0015] Segment b: Lock the root of the blade, and the pan-tilt camera sequentially follows the motion of the root of each blade to take pictures according to a preset order;

[0016] Segment c: Lock the middle part of the blade, and the pan-tilt camera sequentially follows the motion of the middle part of each blade to take pictures according to a preset order;

[0017] Segment d: Lock the tip of the blade, and the pan-tilt camera sequentially follows the motion of the tip of each blade to take pictures according to a preset order.

[0018] Further, the planning of the segment a includes the following steps:

[0019] After setting the pitch angle of the pan-tilt camera according to the length of the blade, the pan-tilt camera scans the top of the nacelle and performs a yaw reciprocating motion at a scanning angle of 0-360°.

[0020] Further, the planning of the segment b includes the following steps:

[0021] b1. Control the yaw motion of the pan-tilt camera, and the yaw direction points to the middle position of the root area;

[0022] b2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder;

[0023] b3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed. The planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the blade root.

[0024] b4. Repeat the above steps b1 to b3 to track the other blades of the large wind turbine in sequence.

[0025] Furthermore, the planning of the c section includes the following steps:

[0026] c1. Control the yaw movement of the pan-tilt camera, and the yaw direction points to the middle position of the middle region of the blade.

[0027] c2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder.

[0028] c3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed. The planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the middle part of the blade.

[0029] c4. Repeat the above steps c1 to c3 to track the other blades of the large wind turbine in sequence.

[0030] Furthermore, the planning of the d section includes the following steps:

[0031] d1. Control the yaw movement of the pan-tilt camera, and the yaw direction points to the middle position of the tip region of the blade.

[0032] d2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder.

[0033] d3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed. The planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the blade tip.

[0034] d4. Repeat the above steps d1 to d3 to track the other blades of the large wind turbine in sequence.

[0035] Furthermore, the step S3 includes the following steps:

[0036] Generate a blade tracking algorithm according to the planned motion path. Denote the height of the center of the pan-tilt camera from the center of the impeller as h, the horizontal distance as d, the radius of the trajectory of the point to be tracked as R, and the angle of the blade to be tracked as φ, φ ∈ (-90°, 90°), where the positive direction is along the blade running direction and the negative direction is against the blade running direction; set the impeller azimuth angle as γ, γ ∈ (0°, 360°), and the angle of the blade to be tracked is:

[0037]

[0038] When the blade angle is φ, the center of the pan-tilt camera is aligned with the point to be tracked, and the yaw angle of the pan-tilt camera needs to be:

[0039]

[0040] The pitch angle of the pan-tilt camera is:

[0041]

[0042] According to the impeller rotation speed and the impeller azimuth angle, the trajectory of φ(γ) is predicted, and then α(γ) and β(γ) are quickly planned.

[0043] Further, the step S4 includes the following steps:

[0044] When the motion information of the pan-tilt camera, that is, the real angle information of the pan-tilt camera, is obtained, calculate the difference between the control command of the central controller for the pan-tilt camera and the actual angle of the pan-tilt camera, and judge whether there is an icing situation for the pan-tilt camera; when the central controller cannot obtain the real angle information of the pan-tilt camera, use the convolutional neural network CNN to judge the blade tracking effect in the video captured by the pan-tilt camera.

[0045] Further, 4-directional convolutions in the up, down, right, and left directions are added to the convolutional neural network CNN; the output of the convolutional neural network CNN includes branch 1 and branch 2. Among them, branch 1 outputs the mask of the blade edge, and branch 2 is the probability of the existence of a line; the annotation information of branch 1 is the blade contour map, and the annotation information of branch 2 is the probability of the existence of the upper and lower edges of the blade. When the edge exists, the value of branch 2 is 1, and when it does not exist, the value of branch 2 is 0; branch 1 uses smooth_l1 loss, and branch 2 uses cross-entropy loss.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] 1. The present invention can use the tracking control of the pan-tilt camera to shoot under sufficient light, avoiding the problem of backlight shooting and being able to clearly shoot the texture on the blade surface;

[0048] 2. The movement route of the pan-tilt camera is combined with the state of the wind turbine, making the shooting effect better;

[0049] 3. The present invention has a closed-loop feedback port for quantifying the pose of the pan-tilt camera, which can transmit the quantified pitch angle and yaw angle of the pan-tilt camera to the central controller. When it is frozen by ice and snow, it can obtain the real angle or evaluate the icing state of the pan-tilt camera itself by combining deep learning methods, and then evaluate the icing state of the blade. Brief Description of the Drawings

[0050] Figure 1Schematic diagram of the yaw movement direction of the pan-tilt camera.

[0051] Figure 2 Schematic diagram of the pitch movement direction of the pan-tilt camera.

[0052] Figure 3 Schematic diagram of the preset scanning path for offline planning.

[0053] Figure 4 Schematic diagram of the initial position of the pan-tilt camera tracking the root of the blade.

[0054] Figure 5 Schematic diagram of the initial position of the pan-tilt camera tracking the middle part of the blade.

[0055] Figure 6 Schematic diagram of the initial position of the pan-tilt camera tracking the tip of the blade.

[0056] Figure 7 Geometric relationship diagram between the pan-tilt camera and the point to be tracked. Detailed implementation mode

[0057] The present invention will be further described below in conjunction with specific embodiments.

[0058] See Figures 1 to 7 As shown, the visual optimization method for blade state evaluation of large wind turbines provided in this embodiment includes the following steps:

[0059] S1. Configure a pan-tilt camera 1 and a central controller (not shown in the figure) for controlling the pan-tilt camera on the large wind turbine, including the following steps:

[0060] The pan-tilt camera is installed on the top of the nacelle 2 of the large wind turbine and is used to record the operation state video of the blades 3 of the large wind turbine. The central controller includes a GPU and a neural computing unit. The central controller is communicatively connected to the pan-tilt camera 1 and is used to control the pan-tilt camera 1 to track the movement of the blade 3, intercept the video stream and perform detailed video analysis.

[0061] S2. Perform motion path planning for the pan-tilt camera, including the following steps:

[0062] Perform motion path planning for the pan-tilt camera, and divide the motion path of the pan-tilt camera into the following four segments:

[0063] Segment a: Scan each component on the top of the nacelle according to a preset path for offline planning, including the following steps:

[0064] After setting the pitch angle of the pan-tilt camera according to the length of the blade, the pan-tilt camera scans the top of the nacelle and performs yaw reciprocating motion at a scanning angle of 0 - 360°.

[0065] Section b: Lock the root part 301 of the blade. The pan-tilt camera follows the movement of the root part of each blade in a preset order for shooting, including the following steps:

[0066] b1. Control the pan-tilt camera to yaw, and the yaw direction points to the middle position of the root area of the blade;

[0067] b2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder;

[0068] b3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the root part 301 of the blade;

[0069] b4. Repeat the above steps b1 to b3 to track the other blades of the large wind turbine in turn.

[0070] Section c: Lock the middle part 302 of the blade. The pan-tilt camera follows the movement of the middle part of each blade in a preset order for shooting, including the following steps:

[0071] c1. Control the pan-tilt camera to yaw, and the yaw direction points to the middle position of the middle area of the blade;

[0072] c2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder;

[0073] c3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the middle part 302 of the blade;

[0074] c4. Repeat the above steps c1 to c3 to track the other blades of the large wind turbine in turn.

[0075] Section d: Lock the tip part 303 of the blade. The pan-tilt camera follows the movement of the tip part of each blade in a preset order for shooting, including the following steps:

[0076] d1. Control the pan-tilt camera to yaw, and the yaw direction points to the middle position of the tip area of the blade;

[0077] d2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera, and match it with the measurement result of the impeller speed encoder;

[0078] d3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the tip part 303 of the blade;

[0079] d4. Repeat the above steps d1 to d3 to track the other blades of the large wind turbine in turn.

[0080] S3. Generate a blade tracking algorithm based on the planned motion path, thereby calculating the motion information of the pan-tilt camera and controlling the motion of the pan-tilt camera, including the following steps:

[0081] Generate a blade tracking algorithm based on the planned motion path. Denote the height from the center of the pan-tilt camera to the center of the impeller as h, the horizontal distance as d, the radius of the trajectory of the point to be tracked as R, and the angle of the blade to be tracked as φ, where φ ∈ (-90°, 90°), with the positive direction being the direction of blade rotation and the negative direction being the opposite direction; set the azimuth angle of the impeller as γ, where γ ∈ (0°, 360°), and the angle of the blade to be tracked is:

[0082]

[0083] When the blade angle is φ, the center of the pan-tilt camera is aligned with the point to be tracked, and the yaw angle of the pan-tilt camera needs to be:

[0084]

[0085] The pitch angle of the pan-tilt camera is:

[0086]

[0087] According to the impeller rotation speed and the impeller azimuth angle, predict the trajectory of φ(γ), and then quickly plan α(γ) and β(γ).

[0088] S4. According to the motion information of the pan-tilt camera, use the deep learning method to evaluate the icing state of the pan-tilt camera, thereby evaluating the icing state of the blades of the large wind turbine, including the following steps:

[0089] When obtaining the motion information of the pan-tilt camera, that is, the real angle information of the pan-tilt camera, calculate the difference between the control command of the central controller for the pan-tilt camera and the actual angle of the pan-tilt camera, and judge whether there is icing on the pan-tilt camera; when the central controller cannot obtain the real angle information of the pan-tilt camera, use the convolutional neural network CNN to judge the blade tracking effect in the video captured by the pan-tilt camera.

[0090] Four types of convolutions in the up, down, right, and left directions are added to the convolutional neural network CNN; the output of the convolutional neural network CNN includes Branch 1 and Branch 2. Among them, Branch 1 outputs the mask of the blade edge, and Branch 2 is the probability of the existence of the line; the annotation information of Branch 1 is the blade contour map, and the annotation information of Branch 2 is the probability of the existence of the upper and lower edges of the blade. When the edge exists, the value of Branch 2 is 1, and when it does not exist, the value of Branch 2 is 0; Branch 1 uses smooth_l1 loss, and Branch 2 uses cross-entropy loss.

[0091] The above-described embodiments are only the preferred embodiments of the present invention, and do not limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A visual optimization method for blade state assessment of large wind turbines, characterized in that, The steps include the following: S1. Configure a pan-tilt camera and a central controller for controlling the pan-tilt camera on a large wind turbine; S2. Plan the movement path of the pan-tilt camera, and divide the movement path of the pan-tilt camera into the following four segments: Segment a: Scan each component on the top of the nacelle according to a preset path for offline planning; Segment b: Lock the root of the blade, and the pan-tilt camera takes pictures in sequence following the movement of the root of each blade according to a preset order; Segment c: Lock the middle part of the blade, and the pan-tilt camera takes pictures in sequence following the movement of the middle part of each blade according to a preset order; Segment d: Lock the tip of the blade, and the pan-tilt camera takes pictures in sequence following the movement of the tip of each blade according to a preset order; S3. Generate a blade tracking algorithm according to the planned movement path, so as to calculate the movement information of the pan-tilt camera and control the movement of the pan-tilt camera; Let the height from the center of the pan-tilt camera to the center of the impeller be \(h\), the horizontal distance be \(d\), the radius of the trajectory of the point to be tracked be \(R\), and the angle of the blade to be tracked be , ∈(-90°,90°), where the direction along the blade operation is positive and the direction against the blade operation is negative; set the azimuth angle of the impeller to be \(\gamma\), \(\gamma\in(0^{\circ}, 360^{\circ})\), and the angle of the blade to be tracked is: ； When the blade angle is , the center of the pan-tilt camera is aligned with the point to be tracked, and the yaw angle of the pan-tilt camera needs to be: ； The pitch angle of the pan-tilt camera is: ； According to the impeller rotation speed and the impeller azimuth angle, predict the trajectory, and then quickly plan and ; S4. According to the movement information of the pan-tilt camera, use the deep learning method to evaluate the icing state of the pan-tilt camera, so as to evaluate the icing state of the blades of the large wind turbine.

2. The visual optimization method for blade state assessment of a large wind turbine according to claim 1, characterized in that The step S1 includes the following steps: The pan-tilt camera is installed on the top of the nacelle of the large wind turbine and is used to record the video of the running state of the blades of the large wind turbine. The central controller includes a GPU and a neural computing unit. The central controller is communicatively connected to the pan-tilt camera and is used to control the pan-tilt camera to track the movement of the blade, intercept the video stream and perform detailed video analysis.

3. A visual optimization method for blade state assessment of large wind turbines according to claim 1, characterized in that The planning of segment a includes the following steps: After setting the pitch angle of the pan-tilt camera according to the length of the blade, the pan-tilt camera scans the top of the nacelle and performs a yaw reciprocating movement at a scanning angle of 0-360°.

4. A visual optimization method for blade state assessment of a large wind turbine, as claimed in claim 1, wherein The planning of segment b includes the following steps: b1. Control the yaw movement of the pan-tilt camera, and the yaw direction points to the middle position of the root area; b2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera and match it with the measurement result of the impeller speed encoder; b3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the root of the blade; b4. Repeat the above steps b1 to b3 to track the other blades of the large wind turbine in sequence.

5. A visual optimization method for blade state assessment of a large wind turbine, as described in claim 1, characterized in that The planning of segment c includes the following steps: c1. Control the yaw movement of the pan-tilt camera, and the yaw direction points to the middle position of the middle area; c2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera and match it with the measurement result of the impeller speed encoder; c3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the middle part of the blade; c4. Repeat the above steps c1 to c3 to track the other blades of the large wind turbine in sequence.

6. A visual optimization method for blade state assessment of large wind turbines according to claim 1, characterized in that The planning of segment d includes the following steps: d1. Control the yaw movement of the pan-tilt camera, and the yaw direction points to the middle position of the tip area; d2. Use the optical flow method to determine the time interval t when the blade sweeps across the field of view of the pan-tilt camera and match it with the measurement result of the impeller speed encoder; d3. Plan the tracking path of the pan-tilt camera for the blade according to the impeller speed, and the planning constraint is that the center of the line of sight of the pan-tilt camera points to the center point of the blade tip; d4. Repeat the above steps d1 to d3 to track the other blades of the large wind turbine in sequence.

7. A visual optimization method for blade state assessment of large wind turbines according to claim 1, characterized in that The step S4 includes the following steps: When the motion information of the pan-tilt camera, that is, the real angle information of the pan-tilt camera, is obtained, calculate the difference between the control command of the central controller for the pan-tilt camera and the actual angle of the pan-tilt camera, and judge whether there is an icing condition for the pan-tilt camera; when the central controller cannot obtain the real angle information of the pan-tilt camera, use the convolutional neural network CNN to judge the blade tracking effect in the video captured by the pan-tilt camera.

8. A visual optimization method for blade state assessment of large wind turbines according to claim 7, characterized in that: Four types of convolutions in the up, down, right, and left directions are added to the convolutional neural network CNN; the output of the convolutional neural network CNN includes branch 1 and branch 2. Among them, branch 1 outputs the mask of the blade edge, and branch 2 is the probability of the existence of the line; the annotation information of branch 1 is the blade contour map, and the annotation information of branch 2 is the probability of the existence of the upper and lower edges of the blade. When the edge exists, the value of branch 2 is 1, and when it does not exist, the value of branch 2 is 0; branch 1 uses smooth_l1 loss, and branch 2 uses cross-entropy loss.

Images