Blade waving displacement monitoring system and method based on binocular polarization camera

Abstract

The invention discloses a blade waving displacement monitoring system based on a binocular polarization camera, the blade waving displacement monitoring system comprises a circular polaroid, a polarization vision system and a remote terminal, the circular polaroid is fixed at a blade tip of a blade, and the polarization vision system is arranged right ahead or laterally ahead of the blade and is used for collecting image information of the circular polaroid at the blade tip; the remote terminal is used for receiving the image information of the polarization vision system and calculating a Stokes parameter, a polarization degree and a polarization angle; the method comprises the following steps: generating an HSV (hue, saturation and value) polarization image, segmenting an area with high polarization degree and an arc edge, carrying out ellipse fitting on an arc in the area, extracting a three-dimensional coordinate of a circle center of a polaroid after three-dimensional reconstruction, and carrying out four-stage coordinate conversion to obtain flapping displacement under a blade coordinate system. The invention further discloses a blade waving displacement monitoring method based on the binocular polarization camera. The invention has the advantages of high detection precision, simple structure, low cost, low environmental sensitivity and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of wind and optical technology, and in particular to a blade flapping detection system and method based on a binocular polarization camera. BACKGROUND

[0002] In recent years, with the rapid development of the wind power industry, the power of land-based wind turbines has gradually increased. The safety of wind turbines during operation is very important, otherwise it will cause premature damage to the wind turbine during its lifetime, resulting in direct economic losses. For wind turbines, the detection of blade flapping displacement is extremely important. The current mainstream methods for detecting blade flapping displacement of wind turbines include computer vision and unmanned aerial vehicle cooperative detection, vibration-displacement indirect sensing, and laser radar flow field inversion.

[0003] For example, a wind turbine group clearance monitoring method, device, medium and system disclosed in Chinese Patent Publication No. CN117404255A measures blade deformation by segmenting and pasting strain gauges in the blade cavity, obtains the blade deformation, and then calculates the distance from the blade tip to the tower surface during the operation of the wind turbine according to the initial distance from the blade tip to the tower surface, the blade pitch angle, etc. The strain gauge is a relatively mature sensor for measuring blade deformation, and the strain and blade deformation angle are correlated.

[0004] For example, a blade tower-sweeping risk monitoring method, device, unit and storage medium disclosed in Chinese Patent Publication No. CN118128704A detect whether the flapping direction moment of the blade exceeds the moment threshold and the position of the blade through a load sensor, which can quickly determine whether there is a tower-sweeping risk caused by insufficient blade clearance.

[0005] The above methods have the disadvantages of low detection accuracy, high cost, complex detection method, and certain damage to the surface of the blade. SUMMARY

[0006] The technical problem to be solved by the embodiments of the present application is to provide a blade flapping displacement monitoring system and method based on a binocular polarization camera, which can detect the flapping displacement of the blade using a simple structure.

[0007] To solve the above-mentioned technical problems, this invention provides a blade flapping displacement monitoring system based on a binocular polarization camera, including a circular polarizer (1), a polarization vision system (2), and a remote terminal (3). The circular polarizer (1) is fixed at the tip of the blade. The polarization vision system (2) is arranged in front of or to the side of the blade to collect image information of the circular polarizer at the tip of the blade. The remote terminal (3) is used to receive the image information of the polarization vision system (2). The polarization vision system (2) consists of two monocular polarization cameras of the same model and two electronically controlled liquid crystal phase delay units. The remote terminal (3) performs noise reduction processing on the image acquired by the polarization vision system (2), and then calculates the Stokes parameter, polarization degree DoLP, and polarization angle AoP; then calculates the hue, saturation, and brightness information to generate an HSV polarization image, and then segments the region with high polarization degree and the region with arc edge according to the polarization image, and performs ellipse fitting on the arc in the region. After three-dimensional reconstruction, the three-dimensional coordinates of the center of the polarizer are extracted, and the flapping displacement in the blade coordinate system is obtained through four-level coordinate transformation.

[0008] The HSV polarization image is generated in the remote terminal (3) in the following manner: Regarding hue, the polarization angle Mapped to the standard color wheel The range, the formula is:

[0009] For saturation, use polarization degree directly. The saturation in the high polarization region is close to 1, and in the low polarization region it is close to 0.

[0010] For brightness, use normalization. Preserve scene brightness information: .

[0011] In the remote terminal (3), the coordinates (uL, vL) and (uR, vR) of the center pixel of the circular polarizer origin in the image are also obtained in the following way: right In the image High polarization region Edge detection is performed, followed by arc detection within the region. The portion containing the arc is segmented and treated as a polarizer processing area. Extract the contour point set of the polarizer obtained from the left and right eye polarizing cameras. , in Given the total number of contour points, the pixel coordinates of the polarizer center obtained from the left and right polarizing cameras are calculated. , ), ( , ):

[0012]

[0013]

[0014] .

[0015] .

[0016] The camera coordinate system transformation of the two-dimensional coordinates of the polarizer center in the remote terminal (3) requires the following steps: Eliminate lens distortion:

[0017]

[0018]

[0019]

[0020]

[0021]

[0022] in, Correct coordinates and for:

[0023]

[0024] Calculate parallax

[0025] Target 3D depth:

[0026] in, The physical focal length of the camera. denoted as U, representing the pixel coordinate U-direction distance from the left side of the image in polarized images captured by the left and right polarization cameras, and b as the baseline distance; The center pixel coordinates of the polarization image from the left-eye polarization camera ( , Convert to camera coordinate system ( , , ):

[0027]

[0028]

[0029] Among them: Among them: , , The physical focal length of the camera. and The physical size of a pixel; These are the coordinates of the image center point.

[0030] The waving displacement value in the remote terminal (3) is calculated using the following method:

[0031] Where: L is the blade length This is the amount of compensation for blade bending deformation. , , The amplitudes of the first and second order vibration modes. , For the natural frequency, To swing and move, The input is the blade coordinate system.

[0032] Accordingly, this invention also provides a method for monitoring blade flapping displacement based on a binocular polarization camera, comprising the following steps: S1: Fix multiple circular polarizers (1) at the tips of three blades in the same position. Place a polarization vision system (2) in front of or to the side of the blades to collect image information of the circular polarizers at the tips of the blades. Obtain the linear polarization angles as follows through the polarization vision system (2): , , , Polarization image; S2: Denoise the polarization image and calculate the Stokes parameters and degree of polarization. and polarization angle ; S3: Use the aforementioned polarization degree and polarization angle Adjust the saturation, hue, and brightness respectively, and then draw the result. image; S4: According to the above The image is used to calculate the center pixel coordinates of the circular polarizer in the images captured by the left and right eyes of the polarization vision system (2). S5: Convert the two-dimensional center coordinates of the circular polarizer in the left and right eye polarized images into three-dimensional camera coordinates; S6: Perform the following transformations on the three-dimensional camera coordinates in sequence: baseline coordinate system, boundary coordinate system, blade coordinate system, and flapping displacement value.

[0033] The center pixel coordinates of the circular polarizer in S4 are calculated using the following method: Regarding the In the image High polarization region Edge detection is performed, followed by arc detection within the region. The portion containing the arc is segmented and treated as a polarizer processing area. Extract the contour point set of the polarizer obtained from the left and right eye polarizing cameras. , in Given the total number of contour points, calculate the center coordinates of the circular polarizer in the pixel coordinate system, and obtain the center pixel coordinates of the polarizer obtained by the left and right eye polarizing cameras. , ), ( , ):

[0034]

[0035]

[0036]

[0037] Define the origin of the pixel coordinate system as the center of the top left pixel of the image, the u-axis as the right side of the origin, and the v-axis as the top of the origin.

[0038] In step S5, the camera coordinate system transformation of the two-dimensional coordinates of the polarizer center requires the following steps: To eliminate lens distortion, obtain the center pixel coordinates of the polarizer circle on the left side:

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] wherein, , is the radial distance from the image center to the point , represents the distance of the point from the optical center; corrected coordinates and are:

[0045]

[0046] wherein, , is the correction amount of the radial distortion in the and directions, , is the correction amount of the tangential distortion in the and directions, , is the total distortion correction value; calculating the polarized image center pixel coordinates of the right eye target after distortion correction , ; calculating the parallax ; target three-dimensional depth: ; wherein, is the physical focal length of the camera, is the length of the pixel coordinate U direction from the left side of the polarized picture taken by the left and right eye polarized cameras, and b is the baseline distance; The conversion formula of the polarized image center pixel coordinates of the left eye polarized camera , and the camera coordinate system , , is:

[0047]

[0048]

[0049] wherein: wherein: , , is the physical focal length of the camera, and are the physical sizes of the pixels; The image center point coordinates.

[0050] The conversion of the camera coordinate system into the baseline coordinate system in S6 is performed by the following steps: The conversion of the camera coordinate system into the baseline coordinate system is to translate the coordinate system with the optical center of the left polarized camera as the origin to the coordinate system with the midpoint of the baseline as the origin.

[0051] The purpose is to unify the left and right camera data to the center point of the binocular system, and the input is the left camera coordinates (x, y) , and the output is the baseline coordinates (x', y') , , The transformation formula is:

[0052] where B is the baseline distance of the left and right cameras; the coordinate origin is moved to the center of the binocular system, and the visual angle asymmetry is eliminated; The origin of the baseline coordinate system is defined as the midpoint of the baseline of the two cameras, the X-axis points to the right camera along the baseline, the Y-axis is perpendicular downward, and the Z-axis is the depth direction; The conversion of the baseline coordinate system into the world coordinate system in S6 is performed by the following formula: The transformation formula is:

[0053] where R is a 3*3 rotation matrix, and T is a translation vector [Tx, Ty, Tz] T ; The origin of the world coordinate system is defined as the center point of the tower drum bottom, the X-axis is the main wind direction, the Y-axis is the horizontal crosswind direction, and the Z-axis is vertically upward; The steps of converting the world coordinate system into the blade coordinate system in S6 include: Translation: moving the origin to the blade root center

[0054] Rotation: compensating for the real-time rotation angle of the blade

[0055] where (x, y) ) is the world coordinates of the blade root center; is the real-time rotation angle of the blade.

[0056] The conversion of the blade coordinate system into the flapping displacement value of the blade in S6 is calculated by the following formula:

[0057] where L is the length of the blade, is the flapping displacement value, The blade bending deformation compensation amount is:

[0058] wherein, , is the first-order and second-order mode amplitude, , is the natural frequency.

[0059] The embodiment of the present application has the advantages of simple operation, high detection precision, simple structure, low cost, low sensitivity to the environment, easy deployment and the like. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a schematic diagram of the entire detection system in the embodiment of the present application; Figure 2 is a schematic diagram of a binocular polarization camera in the embodiment of the present application; Figure 3 is a ranging principle diagram of a binocular camera in the embodiment of the present application; Figure 4 is a flowchart for solving the blade flapping displacement in the embodiment of the present application.

[0061] wherein, 1, circular polarizer, 2, polarization vision system, 3, remote terminal, 2.1, right eye polarization camera; 2.2, optical center of the right eye polarization camera; 2.3, baseline midpoint; 2.4, optical center of the left eye polarization camera; 2.5, left eye polarization camera; 2.6, slide rail for controlling the distance between the optical centers of the two polarization cameras; 3.1, length of the photo taken by the left eye polarization camera; 3.2, length of the photo taken by the left eye polarization camera; 3.3, optical center of the left eye polarization camera; 3.4, center of the polarizer; 3.5, baseline length of the left and right eye polarization cameras; 3.6, distance from the center of the polarizer to the camera; 3.7, length of the photo taken by the right eye polarization camera, which is the same as 3.1; 3.8, length of the photo taken by the right eye polarization camera, which is the same as 3.1; 3.9, optical center of the right eye polarization camera. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings.

[0063] The first aspect of the embodiment provides a blade flapping displacement monitoring system based on a binocular polarization camera, which combines Figure 1As shown, it includes a circular polarizer 1, a polarization vision system 2, and a remote terminal 3. The polarization vision system 2 is placed in front of the blade or in front of the side of the blade to collect the image information of the circular polarizer at the blade tip, and the remote terminal 3 is located in the observation room away from the sea surface to receive the image information of the polarization vision system 2, analyze and calculate the blade waving information.

[0064] The circular polarizer 1 is composed of a λ / 4 wave plate and a linearly polarized reflective film to generate a rotationally invariant polarization characteristic.

[0065] The polarization vision system 2 is composed of two same model monocular polarization cameras and two electrically controlled liquid crystal phase delay devices. The polarization cameras are high frame rate cameras triggered synchronously and arranged in parallel with a fixed baseline distance.

[0066] The polarization vision system is fixedly placed in front of the impeller, and the placement distance is within the maximum depth of field of the camera to obtain clear photos, and the baseline of the two cameras is parallel to the horizontal line. The remote terminal is located in a room away from the sea surface or the monitoring point to perform image processing and coordinate conversion operations.

[0067] Combined Figure 2 As shown, the polarization vision system 2 includes left and right monocular polarization cameras 2.5 and 2.1 and a sliding guide rail 2.6 for controlling the baseline length of the two cameras to perform distortion removal and stereo rectification. The fixed baseline distance and the camera shooting parameters are the same as the frequency of the polarizer photos to ensure the accuracy of the calculation of the blade waving displacement.

[0068] The electrically controlled liquid crystal phase delay device is placed in front of the left and right monocular polarization camera lenses, and the polarization direction of the linearly polarized light passing through it is changed by adjusting the voltage, so as to quickly obtain polarization images with linear polarization angles of 0°, 45° and 90°, respectively. 、 、 、 The functions of each polarization analysis state are as follows: 、 Polarized light is mainly used to reduce the reflection of natural light from the sky and enhance the adaptability of non-electric buoys to different weather. and Polarized light provides intermediate direction information and enhances the outline of the circular polarizer. After the two-side polarization cameras complete the same frequency shooting, the polarization vision system numbers and sends the collected images to the remote terminal to solve the circular center pixel coordinates of the circular polarizer, and finally calculates the waving displacement of the blade through coordinate conversion.

[0069] Combined Figure 3As shown, 3.9, 3.3 are the optical centers of the right and left polarization cameras respectively, and the distance between them is 3.5, which is called the baseline distance b, and the midpoint of the baseline is the origin of the baseline coordinate system converted from the camera coordinate system. 3.1 and 3.7 are the pixel lengths of the photos taken by the left and right polarization cameras, and their values are equal. 3.2 and 3.8 are the lengths of the pixel coordinates U direction of the center of the polarizer in the polarization pictures taken by the left and right polarization cameras, respectively, which are expressed as 3.4 is the center P of the polarizer at the tip of the leaf, and 3.6 is the actual distance Z of the circular shape from the binocular polarization camera. For the binocular polarization camera to measure the distance from the object, the calculation formula is as follows:

[0070] =

[0071] wherein, is the physical focal length of the camera, is the length of the pixel coordinates U direction from the left side of the picture in the polarization pictures taken by the left and right polarization cameras, and b is the baseline distance.

[0072] The remote terminal 3 performs calibration and correction of the camera before receiving the picture information taken by the polarization vision system 2, and then performs de-distortion and stereo correction of the image to ensure that the corresponding points are located on the same row scan line and satisfy the epipolar geometry condition.

[0073] The camera calibration and correction includes the following steps: each monocular polarization camera takes at least 20 photos of the checkerboard at different angles, and the checkerboard has a size of 9*9 and a side length of 20mm. The obtained pictures are imported into the CameraCalibrator module in Matlab to obtain the intrinsic matrix K and distortion parameters (radial k1, k2, k3, tangential p1, p2) of each monocular polarization camera, calculate the re-projection error, and ensure that it is <=0.05 pixels to confirm the accuracy of the parameters. The binocular camera synchronously takes at least 20 photos of the checkerboard at different angles, and the obtained pictures are imported into the Lidar Camera Calibrator module in Matlab to obtain external parameters such as the relative rotation matrix R and the translation vector T. Calculate the re-projection error to ensure that it is less than or equal to 0.1 pixels, and the epipolar line matching error (i.e. the vertical deviation of the corrected points) is less than or equal to 0.1 pixels.

[0074] The de-distortion and stereo correction includes the following steps: using the obtained intrinsic and extrinsic parameters, the radial and tangential distortion of the image is removed. The Bouguet algorithm is applied to generate mapping matrices HL and HR to realize epipolar line horizontal correction, and the left and right images after correction satisfy that the corresponding points are located on the same row.

[0075] Remote terminal 3 performs noise reduction processing on the image acquired by polarization vision system 2, then calculates Stokes parameters, degree of polarization (DoLP), and angle of polarization (AoP); next, it calculates hue, saturation, and brightness information to generate an HSV polarization image. Based on the polarization image, it segments regions with high polarization and those containing circular arc edges, and performs ellipse fitting on the arcs within these regions. After 3D reconstruction, the 3D coordinates of the polarizer center are extracted, and the flapping displacement in the blade coordinate system is obtained through a four-level coordinate transformation.

[0076] In practice, the following formula is used to denoise the polarization image: ; Where: G is the Gaussian kernel. is the standard deviation of the Gaussian function.

[0077] Stokes parameters , , The calculation method is as follows:

[0078]

[0079] .

[0080] The method for calculating the degree of polarization DoLP and the polarization angle AoP is as follows:

[0081]

[0082] Draw The specific steps for image processing include: Regarding hue, Mapped to the standard color wheel The range, the formula is:

[0083] For saturation, use directly. The saturation in the high polarization region is close to 1, and in the low polarization region it is close to 0.

[0084] For brightness, use normalization. Preserve scene brightness information: .

[0085] right In the image High polarization region Edge detection, then arc detection in the region, the existence of arc segment, as a polarizer processing area; Extracting the contour point set of the polarizer from the left and right polarized cameras 、 Where is the total number of contour points, the polarizer center pixel coordinates (x, y) calculated from the left and right polarized cameras are: 、 、

[0086]

[0087]

[0088]

[0089] The two-dimensional coordinates of the polarizer center need to be converted to the camera coordinate system through the following steps: Eliminate lens distortion to get the polarizer center pixel coordinates of the left eye:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] Where, , is the radial distance from the image center to the point , and represents the distance of the point from the optical center.

[0096] The corrected coordinates and are:

[0097]

[0098] Where: 、 Radial distortion in and ​​correction amount of tangential distortion in the direction of and the direction of. These values are calculated from the tangential distortion formula and represent the pixel offset that needs to be adjusted due to tangential distortion; 、 — the total distortion correction value, which is the sum of the radial and tangential distortion correction amounts. 、 — the total distortion correction value, which is the sum of the radial and tangential distortion correction amounts. 、 — the total distortion correction value, which is the sum of the radial and tangential distortion correction amounts.

[0099] Similarly, the polarized lens center pixel coordinates of the right eye target after distortion correction are obtained as 、 .

[0100] Calculate the disparity

[0101] Target three-dimensional depth:

[0102] wherein, is the physical focal length of the camera, is the length of the pixel coordinate U in the polarized picture taken by the left and right eye polarized cameras from the left side of the picture, and b is the baseline distance.

[0103] The polarized image center pixel coordinates of the left eye polarized camera (u, v) are converted into the camera coordinate system (X, Y, Z) (using the corrected coordinates): 、 、 、

[0104]

[0105]

[0106] wherein: , , is the physical focal length of the camera, and are the physical sizes of the pixels; is the image center point coordinate.

[0107] The origin of the pixel coordinate system is the center of the upper left pixel of the image, the u-axis is positive to the right of the origin, and the v-axis is positive upward. The center of the camera coordinate system is at the optical center of the left polarized camera, the Z-axis points to the scene (depth of field direction) along the optical axis, the X-axis is parallel to the image plane and points to the right, and the Y-axis is perpendicular downward and opposite to the v-axis.

[0108] ​​The first level of the four-level coordinate conversion is to convert the camera coordinate system into the baseline coordinate system by the following content: Purpose: unify the left and right camera data to the center point of the binocular system Input: left camera coordinates ( ) Output: baseline coordinates ( , , ) The transformation formula is:

[0109] B is the baseline distance of the left and right cameras; move the coordinate origin to the center of the binocular system and eliminate the asymmetry of the viewing angle The origin of the baseline coordinate system is the midpoint of the baseline of the two cameras, the X-axis points to the right camera along the baseline, the Y-axis is perpendicular downward, and the Z-axis is the depth direction.

[0110] The second level of the four-level coordinate conversion is to convert the baseline coordinate system into the world coordinate system by the following content: Purpose: convert the measurement value to the global coordinate system of the fan Input: ( ) Output: world coordinates ( ) Transformation formula:

[0111] Where: R is a 3*3 rotation matrix (calculated by the tower drum calibration point) T is the translation vector [Tx, Ty, Tz] T , calibration method: arrange 4 known world coordinate marker points on the tower drum, and solve R and T by least squares method.

[0112] The origin of the world coordinate system is the center point of the tower drum bottom, the X-axis is the main wind direction (historical average wind direction), the Y-axis is the horizontal crosswind direction, and the Z-axis is vertically upward (opposite direction of gravity), wherein: the tower drum center coordinates can be measured by GNSS, and the main wind direction can be determined by electronic compass.

[0113] The third level of the four-level coordinate conversion is to convert the world coordinate system into the blade coordinate system by the following steps: Purpose: compensate for the blade rotation and obtain the relative displacement of the blade.

[0114] Input: world coordinates ( ).

[0115] Output: blade coordinates ( ).

[0116] Translation: move the origin to the blade root center

[0117] Rotation: compensate the real-time rotation angle of the blade

[0118] wherein: is the world coordinate of the blade root center; is the real-time rotation angle of the blade (which can be obtained by an encoder) The origin of the blade coordinate system is the geometric center of the blade root section, the X-axis points to the leading edge in the chord direction, the Y-axis points to the pressure surface in the normal direction, and the Z-axis points to the blade tip along the blade axis. The blade root center can be measured by a laser tracker and the rotation angle encoder provides real-time feedback.

[0119] The fourth level of coordinate conversion is the conversion from the blade coordinate system to the flapping displacement, which is carried out by the following steps.

[0120] Purpose: output the displacement of the blade tip relative to the blade root Input: blade coordinates Output flapping displacement Calculation formula:

[0121] wherein: L is the length of the blade is the compensation amount of blade bending deformation

[0122] , is the amplitude of the first and second order mode shape , is the natural frequency For the above blade tip displacement coordinates, the displacement in the y direction is the flapping displacement The second aspect of the embodiment provides a blade flapping displacement monitoring method based on a binocular polarized camera, which combines Figure 4 as shown, and is implemented by the following steps.

[0123] S1: Fix a plurality of circular polarizers 1 at the blade tips of the three blades, and fix the polarized vision system 2 in front of the blades or in the side front for collecting image information of the circular polarizers at the blade tips.

[0124] Before starting to obtain the image information of the circular polarizers at the blade tips, the camera needs to be calibrated and corrected.

[0125] ​​Each monocular polarization camera takes at least 20 photos of the checkerboard from different angles, with the checkerboard being 9*9 and the side length being 20mm. The obtained pictures are imported into the Camera Calibrator module in Matlab to obtain the intrinsic matrix K of each monocular polarization camera, the distortion parameters (radial k1, k2, k3, tangential p1, p2), calculate the reprojection error, and ensure that it is <=0.05 pixels to confirm the accuracy of the parameters. The binocular camera synchronously takes at least 20 photos of the checkerboard from different angles, and the obtained pictures are imported into the Lidar Camera Calibrator module in Matlab to obtain the external parameters such as the relative rotation matrix R and the translation vector T. The reprojection error is calculated to ensure that it is less than or equal to 0.1 pixels, and the epipolar line matching error (i.e. the vertical deviation of the corrected points) is less than or equal to 0.1 pixels.

[0126] wherein the intrinsic matrix K is , , , is the physical focal length of the camera, and are the physical sizes of the pixels, and are the principal point offsets. S2: Image shooting and noise reduction.

[0127] The two side polarization cameras of the polarization vision system 2 are made to shoot the same circular polarizer at the same frequency, and polarized images with linear polarization angles of , , , are obtained, which are respectively denoted as , , , , and each image is subjected to noise reduction processing using Gaussian filtering:

[0128] wherein G is the Gaussian kernel, is the standard deviation of the Gaussian function.

[0129] S3: Calculation of polarization parameters.

[0130] The denoised polarization images are used to calculate the Stokes parameters , , , and the global maximum value and the minimum value of are counted for subsequent normalization:

[0131]

[0132]

[0133] Subsequently, the polarization degree is calculated and the polarization angle :

[0134] .

[0135] S4: HSV fusion image generation.

[0136] For hue, the is mapped to the range of the standard hue circle, and the formula is:

[0137] For saturation, the is directly used, and the saturation of the high polarization area is close to 1, and the saturation of the low polarization area is close to 0:

[0138] For lightness, the normalized is used, and the scene brightness information is retained:

[0139] Subsequently, the image can be drawn according to the above operation results.

[0140] S5: Calculation of the center pixel coordinates of the circular polarizer in the images taken by the left and right eye cameras.

[0141] Edge detection is performed on the high polarization area of in the image , and then arc detection is performed on the area, and the part where the arc exists is divided and regarded as the polarizer processing area; The contour point set of the polarizer taken by the left and right eye polarization cameras is extracted , , wherein is the total number of contour points, and the center coordinates of the circular polarizer in the pixel coordinate system are calculated, and the center pixel coordinates of the polarizer taken by the left and right eye polarization cameras are obtained , , , :

[0142]

[0143]

[0144]

[0145] The origin of the pixel coordinate system is defined as the center of the upper left pixel of the image, the u-axis is the positive right direction of the origin, and the v-axis is the positive upward direction of the origin.

[0146] S6: Convert the two-dimensional circular center coordinates of the circular polarizer in the left and right polarized pictures into three-dimensional camera coordinates.

[0147] The following steps are required for camera coordinate system conversion of the two-dimensional coordinates of the polarizer center: Eliminate lens distortion to obtain the polarizer center pixel coordinates of the left eye:

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] wherein, , is the radial distance from the center of the image to the point , represents the distance of the point from the optical center.

[0154] The corrected coordinates and are:

[0155]

[0156] wherein: , respectively represent the correction amount of the radial distortion in the and directions, which are calculated by the radial distortion formula, and represent the pixel offset that needs to be adjusted due to radial distortion; , respectively represent the correction amount of the tangential distortion in the and directions, which are calculated by the tangential distortion formula, and represent the pixel offset that needs to be adjusted due to tangential distortion; , respectively, are the total distortion correction values, which are the sum of the radial distortion and tangential distortion correction amounts.

[0157] Similarly, the polarized image center pixel coordinates of the right eye after distortion correction are obtained as , .

[0158] Calculate the parallax

[0159] Target three-dimensional depth:

[0160] wherein, is the physical focal length of the camera, is the length of the pixel coordinate U direction from the left side of the polarized picture taken by the left and right polarized cameras, and b is the baseline distance.

[0161] The conversion formula of the polarized image center pixel coordinates of the left eye polarized camera , and the camera coordinate system , , :

[0162]

[0163]

[0164] wherein: wherein: , , is the physical focal length of the camera, and are the physical sizes of the pixels; is the image center point coordinate.

[0165] The origin of the camera coordinate system is defined at the optical center of the left polarized camera, the Z axis points to the scene along the optical axis (depth of field direction), the X axis is parallel to the image plane and points to the right and is in the same direction as u, and the Y axis is perpendicular and points downward and is in the opposite direction of v.

[0166] S7: Convert the camera coordinate system to the baseline coordinate system Converting the camera coordinate system to the baseline coordinate system is to translate the coordinate system with the optical center of the left polarized camera as the origin to the coordinate system with the midpoint of the baseline 3.5 as the origin.

[0167] The purpose is to unify the left and right camera data to the center point of the binocular system, the input is the left camera coordinate , and the output is the baseline coordinate , , Its transformation formula is:

[0168] B represents the baseline distance between the left and right cameras; the origin of the coordinate system is moved to the center of the binocular system to eliminate the asymmetry of the viewing angle.

[0169] The origin of the baseline coordinate system is defined as the midpoint between the baselines of the two cameras. The X-axis points along the baseline to the right camera, the Y-axis points vertically downwards, and the Z-axis is the depth of field direction.

[0170] S8: Convert the baseline coordinate system to the world coordinate system Its purpose is to transform the measured values ​​to the wind turbine's global coordinate system, with the baseline coordinate system as the input. The output is in world coordinates ( ), Transformation formula:

[0171] Where: R is a 3x3 rotation matrix (calculated using the tower calibration points), and T is the translation vector [Tx, Ty, Tz]. T Calibration method: Four markers with known world coordinates are placed on the tower, and R and T are solved by the least squares method.

[0172] The origin of the world coordinate system is defined as the center point of the tower bottom. The X-axis is the prevailing wind direction (historical average wind direction), the Y-axis is the horizontal crosswind direction, and the Z-axis is vertically upward (opposite to gravity). The GNSS measures the center coordinates of the tower and the electronic compass determines the prevailing wind direction.

[0173] S9: Transform the world coordinate system into the blade coordinate system Its purpose is to compensate for the rotational motion of the blades and obtain their relative displacement. Its input is the world coordinate system (…). The output is in the blade coordinate system. The conversion involves the following two steps: Translation: Move the origin to the center of the leaf root.

[0174] Rotation: Compensation for real-time rotation angle of the blades

[0175] in:( () represents the world coordinates of the leaf root center; The real-time rotation angle of the blade (obtained by the encoder).

[0176] Define the origin of the blade coordinate system as the geometric center of the blade root section, with the X-axis pointing chordally to the leading edge, the Y-axis pointing normally to the pressure surface, and the Z-axis pointing along the blade axis to the blade tip.

[0177] S10: Convert the blade coordinate system into the flap displacement value of the blade The purpose is to output the displacement of the blade tip relative to the blade root. The input is the blade coordinate system (x, y) ), the output is the flap displacement (y) , and the calculation formula is:

[0178] Where: L is the length of the blade is the blade bending deformation compensation amount

[0179] , is the first-order and second-order mode amplitude , is the natural frequency For the above blade tip displacement coordinate, the displacement in the y direction is the flap displacement value.

[0180] The various coordinate conversions described in S7-S10 can be solved by stage error control, dynamic parameter decoupling and physical constraint display modeling, solving the three core problems in blade flap displacement monitoring: precision decay, dynamic response and environmental adaptability.

[0181] The above only discloses a preferred embodiment of the present application, of course, cannot be limited by the scope of the patent rights of the present application, therefore, the equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. A system for monitoring leaf flutter displacement based on a binocular polarized camera, characterized in that, The application relates to a wind turbine blade tip displacement measurement system, which comprises a circular polarizer (1), a polarization vision system (2) and a remote terminal (3), the circular polarizer (1) is fixed at the tip of a blade, the polarization vision system (2) is arranged in front of or on the side of the blade for collecting image information of the circular polarizer at the tip of the blade, and the remote terminal (3) is used for receiving the image information of the polarization vision system (2), wherein the polarization vision system (2) is composed of two single-lens polarization cameras and two electrically-controlled liquid crystal phase delay devices. The remote terminal (3) carries out noise reduction processing on the image collected by the polarization vision system (2), and then calculates Stokes parameters, a polarization degree DoLP and a polarization angle AoP; then, hue, saturation and brightness information is calculated, an HSV polarization image is generated, a high polarization degree and a circular arc edge existing region are segmented from the polarization image, an ellipse fitting is carried out on the circular arc in the region, three-dimensional coordinates of a center of the polarization plate are extracted after three-dimensional reconstruction, and a flapwise displacement in a blade coordinate system is obtained through four-stage coordinate conversion.

2. The dual-view polarimetric camera based leaf flutter displacement monitoring system according to claim 1, wherein, The HSV polarization image is generated in the remote terminal (3) by the following method: For hue, the polarization angle is mapped to the standard hue circle ranging from 0 to 360 degrees, with the formula: For saturation, directly use the polarization degree , high polarization region saturation close to 1, low polarization region close to 0: For lightness, the normalized , preserving scene brightness information: 。 3. The dual-view polarimetric camera based leaf flutter displacement monitoring system according to claim 2, wherein, In the remote terminal (3), the circular polarizer origin center pixel coordinates (uL, vL) and (uR, vR) in the picture are obtained by the following method: right In the image High polarization region Edge detection is performed, followed by arc detection within the region. The portion containing the arc is segmented and treated as a polarizer processing area. Extracting contour point sets of a polaroid taken by left and right polarized cameras , wherein is the total number of contour points, polaroid center pixel coordinates (x, y) taken by left and right polarized cameras are calculated , , , ​ 。 。 4. The dual-view polarimetric camera based leaf flutter displacement monitoring system according to claim 3, wherein, In the remote terminal (3), the following steps are required for camera coordinate system conversion of the two-dimensional coordinates of the center of the polarization plate: Eliminate lens distortion: wherein , the modified coordinates and are: Computing parallax Target three-dimensional depth: wherein, is the physical focal length of the camera, is the length of the U direction in the polarized picture taken by the left and right polarization cameras, and b is the baseline distance. Polarization image center pixel coordinates of left eye polarization camera (xL, yL) , ) converted to camera coordinate system (xL, yL) , , ): wherein: wherein: , , is the physical focal length of the camera, and is the physical size of the pixel; is the image center point coordinate.

5. The dual-view polarimetric camera based leaf flutter displacement monitoring system according to claim 4, wherein, In the remote terminal (3), the flapwise displacement value is calculated by the following method: wherein: L is the blade length, is the blade bending deformation compensation, , , is the first and second mode shape amplitude, , is the natural frequency, is the edgewise displacement, is the input in the blade coordinate system.

6. A method for monitoring blade flapwise displacement based on a binocular polarized camera, characterized in that, The method comprises the following steps: S1: Fix multiple circular polarizers (1) at the tips of three blades in the same position. Place a polarization vision system (2) in front of or to the side of the blades to collect image information of the circular polarizers at the tips of the blades. Obtain the linear polarization angles as follows through the polarization vision system (2): , , , Polarization image; S2: denoising the polarized image, calculating Stokes parameters, degree of polarization and polarization angle ; S3: using the polarization degree and the polarization angle adjusting the saturation, hue and lightness, respectively, and plotting the image; S4: According to the The center pixel coordinates of the circular polarizer in the images taken by the left and right eyes of the polarized vision system (2) are calculated. S5: converting the two-dimensional center coordinates of the circular polarizer in the left and right polarization pictures into three-dimensional camera coordinates; S6: sequentially converting the three-dimensional camera coordinates into a baseline coordinate system, a boundary coordinate system, a blade coordinate system and a flapwise displacement value.

7. The bimodal polarimetric camera based leaf flutter displacement monitoring method according to claim 6, wherein, The center pixel coordinates of the circular polarizer in the S4 are calculated by the following method: Regarding the In the image High polarization region Edge detection is performed, followed by arc detection within the region. The portion containing the arc is segmented and treated as a polarizer processing area. Extracting contour point sets of polaroid taken by left and right polarized cameras 、 wherein is the total number of contour points, the center coordinates of the circular polaroid in the pixel coordinate system are calculated, and the center pixel coordinates of the polaroid taken by the left and right polarized cameras are obtained 、 , 、 : The origin of the pixel coordinate system is defined as the pixel center of the upper left corner of the image, the u-axis is defined as the positive right direction of the origin, and the v-axis is defined as the positive upward direction of the origin.

8. The bimodal polarimetric camera based leaf flutter displacement monitoring method according to claim 7, wherein, In the S5, the following steps are required for camera coordinate system conversion of the two-dimensional coordinates of the center of the polarization plate: Eliminate lens distortion to obtain the polarization plate center pixel coordinates of the left camera: wherein , is the radial distance from the image center to the point , denotes the distance of the point from the optical center; corrected coordinates and are: wherein , is the correction amount of the radial distortion in the and directions, , is the correction amount of the tangential distortion in the and directions, , is the total distortion correction value; calculating the polarizing plate center pixel coordinates of the right eye after distortion correction , ; Computing parallax ; Target three-dimensional depth: ; wherein, is the physical focal length of the camera, is the length of the U direction in the polarized picture taken by the left and right polarization cameras, and b is the baseline distance. The center pixel coordinates of the polarization image from the left-eye polarization camera ( , ) and camera coordinate system ( , , The conversion formula for ) is: wherein: wherein: , , is the physical focal length of the camera, and is the physical size of the pixel; is the image center point coordinate.

9. The bimodal polarimetric camera based leaf flutter displacement monitoring method according to claim 8, wherein, In the S6, the camera coordinate system is converted into the baseline coordinate system by the following steps: Converting the camera coordinate system into the baseline coordinate system is to translate the coordinate system with the optical center of the left polarization camera as the origin into the coordinate system with the midpoint of the baseline as the origin. The purpose is to unify the left and right camera data to the center point of the binocular system, the input is the left camera coordinate ( ), the output is the baseline coordinate ( , , ), and the transformation formula is: Wherein, B is the baseline distance of the left and right cameras; the coordinate origin is moved to the center of the binocular system to eliminate the visual angle asymmetry; The origin of the baseline coordinate system is defined as the midpoint of the baseline of the two cameras, the X-axis is defined as the direction of the right camera along the baseline, the Y-axis is defined as the vertical downward direction, and the Z-axis is defined as the depth direction; In the S6, the baseline coordinate system is converted into the world coordinate system by the following formula: The transformation formula is: wherein: R is a 3*3 rotation matrix, T is a translation vector [Tx, Ty, Tz] T ; The origin of the world coordinate system is defined as the center point of the tower drum bottom, the X-axis is defined as the main wind direction, the Y-axis is defined as the horizontal crosswind direction, and the Z-axis is defined as the vertical upward direction; In the S6, the steps of converting the world coordinate system into the blade coordinate system comprise: Translation: moving the origin to the center of the blade root Rotation: compensating for the real-time rotation angle of the blade wherein: ) is the blade root center world coordinate; is the blade real-time rotation angle.

10. The bimodal polarimetric camera based blade flapwise displacement monitoring method according to claim 9, wherein, The conversion of the blade coordinate system into the flapwise displacement value of the blade in the S6 is calculated by the following formula: wherein: L is the length of the blade, is the flapwise displacement value, is the blade bending deformation compensation amount: wherein , is the first and second mode shape amplitude, , is the natural frequency.

Images