Real-time monitoring system and method for verticality of fan tower drum based on machine vision recognition

Abstract

The application provides a wind turbine tower verticality real-time monitoring system and method based on machine vision recognition, and belongs to the field of wind power generation equipment monitoring, and comprises the following steps: determining a plurality of camera arrangement positions around a wind turbine tower, calculating a compensation pitch angle according to terrain slope parameters of each arrangement position and a camera azimuth relationship; controlling all cameras to synchronously collect tower images, determining a master position camera and a slave position camera set; calculating an edge definition index of the master position camera image, and enhancing the master position camera image; dividing the enhanced master position camera image into blocks along a height direction, performing edge detection on each block by using a corresponding threshold to obtain contour pixel coordinates; calculating a tilt angle, and outputting an early warning level and a monitoring report according to a relationship between the tilt angle and a warning threshold. The application realizes site-specific pitch angle compensation, automatically switches an optimal measurement visual angle according to real-time image quality and a tilt direction, and improves the accuracy and stability of edge detection.

Description

Technical Field

[0001] This invention relates to the field of wind power equipment monitoring technology, and in particular to a real-time monitoring system and method for the verticality of wind turbine towers based on machine vision recognition. Background Technology

[0002] During long-term operation, wind turbine towers can tilt due to uneven foundation settlement, accumulated wind load fatigue, and loose bolt connections. If the tower tilt exceeds a safe threshold, it can lead to increased turbine vibration, uneven load on the transmission system, and stress concentration in the structure, potentially causing collapse in severe cases. Therefore, real-time monitoring of the wind turbine tower's verticality is crucial for ensuring the safe operation of the unit.

[0003] Existing methods for monitoring the verticality of towers mainly include tilt sensor methods, laser ranging methods, and single-camera vision measurement methods. The tilt sensor method requires installing sensors and laying cables inside the tower, resulting in high installation and maintenance costs and long-term drift issues. While laser ranging offers high accuracy, the equipment is expensive, and its reliability decreases in adverse weather conditions such as fog, rain, and snow. Single-camera vision measurement methods are low-cost and non-contact, but they have two limitations: first, the perspective distortion of the cylinder is severe under a single viewpoint, limiting the accuracy of edge contour extraction; second, outdoor lighting conditions vary greatly, and image quality deteriorates significantly when the camera is in backlight or in foggy weather, leading to measurement failure.

[0004] Chinese patent CN116539001A discloses a method for detecting the verticality of offshore wind turbine towers based on unmanned aerial vehicles (UAVs). This method uses a camera and a dual-axis tilt sensor mounted on the UAV to generate a horizontal image through attitude angles acquired by the sensors, employs a Mask R-CNN neural network for tower target recognition, extracts tower edge features using Canny edge detection, and finally fits the centerline to calculate the verticality. However, its measurement accuracy is limited in complex terrain environments such as mountainous areas, and it struggles to maintain detection quality across different regions of the image in outdoor scenes with uneven lighting. Summary of the Invention

[0005] In view of this, the present invention proposes a real-time monitoring system and method for wind turbine tower verticality based on machine vision recognition, which solves the technical problem of detection accuracy affected by the environment in the prior art, improves the accuracy and stability of edge detection, and realizes real-time monitoring of wind turbine tower verticality with high robustness and high precision.

[0006] The technical solution of this invention is implemented as follows: On one hand, the present invention provides a real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition, comprising: The calibration module is used to determine the deployment positions of multiple cameras around the wind turbine tower. It calculates the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each deployment position, installs cameras at each deployment position according to the compensation pitch angle, obtains the intrinsic and extrinsic parameters of each camera, and establishes a global coordinate system with the center of the bottom of the tower as the origin. The camera selection module is used to control all cameras to synchronously acquire images of the tower. It scores the quality of each camera image based on contrast, sharpness, and brightness deviation. It obtains the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction. Based on the image quality score, the initial tilt direction angle, and the tilt angle, it determines the set of main position cameras and slave position cameras. The image processing module is used to calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than a preset threshold, the contour detected from the set of position cameras is projected onto the main position camera view to obtain contour guidance. Based on the contour guidance, the main position camera image is enhanced to obtain the enhanced main position camera image. The contour extraction module is used to divide the enhanced main position camera image into blocks along the height direction. For each block, the edge detection threshold is adaptively calculated based on the gradient statistical features and the brightness difference between blocks. The corresponding threshold is used to perform edge detection on each block and merge them to obtain a complete edge point set. The left and right contour lines are fitted by the least squares method and the contour pixel coordinates are extracted. The measurement output module is used to convert the contour pixel coordinates into three-dimensional spatial coordinates through the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. It calculates the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle, and outputs the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

[0007] Based on the above technical solutions, preferably, the deployment and calibration module specifically includes: Determine the locations of multiple cameras around the wind turbine tower. The cameras are evenly distributed around the tower to form a ring array. The horizontal distance from each camera location to the central axis of the tower is set according to the tower height. Measure the slope angle and aspect angle of each camera placement position relative to the horizontal reference plane, where the aspect angle is zero degrees in the due north direction and positive in the clockwise direction; The ideal pitch angle is determined based on the height of the wind turbine tower and the horizontal distance of each camera from the central axis of the tower: ; in, Indicates the ideal pitch angle. Indicates the height of the wind turbine tower. This indicates the horizontal distance from the camera's location to the central axis of the tower. The ideal pitch angle is compensated based on the terrain slope angle, aspect angle, and camera azimuth angle at each camera deployment location to obtain the compensated pitch angle: ; in For the first Camera No. 1 compensates for pitch angle. The slope angle of the terrain. The slope angle, For the first The azimuth angle of camera number 1; Install the cameras at their respective locations according to the compensated pitch angle; Using the focal length, principal point coordinates, and distortion coefficients of each camera as intrinsic parameters, and obtaining the rotation matrix and translation vector of each camera relative to the center coordinate system of the tower as extrinsic parameters, a global coordinate system with the center of the bottom of the tower as the origin is established, where the vertical direction upward is the Z-axis, the horizontal direction due north is the X-axis, and the principal point coordinates are the positions of the intersection of the camera's optical axis and the image plane in the pixel coordinate system.

[0008] Based on the above technical solutions, preferably, the step of obtaining the preliminary tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction specifically includes: Canny edge detection is used to obtain edge point sets for each camera image. Hough line transform is used to detect the left and right contour lines of the tower, thus obtaining two-dimensional contour points in each camera image. By using the external parameter rotation matrix and translation vector of each camera, the two-dimensional contour points are transformed from the camera coordinate system to the global coordinate system, and the three-dimensional contour line of the tower in the global coordinate system is obtained, where the three-dimensional contour line includes the three-dimensional contour points. The average horizontal coordinates of the left and right three-dimensional contour points obtained by each camera at each height are calculated to obtain the position point of the tower's central axis at the corresponding height. The central axis position point is then fitted with a three-dimensional spatial straight line to obtain the central axis direction vector corresponding to each camera. The average tilt direction angle and tilt angle of the tower are obtained by averaging the direction vectors of the central axis corresponding to all cameras. The tilt direction angle is the clockwise angle between the projection of the central axis on the horizontal plane and the due north direction, and the tilt angle is the angle between the central axis and the vertical direction.

[0009] Based on the above technical solutions, preferably, the step of determining the set of primary and secondary cameras based on image quality scores, preliminary tilt direction angles, and tilt angles specifically includes: Determine if the initial tilt angle exceeds the initial threshold. If it does, the tower is confirmed to be tilted, and the master-slave camera selection process begins. The tilt direction sector is determined based on the initial tilt direction angle, and the camera with the highest image quality score within the tilt direction sector is selected as the main position camera, thus obtaining the main position camera number; wherein, the tilt direction sector includes cameras whose absolute value of the difference between the azimuth angle and the initial tilt direction angle is less than a threshold. In addition to the main camera, cameras with image quality scores greater than a quality threshold are selected as the slave camera set, resulting in a slave camera number set. The slave cameras are used to correct the edge detection results of the main camera image.

[0010] Based on the above technical solutions, preferably, the formula for calculating the edge sharpness is as follows: ; in For edge sharpness indicators, The set of edge points in the main position camera image The number of edge points. Main position camera image in pixel coordinates The gradient magnitude at that point.

[0011] Based on the above technical solutions, preferably, the image processing module specifically includes: Contour detection is performed on the images of each slave camera in the slave camera set to obtain two-dimensional contour points. The two-dimensional contour points are then transformed to the global coordinate system through the extrinsic parameters of the slave camera to obtain three-dimensional contour points. Finally, the three-dimensional contour points are projected onto the image plane of the master camera through the extrinsic parameters of the master camera to obtain the projection position of the slave camera contours under the view of the master camera. Calculate the Euclidean distance from each pixel in the main position camera image to the nearest point in all the projection contours from the position camera as the contour guiding distance, and construct a contour guiding distance field covering the entire main position camera image; The contour-guided distance field is converted into edge-guided enhancement weights using a rational function kernel form. The calculation formula is as follows: ; in pixel coordinates Edge guidance enhances weight, pixel coordinates The outline guides the distance. For scale parameters; An edge enhancement signal is constructed based on the Laplacian operator and gradient magnitude of the image from the main camera position. The calculation formula is as follows: in pixel coordinates Edge enhancement signal at the location, Main position camera image pixel coordinates The Laplace operator at the location, For symbolic functions, Indicates the gradient magnitude; The enhanced main camera image is obtained by adding an enhancement intensity coefficient to the grayscale value of the original image from the main camera, and then multiplying this by the edge-guided enhancement weight and the edge enhancement signal. The calculation formula is as follows: in, Represents pixel coordinates Enhanced main position camera image, Main position camera raw image pixel coordinates grayscale value at that location To enhance the strength coefficient.

[0012] Based on the above technical solutions, preferably, the contour extraction module specifically includes: After Gaussian filtering to denoise the enhanced main position camera image, the Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel; The enhanced main position camera image is evenly divided into five horizontal strip blocks along the height direction; For each horizontal strip block, calculate the mean and standard deviation of the gradient magnitude of the pixels within the block. Then, calculate the maximum mean gradient magnitude across all horizontal strip blocks as the global maximum gradient level. Based on the mean gradient magnitude, standard deviation of the gradient magnitude of each horizontal strip block, and the global maximum gradient level, dynamically adjust the high threshold of each horizontal strip block. The calculation formula is as follows: in, Indicates the first High threshold of horizontal strip blocks For the first The average gradient magnitude of each horizontal strip block For the first The standard deviation of the gradient magnitude of each horizontal strip block. For high threshold coefficients, This represents the maximum value of the average gradient magnitude across all horizontal strip blocks. The low threshold of each horizontal strip block is obtained by multiplying the high threshold of each block by the low-high threshold ratio coefficient, where the low-high threshold ratio coefficient represents the ratio between the low threshold and the high threshold. Non-maximum suppression and hysteresis edge connection are performed on each horizontal strip block using corresponding high and low thresholds. The edge points detected in each block are merged to obtain a complete edge point set. Calculate the median of the horizontal coordinates of all edge points in the complete edge point set, and select edge points with horizontal coordinates less than the median as the left contour candidate point set and edge points with horizontal coordinates greater than the median as the right contour candidate point set. The least squares method is used to fit the linear equations of the vertical coordinates with respect to the horizontal coordinates for the candidate point sets of the left and right contours, respectively, and the slope and intercept of the left contour line and the right contour line are calculated. At several sampling heights uniformly selected along the height direction, the horizontal coordinates of the left and right contours are calculated according to the fitted straight line equations to obtain the contour pixel coordinates. More preferably, the step of converting the contour pixel coordinates into three-dimensional spatial coordinates and fitting them to obtain the direction vector of the tower's central axis using the intrinsic and extrinsic parameters of the main position camera specifically includes: The contour pixel coordinates are converted into normalized image plane coordinates using the focal length and principal point coordinates in the intrinsic parameter matrix of the main position camera, and the ray direction vector from the normalized image plane to the camera coordinate system is calculated. The ray direction vector is transformed from the camera coordinate system to the global coordinate system by using the extrinsic rotation matrix of the main position camera, thus obtaining the ray direction vector in the global coordinate system. Based on the geometric characteristics of the tower, a cylindrical surface equation is established. An equation is established that starts from the optical center of the main position camera and intersects the cylindrical surface of the tower along the ray direction. The equation is then solved to obtain the three-dimensional spatial coordinates of the contour points. For each sampling height, the average of the three-dimensional coordinates of the left and right contour points in the horizontal direction is taken to obtain the position of the tower's central axis at the sampling height. The direction vector of the tower's central axis is obtained by fitting a straight line in three-dimensional space using the least squares method at all heights of the central axis location points; The angle between the direction vector of the tower's central axis and the vertical direction vector is calculated as the tilt angle, using the following formula: in, The tower's tilt angle, The direction vector of the tower's central axis. It is a vertical vector; The azimuth angle of the projection of the tower's central axis onto the horizontal plane is calculated as the tilt direction angle.

[0013] Based on the above technical solutions, preferably, the step of outputting the warning level and monitoring report according to the relationship between the tilt angle and the warning threshold specifically includes: A three-tiered early warning threshold system is established, wherein the first-tier early warning threshold is lower than the second-tier early warning threshold, and the second-tier early warning threshold is lower than the third-tier early warning threshold; The tower is considered to be in normal condition when the tilt angle is less than the first-level warning threshold. When the tilt angle is not less than the first-level warning threshold and less than the second-level warning threshold, a first-level warning recommendation is issued, which is to increase the monitoring frequency. When the tilt angle is not less than the Level II warning threshold and less than the Level III warning threshold, a Level II warning suggestion is issued, which suggests conducting an on-site inspection. A Level 3 warning is issued when the tilt angle is not less than the Level 3 warning threshold, and the warning recommendation is to immediately stop the machine for maintenance. Monitoring reports are generated and stored in a database based on tilt angle, tilt direction angle, warning level, timestamp, and measurement data and quality scores from each camera.

[0014] On the other hand, the present invention provides a method for real-time monitoring of wind turbine tower verticality based on machine vision recognition, applied to the aforementioned real-time monitoring system for wind turbine tower verticality based on machine vision recognition, comprising the following steps: S1. Determine the locations of multiple cameras around the wind turbine tower, calculate the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each location, install cameras at each location according to the compensation pitch angle, obtain the intrinsic and extrinsic parameters of each camera, and establish a global coordinate system with the center of the bottom of the tower as the origin. S2. Control all cameras to synchronously acquire tower images, comprehensively evaluate the contrast, sharpness and brightness deviation of each camera image, obtain the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction, and determine the set of master position camera and slave position camera based on image quality score, initial tilt direction angle and tilt angle. S3. Calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than the preset threshold, the contour detected by the set of position cameras is projected onto the view of the main position camera to obtain contour guidance. The main position camera image is enhanced based on the contour guidance to obtain the enhanced main position camera image. S4. Divide the enhanced main position camera image into blocks along the height direction. For each block, adaptively calculate the edge detection threshold based on the gradient statistical features and the brightness difference between blocks. Use the corresponding threshold to perform edge detection on each block and merge them to obtain a complete edge point set. Fit the left and right contour lines using the least squares method and extract the contour pixel coordinates. S5. Convert the contour pixel coordinates into three-dimensional spatial coordinates using the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. Calculate the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle. Output the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

[0015] The real-time monitoring system and method for wind turbine tower verticality based on machine vision recognition of the present invention has the following advantages over the prior art: (1) By setting up multiple cameras around the tower, taking into account terrain slope compensation and dynamic selection of master and slave cameras, the spatial coupling relationship between terrain slope and camera orientation is established by using cosine function, which realizes pitch angle compensation according to local conditions, ensures the consistency of measurement reference of each camera, and improves the consistency and fusion accuracy of multi-camera measurement results. (2) By selecting the master and slave cameras, tower attitude information can be obtained from different perspectives, and the optimal measurement perspective can be automatically switched according to the real-time image quality and tilt direction, which not only ensures measurement accuracy, but also improves robustness to environmental factors such as light changes and haze. (3) By projecting the contour detected by the camera onto the view of the main camera to construct the contour-guided distance field, the distance field is converted into spatially selective enhancement weights in the form of rational function kernels. This ensures the edge enhancement effect while maintaining the main structure and background information of the main camera image, thus improving the accuracy and stability of edge detection. (3) By dividing the horizontal strip into blocks along the height direction and independently calculating the dual threshold with brightness adaptive factor, the edge detection threshold of each region can adapt to local lighting conditions and gradient features, thereby improving the accuracy and completeness of contour edge detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition according to the present invention. Figure 2 This is a top view of the wind turbine tower of the wind turbine tower real-time verticality monitoring system based on machine vision recognition according to the present invention. Figure 3 This is a side view of the wind turbine tower of the wind turbine tower real-time verticality monitoring system based on machine vision recognition according to the present invention. Figure 4 This is a schematic diagram of image enhancement for the real-time monitoring system for wind turbine tower verticality based on machine vision recognition according to the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this invention provides a real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition, comprising: The calibration module is used to determine the deployment positions of multiple cameras around the wind turbine tower. It calculates the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each deployment position, installs cameras at each deployment position according to the compensation pitch angle, obtains the intrinsic and extrinsic parameters of each camera, and establishes a global coordinate system with the center of the bottom of the tower as the origin. The camera selection module is used to control all cameras to synchronously acquire images of the tower. It scores the quality of each camera image based on contrast, sharpness, and brightness deviation. It obtains the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction. Based on the image quality score, the initial tilt direction angle, and the tilt angle, it determines the set of main position cameras and slave position cameras. The image processing module is used to calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than a preset threshold, the contour detected from the set of position cameras is projected onto the main position camera view to obtain contour guidance. Based on the contour guidance, the main position camera image is enhanced to obtain the enhanced main position camera image. The contour extraction module is used to divide the enhanced main position camera image into blocks along the height direction. For each block, the edge detection threshold is adaptively calculated based on the gradient statistical features and the brightness difference between blocks. The corresponding threshold is used to perform edge detection on each block and merge them to obtain a complete edge point set. The left and right contour lines are fitted by the least squares method and the contour pixel coordinates are extracted. The measurement output module is used to convert the contour pixel coordinates into three-dimensional spatial coordinates through the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. It calculates the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle, and outputs the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

[0020] like Figure 2 and Figure 3 As shown, in one embodiment of the present invention, the deployment calibration module specifically includes: Determine the locations of multiple cameras around the wind turbine tower. The cameras are evenly distributed around the tower to form a ring array. The horizontal distance from each camera location to the central axis of the tower is set according to the tower height. Measure the slope angle and aspect angle of each camera placement position relative to the horizontal reference plane, where the aspect angle is zero degrees in the due north direction and positive in the clockwise direction; The ideal pitch angle is determined based on the height of the wind turbine tower and the horizontal distance of each camera from the central axis of the tower: ; in, Indicates the ideal pitch angle. Indicates the height of the wind turbine tower. This indicates the horizontal distance from the camera's location to the central axis of the tower. The ideal pitch angle is compensated based on the terrain slope angle, aspect angle, and camera azimuth angle at each camera deployment location to obtain the compensated pitch angle: ; in For the first Camera No. 1 compensates for pitch angle. The slope angle of the terrain. The slope angle, For the first The azimuth angle of camera number 1; Install the cameras at their respective locations according to the compensated pitch angle; Using the focal length, principal point coordinates, and distortion coefficients of each camera as intrinsic parameters, and obtaining the rotation matrix and translation vector of each camera relative to the center coordinate system of the tower as extrinsic parameters, a global coordinate system with the center of the bottom of the tower as the origin is established, where the vertical direction upward is the Z-axis, the horizontal direction due north is the X-axis, and the principal point coordinates are the positions of the intersection of the camera's optical axis and the image plane in the pixel coordinate system.

[0021] This invention solves the problem of inconsistent camera installation benchmarks caused by uneven terrain in actual wind farms by deploying multiple cameras around the tower and comprehensively considering terrain slope compensation and dynamic selection of master and slave cameras. It can adapt to complex outdoor environmental conditions and maintain stable measurement performance even under adverse factors such as fog and backlight. By establishing a spatial coupling relationship between terrain slope and camera orientation through a cosine function, it achieves pitch angle compensation adapted to local conditions, ensuring the consistency of measurement benchmarks for each camera and improving the consistency and fusion accuracy of multi-camera measurement results.

[0022] Preferably, six camera locations are set around the tower, with equal azimuth intervals between each location, and the horizontal distance from each camera location to the tower's central axis is set. Tower height The distance is twice that of the tower, ensuring that the camera's field of view covers the entire height of the tower while avoiding excessive image distortion due to being too close or resolution degradation due to being too far. When measuring terrain slope parameters, a digital level or total station is used to measure the elevation of each camera's location relative to the horizontal reference plane. The terrain slope angle is calculated by the elevation difference and horizontal distance between adjacent measuring points. The slope angle ranges from 0 to 20 degrees, and the aspect angle... This represents the clockwise angle between the direction of the maximum slope of the terrain and true north, with a value ranging from 0 to 360 degrees. For flat terrain, the slope angle is close to 0 degrees, at which point the compensation term is close to 0, and the ideal pitch angle is the actual pitch angle.

[0023] In one embodiment of the present invention, when installing the camera, an adjustable pitch angle tripod or fixed bracket is used, and an electronic angle measuring instrument or tilt sensor is used to assist in adjusting the pitch angle to the set value of the compensated pitch angle. When acquiring the camera's intrinsic parameters, the Zhang Zhengyou calibration method is used for camera calibration. When acquiring the camera's extrinsic parameters, calibration points with known three-dimensional coordinates are set on the tower surface. Images of the tower are captured by each camera, and the pixel coordinates of the calibration points are identified in the images. A correspondence between the three-dimensional spatial coordinates and the two-dimensional image coordinates is established. The perspective transformation equation is used to solve for the camera's extrinsic parameter matrix, where the extrinsic parameter matrix includes a rotation matrix and a three-dimensional translation vector.

[0024] In one embodiment of the present invention, obtaining the preliminary tilt direction angle and tilt angle of the tower through edge detection and three-dimensional reconstruction specifically includes: Canny edge detection is used to obtain edge point sets for each camera image. Hough line transform is used to detect the left and right contour lines of the tower, thus obtaining two-dimensional contour points in each camera image. By using the external parameter rotation matrix and translation vector of each camera, the two-dimensional contour points are transformed from the camera coordinate system to the global coordinate system, and the three-dimensional contour line of the tower in the global coordinate system is obtained, where the three-dimensional contour line includes the three-dimensional contour points. The average horizontal coordinates of the left and right three-dimensional contour points obtained by each camera at each height are calculated to obtain the position point of the tower's central axis at the corresponding height. The central axis position point is then fitted with a three-dimensional spatial straight line to obtain the central axis direction vector corresponding to each camera. The average tilt direction angle and tilt angle of the tower are obtained by averaging the direction vectors of the central axis corresponding to all cameras. The tilt direction angle is the clockwise angle between the projection of the central axis on the horizontal plane and the due north direction, and the tilt angle is the angle between the central axis and the vertical direction.

[0025] Specifically, transforming the 2D contour points from the camera coordinate system to the global coordinate system using the extrinsic rotation matrix and translation vector of each camera includes: first, transforming the pixel coordinates of the 2D contour points... Radial and tangential distortion models are used to correct pixel coordinates, and then the camera intrinsic parameter matrix is ​​used to convert the pixel coordinates into normalized image plane coordinates. The normalized image planar coordinates are combined with depth information to convert them into three-dimensional coordinates in the camera coordinate system. The depth information is obtained by assuming that the contour points are located on the cylindrical surface of the tower and by combining the tower radius estimation. Finally, the external parameter rotation matrix is ​​used. Translation vector The formula for transforming the 3D coordinates in the camera coordinate system to the global coordinate system is as follows: in, The horizontal component represents the coordinates of the principal point. Represents the vertical component of the principal point's coordinates. This indicates the camera's focal length in the horizontal direction. This indicates the camera's focal length in the vertical direction. This represents the three-dimensional coordinates of a point in the global coordinate system. This represents the three-dimensional coordinates of a point in the camera coordinate system.

[0026] Understandably, actual wind turbine towers typically have a tapered structure, tapering from top to bottom, with the radius decreasing linearly along the height. In 3D reconstruction, the cylindrical surface equation is modified into a conical surface equation, establishing a linear radius-height relationship using pre-measured top and bottom radii of the tower. ,in Let be the taper coefficient. When solving for the intersection of the ray and the cone, the cylindrical surface constraint condition should be applied. Replace with conical surface constraint The coordinates of the three-dimensional profile points, taking into account the taper, are obtained by solving a quadratic equation. This process allows the central axis fitting to more accurately reflect the actual geometry of the tower, avoiding systematic deviations caused by the ideal cylinder assumption.

[0027] In one embodiment of the present invention, determining the set of primary and secondary cameras based on image quality scores, preliminary tilt direction angles, and tilt angles specifically includes: Determine if the initial tilt angle exceeds the initial threshold. If it does, the tower is confirmed to be tilted, and the master-slave camera selection process begins. The tilt direction sector is determined based on the initial tilt direction angle, and the camera with the highest image quality score within the tilt direction sector is selected as the main position camera, thus obtaining the main position camera number; wherein, the tilt direction sector includes cameras whose absolute value of the difference between the azimuth angle and the initial tilt direction angle is less than a threshold. In addition to the main camera, cameras with image quality scores greater than a quality threshold are selected as the slave camera set, resulting in a slave camera number set. The slave cameras are used to correct the edge detection results of the main camera image.

[0028] The quality of images from each camera is scored by comprehensively considering contrast, sharpness, and brightness deviation. Based on the initial tilt angle, the tilt direction sector is determined, and the camera with the best quality in this sector is selected as the master camera. This solves the problems of limited measurement accuracy and poor environmental adaptability of single-view measurement. By selecting master and slave cameras, tower attitude information can be obtained from different perspectives, and the optimal measurement perspective is automatically switched according to the real-time image quality and tilt direction. This not only ensures measurement accuracy but also improves robustness to environmental factors such as changes in lighting and haze.

[0029] Specifically, the formula for calculating image quality score is as follows: in, Indicates image quality score, The weighting coefficient representing contrast. Weighting coefficients representing sharpness. The weighting coefficient representing the brightness deviation. This represents the contrast ratio after normalization. This indicates the sharpness after normalization. This represents the normalized brightness deviation.

[0030] Understandably, contrast is represented by the standard deviation of the grayscale histogram, sharpness is represented by the average value of the Laplacian operator response, and brightness deviation is represented by the absolute value of the difference between the average grayscale value and the ideal grayscale value.

[0031] In one embodiment of the present invention, the formula for calculating the edge sharpness is: ; in For edge sharpness indicators, The set of edge points in the main position camera image The number of edge points. Main position camera image in pixel coordinates The gradient magnitude at that point.

[0032] like Figure 4 As shown, in one embodiment of the present invention, the image processing module specifically includes: Contour detection is performed on the images of each slave camera in the slave camera set to obtain two-dimensional contour points. The two-dimensional contour points are then transformed to the global coordinate system through the extrinsic parameters of the slave camera to obtain three-dimensional contour points. Finally, the three-dimensional contour points are projected onto the image plane of the master camera through the extrinsic parameters of the master camera to obtain the projection position of the slave camera contours under the view of the master camera. Calculate the Euclidean distance from each pixel in the main position camera image to the nearest point in all the projection contours from the position camera as the contour guiding distance, and construct a contour guiding distance field covering the entire main position camera image; The contour-guided distance field is converted into edge-guided enhancement weights using a rational function kernel form. The calculation formula is as follows: ; in pixel coordinates Edge guidance enhances weight, pixel coordinates The outline guides the distance. For scale parameters; An edge enhancement signal is constructed based on the Laplacian operator and gradient magnitude of the image from the main camera position. The calculation formula is as follows: in pixel coordinates Edge enhancement signal at the location, Main position camera image pixel coordinates The Laplace operator at the location, For symbolic functions, Indicates the gradient magnitude; The enhanced main camera image is obtained by adding an enhancement intensity coefficient to the grayscale value of the original image from the main camera, and then multiplying this by the edge-guided enhancement weight and the edge enhancement signal. The calculation formula is as follows: in, Represents pixel coordinates Enhanced main position camera image, Main position camera raw image pixel coordinates grayscale value at that location To enhance the strength coefficient.

[0033] Understandably, the contour-guided distance field is a two-dimensional matrix with the same size as the image from the main camera, where each element represents the nearest distance from the corresponding pixel to the contour. When a pixel is near the contour location... When the weight is close to 0, Approaching 1; when the pixel is far from the contour position, i.e. When the weight is large, The decay rate rapidly approaches zero. The decay rate of this weighting function is determined by the scale parameter. control, The smaller the value, the faster the decay. Near the contour location indicated by the position camera, the edge guidance enhancement weight is larger, enhancing the edge response; in areas far from the contour location, the edge guidance enhancement weight is close to zero, keeping the original image unchanged.

[0034] This invention constructs a contour-guided distance field by projecting the contour detected by the camera onto the viewpoint of the main camera. It then uses a rational function kernel to convert the distance field into spatially selective enhancement weights. This ensures the edge enhancement effect while maintaining the main structure and background information of the main camera image. It avoids the image blurring or artifact problems caused by traditional pixel-level fusion, and improves the accuracy and stability of edge detection.

[0035] In one embodiment of the present invention, the contour extraction module specifically includes: After Gaussian filtering to denoise the enhanced main position camera image, the Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel; The enhanced main position camera image is evenly divided into five horizontal strip blocks along the height direction; For each horizontal strip block, calculate the mean and standard deviation of the gradient magnitude of the pixels within the block. Then, calculate the maximum mean gradient magnitude across all horizontal strip blocks as the global maximum gradient level. Based on the mean gradient magnitude, standard deviation of the gradient magnitude of each horizontal strip block, and the global maximum gradient level, dynamically adjust the high threshold of each horizontal strip block. The calculation formula is as follows: in, Indicates the first High threshold of horizontal strip blocks For the first The average gradient magnitude of each horizontal strip block For the first The standard deviation of the gradient magnitude of each horizontal strip block. For high threshold coefficients, This represents the maximum value of the average gradient magnitude across all horizontal strip blocks. The low threshold of each horizontal strip block is obtained by multiplying the high threshold of each block by the low-high threshold ratio coefficient, where the low-high threshold ratio coefficient represents the ratio between the low threshold and the high threshold. Non-maximum suppression and hysteresis edge connection are performed on each horizontal strip block using corresponding high and low thresholds. The edge points detected in each block are merged to obtain a complete edge point set. Calculate the median of the horizontal coordinates of all edge points in the complete edge point set, and select edge points with horizontal coordinates less than the median as the left contour candidate point set and edge points with horizontal coordinates greater than the median as the right contour candidate point set. The least squares method is used to fit the linear equations of the vertical coordinates with respect to the horizontal coordinates for the candidate point sets of the left and right contours, respectively, and the slope and intercept of the left contour line and the right contour line are calculated. The horizontal coordinates of the left and right contours are calculated based on the fitted straight line equation at several sampling heights uniformly selected along the height direction, thus obtaining the contour pixel coordinates.

[0036] Understandably, the horizontal strip division takes into account the vertical distribution characteristics of the tower in the image: the upper part corresponds to the sky background, the middle part to the tower body, and the lower part to the ground background. The lighting conditions and gradient characteristics of different regions differ significantly. The brightness adaptive factor in the calculation formula... The function is as follows: for horizontal strip blocks with a large gradient mean, the factor is smaller, reducing the high threshold to avoid excessive edge suppression; for horizontal strip blocks with a small gradient mean, the factor is larger, maintaining a high threshold to suppress noise.

[0037] This invention divides the tower into blocks along the height direction and independently calculates dual thresholds with brightness adaptive factors for each horizontal strip block. This allows the edge detection threshold of each region to adapt to local lighting conditions and gradient features, effectively solving the detection contradiction of traditional global thresholding methods when there is a significant difference in brightness between the sky background above the tower and the ground background below. This improves the accuracy and completeness of contour edge detection.

[0038] In one embodiment of the present invention, the formula for calculating the low threshold is: in This is the ratio coefficient for low and high thresholds.

[0039] In one embodiment of the present invention, during non-maximum suppression, it is determined whether the gradient magnitude of each pixel is a local maximum along the gradient direction. The specific method is as follows: Based on the gradient direction, determine the adjacent pixels perpendicular to the gradient direction, compare the gradient magnitude of the current pixel with the gradient magnitude of the adjacent pixels, and retain the pixel if the current pixel has the largest gradient magnitude; otherwise, suppress it to zero.

[0040] Nonmaximum suppression can refine wide edges into edges that are only one pixel wide.

[0041] In one embodiment of the present invention, during hysteresis edge connection, pixels with gradient magnitudes greater than a high threshold are first marked as strong edge points. Then, starting from the strong edge points, pixels with gradient magnitudes greater than a low threshold are traced and connected along the gradient direction, marked as weak edge points, and connected to the strong edge points. Weak edge points not connected to strong edge points are suppressed. Hysteresis edge connection can suppress isolated noise points while ensuring edge continuity.

[0042] In one embodiment of the present invention, when fitting a contour line, the objective function is minimized as follows: By taking the derivative and setting it to zero, we obtain the normal system of equations, and solving it yields... and ,in, Vertical coordinates The horizontal coordinate is... The slope This is the intercept.

[0043] In one embodiment of the present invention, the step of converting the contour pixel coordinates into three-dimensional spatial coordinates and fitting the tower central axis direction vector using the intrinsic and extrinsic parameters of the main position camera specifically includes: The contour pixel coordinates are converted to normalized image plane coordinates using the focal length and principal point coordinates in the intrinsic parameter matrix of the primary camera, and the ray direction vector from the normalized image plane to the camera coordinate system is calculated: in, This represents the unit vector representation of the ray direction vector in the camera coordinate system. Represents the ray direction vector; The ray direction vector is transformed from the camera coordinate system to the global coordinate system using the extrinsic rotation matrix of the main position camera, resulting in the ray direction vector in the global coordinate system: in, This represents the ray direction vector in the global coordinate system. This represents the extrinsic rotation matrix of the main position camera. This represents the position coordinates of the optical center of the master camera in the global coordinate system. This represents the extrinsic translation vector of the main position camera; Based on the geometric characteristics of the tower, a cylindrical surface equation is established. An equation is established that starts from the optical center of the main position camera and intersects the cylindrical surface of the tower along the ray direction. The equation is then solved to obtain the three-dimensional spatial coordinates of the contour points. For each sampling height, the average of the three-dimensional coordinates of the left and right contour points in the horizontal direction is taken to obtain the position of the tower's central axis at the sampling height. The direction vector of the tower's central axis is obtained by fitting a straight line in three-dimensional space using the least squares method at all heights of the central axis location points; The angle between the direction vector of the tower's central axis and the vertical direction vector is calculated as the tilt angle, using the following formula: in, The tower's tilt angle, The direction vector of the tower's central axis. It is a vertical vector; The azimuth angle of the projection of the tower's central axis onto the horizontal plane is calculated as the tilt direction angle.

[0044] In one embodiment of the present invention, the step of outputting the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold specifically includes: A three-tiered early warning threshold system is established, wherein the first-tier early warning threshold is lower than the second-tier early warning threshold, and the second-tier early warning threshold is lower than the third-tier early warning threshold; The tower is considered to be in normal condition when the tilt angle is less than the first-level warning threshold. When the tilt angle is not less than the first-level warning threshold and less than the second-level warning threshold, a first-level warning recommendation is issued, which is to increase the monitoring frequency. When the tilt angle is not less than the Level II warning threshold and less than the Level III warning threshold, a Level II warning suggestion is issued, which suggests conducting an on-site inspection. A Level 3 warning is issued when the tilt angle is not less than the Level 3 warning threshold, and the warning recommendation is to immediately stop the machine for maintenance. Monitoring reports are generated and stored in a database based on tilt angle, tilt direction angle, warning level, timestamp, and measurement data and quality scores from each camera.

[0045] This invention converts the contour pixel coordinates into three-dimensional spatial coordinates using the intrinsic and extrinsic parameters of the main position camera. Based on the geometric characteristics of the tower cylinder, it establishes the equation for the intersection of the ray and the cylindrical surface to solve for the three-dimensional coordinates of the contour points. It also performs three-dimensional spatial straight line fitting on the central axis position points at multiple sampling heights to obtain the central axis direction vector. Furthermore, it sets up a three-level early warning threshold system to automatically output the early warning level and monitoring report based on the tilt angle, thus realizing a complete closed loop from data acquisition to early warning output.

[0046] This invention also provides a method for real-time monitoring of wind turbine tower verticality based on machine vision recognition, applied to the aforementioned real-time monitoring system for wind turbine tower verticality based on machine vision recognition, comprising the following steps: S1. Determine the locations of multiple cameras around the wind turbine tower, calculate the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each location, install cameras at each location according to the compensation pitch angle, obtain the intrinsic and extrinsic parameters of each camera, and establish a global coordinate system with the center of the bottom of the tower as the origin. S2. Control all cameras to synchronously acquire tower images, comprehensively evaluate the contrast, sharpness and brightness deviation of each camera image, obtain the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction, and determine the set of master position camera and slave position camera based on image quality score, initial tilt direction angle and tilt angle. S3. Calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than the preset threshold, the contour detected by the set of position cameras is projected onto the view of the main position camera to obtain contour guidance. The main position camera image is enhanced based on the contour guidance to obtain the enhanced main position camera image. S4. Divide the enhanced main position camera image into blocks along the height direction. For each block, adaptively calculate the edge detection threshold based on the gradient statistical features and the brightness difference between blocks. Use the corresponding threshold to perform edge detection on each block and merge them to obtain a complete edge point set. Fit the left and right contour lines using the least squares method and extract the contour pixel coordinates. S5. Convert the contour pixel coordinates into three-dimensional spatial coordinates using the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. Calculate the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle. Output the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

[0047] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition, characterized in that: include: The calibration module is used to determine the deployment positions of multiple cameras around the wind turbine tower. It calculates the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each deployment position, installs cameras at each deployment position according to the compensation pitch angle, obtains the intrinsic and extrinsic parameters of each camera, and establishes a global coordinate system with the center of the bottom of the tower as the origin. The camera selection module is used to control all cameras to synchronously acquire images of the tower. It scores the quality of each camera image based on contrast, sharpness, and brightness deviation. It obtains the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction. Based on the image quality score, the initial tilt direction angle, and the tilt angle, it determines the set of main position cameras and slave position cameras. The image processing module is used to calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than a preset threshold, the contour detected from the set of position cameras is projected onto the main position camera view to obtain contour guidance. Based on the contour guidance, the main position camera image is enhanced to obtain the enhanced main position camera image. The contour extraction module is used to divide the enhanced main position camera image into blocks along the height direction. For each block, the edge detection threshold is adaptively calculated based on the gradient statistical features and the brightness difference between blocks. The corresponding threshold is used to perform edge detection on each block and merge them to obtain a complete edge point set. The left and right contour lines are fitted by the least squares method and the contour pixel coordinates are extracted. The measurement output module is used to convert the contour pixel coordinates into three-dimensional spatial coordinates through the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. It calculates the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle, and outputs the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

2. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 1, characterized in that: The deployment calibration module specifically includes: Determine the locations of multiple cameras around the wind turbine tower. The cameras are evenly distributed around the tower to form a ring array. The horizontal distance from each camera location to the central axis of the tower is set according to the tower height. Measure the slope angle and aspect angle of each camera placement position relative to the horizontal reference plane, where the aspect angle is zero degrees in the due north direction and positive in the clockwise direction; The ideal pitch angle is determined based on the height of the wind turbine tower and the horizontal distance of each camera from the central axis of the tower: ； in, Indicates the ideal pitch angle. Indicates the height of the wind turbine tower. This indicates the horizontal distance from the camera's location to the central axis of the tower. The ideal pitch angle is compensated based on the terrain slope angle, aspect angle, and camera azimuth angle at each camera deployment location to obtain the compensated pitch angle: ； in For the first Camera No. 1 compensates for pitch angle. The slope angle of the terrain. The slope angle, For the first The azimuth angle of camera number 1; Install the cameras at their respective locations according to the compensated pitch angle; Using the focal length, principal point coordinates, and distortion coefficients of each camera as intrinsic parameters, and obtaining the rotation matrix and translation vector of each camera relative to the center coordinate system of the tower as extrinsic parameters, a global coordinate system with the center of the bottom of the tower as the origin is established, where the vertical direction upward is the Z-axis, the horizontal direction due north is the X-axis, and the principal point coordinates are the positions of the intersection of the camera's optical axis and the image plane in the pixel coordinate system.

3. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 1, characterized in that: The process of obtaining the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction specifically includes: Canny edge detection is used to obtain edge point sets for each camera image. Hough line transform is used to detect the left and right contour lines of the tower, thus obtaining two-dimensional contour points in each camera image. By using the external parameter rotation matrix and translation vector of each camera, the two-dimensional contour points are transformed from the camera coordinate system to the global coordinate system, and the three-dimensional contour line of the tower in the global coordinate system is obtained, where the three-dimensional contour line includes the three-dimensional contour points. The average horizontal coordinates of the left and right three-dimensional contour points obtained by each camera at each height are calculated to obtain the position point of the tower's central axis at the corresponding height. The central axis position point is then fitted with a three-dimensional spatial straight line to obtain the central axis direction vector corresponding to each camera. The average tilt direction angle and tilt angle of the tower are obtained by averaging the direction vectors of the central axis corresponding to all cameras. The tilt direction angle is the clockwise angle between the projection of the central axis on the horizontal plane and the due north direction, and the tilt angle is the angle between the central axis and the vertical direction.

4. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 3, characterized in that: The determination of the master and slave camera sets based on image quality scores, preliminary tilt direction angles, and tilt angles specifically includes: Determine if the initial tilt angle exceeds the initial threshold. If it does, the tower is confirmed to be tilted, and the master-slave camera selection process begins. The tilt direction sector is determined based on the initial tilt direction angle, and the camera with the highest image quality score within the tilt direction sector is selected as the main position camera, thus obtaining the main position camera number; wherein, the tilt direction sector includes cameras whose absolute value of the difference between the azimuth angle and the initial tilt direction angle is less than a threshold. In addition to the main camera, cameras with image quality scores greater than a quality threshold are selected as the slave camera set, resulting in a slave camera number set. The slave cameras are used to correct the edge detection results of the main camera image.

5. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 1, characterized in that: The formula for calculating the edge sharpness is: ； in For edge sharpness indicators, The set of edge points in the main position camera image. The number of edge points. Main position camera image in pixel coordinates The gradient magnitude at that point.

6. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 5, characterized in that: The image processing module specifically includes: Contour detection is performed on the images of each slave camera in the slave camera set to obtain two-dimensional contour points. The two-dimensional contour points are then transformed to the global coordinate system through the extrinsic parameters of the slave camera to obtain three-dimensional contour points. Finally, the three-dimensional contour points are projected onto the image plane of the master camera through the extrinsic parameters of the master camera to obtain the projection position of the slave camera contours under the view of the master camera. The Euclidean distance from each pixel in the main position camera image to the nearest point in all the projection contours from the position camera is calculated as the contour guiding distance, and a contour guiding distance field covering the entire main position camera image is constructed. The contour-guided distance field is converted into edge-guided enhancement weights using a rational function kernel form. The calculation formula is as follows: ； in pixel coordinates Edge guidance enhances weight, pixel coordinates The outline guides the distance. For scale parameters; An edge enhancement signal is constructed based on the Laplacian operator and gradient magnitude of the image from the main camera position. The calculation formula is as follows: in pixel coordinates Edge enhancement signal at the location, Main position camera image pixel coordinates The Laplace operator at the location, For symbolic functions, Indicates the gradient magnitude; The enhanced main camera image is obtained by adding an enhancement intensity coefficient to the grayscale value of the original image from the main camera, and then multiplying this by the edge-guided enhancement weight and the edge enhancement signal. The calculation formula is as follows: in, Represents pixel coordinates Enhanced main position camera image, Main position camera raw image pixel coordinates grayscale value at that location To enhance the strength coefficient.

7. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 1, characterized in that: The contour extraction module specifically includes: After Gaussian filtering to denoise the enhanced main position camera image, the Sobel operator is used to calculate the gradient magnitude and gradient direction of each pixel; The enhanced main position camera image is evenly divided into five horizontal strip blocks along the height direction; For each horizontal strip block, calculate the mean gradient magnitude and standard deviation of the gradient magnitude of the pixels within the block. Then, calculate the maximum mean gradient magnitude across all horizontal strip blocks as the global maximum gradient level. Based on the mean gradient magnitude, standard deviation of the gradient magnitude of each horizontal strip block, and the global maximum gradient level, dynamically adjust the high threshold of each horizontal strip block. The calculation formula is as follows: in, Indicates the first High threshold of horizontal strip blocks For the first The average gradient magnitude of each horizontal strip block For the first The standard deviation of the gradient magnitude of each horizontal strip block. For high threshold coefficients, This represents the maximum value of the average gradient magnitude across all horizontal strip blocks. The low threshold of each horizontal strip block is obtained by multiplying the high threshold of each block by the low-high threshold ratio coefficient, where the low-high threshold ratio coefficient represents the ratio between the low threshold and the high threshold. Non-maximum suppression and hysteresis edge connection are performed on each horizontal strip block using corresponding high and low thresholds. The edge points detected in each block are merged to obtain a complete edge point set. Calculate the median of the horizontal coordinates of all edge points in the complete edge point set, and select edge points with horizontal coordinates less than the median as the left contour candidate point set and edge points with horizontal coordinates greater than the median as the right contour candidate point set. The least squares method is used to fit the linear equations of the vertical coordinates with respect to the horizontal coordinates for the candidate point sets of the left and right contours, respectively, and the slope and intercept of the left contour line and the right contour line are calculated. The horizontal coordinates of the left and right contours are calculated based on the fitted straight line equation at several sampling heights uniformly selected along the height direction, thus obtaining the contour pixel coordinates.

8. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 1, characterized in that: The process of converting contour pixel coordinates into three-dimensional spatial coordinates and fitting them to obtain the direction vector of the tower's central axis using the intrinsic and extrinsic parameters of the main position camera specifically includes: The contour pixel coordinates are converted into normalized image plane coordinates using the focal length and principal point coordinates in the intrinsic parameter matrix of the main position camera, and the ray direction vector from the normalized image plane to the camera coordinate system is calculated. The ray direction vector is transformed from the camera coordinate system to the global coordinate system by using the extrinsic rotation matrix of the main position camera, thus obtaining the ray direction vector in the global coordinate system. Based on the geometric characteristics of the tower, a cylindrical surface equation is established. An equation is established that starts from the optical center of the main position camera and intersects the cylindrical surface of the tower along the ray direction. The equation is then solved to obtain the three-dimensional spatial coordinates of the contour points. The position of the tower's central axis at each sampling height is obtained by averaging the three-dimensional coordinates of the left and right contour points in the horizontal direction at each sampling height. The direction vector of the tower's central axis is obtained by fitting a straight line in three-dimensional space using the least squares method at all heights of the central axis location points; The angle between the direction vector of the tower's central axis and the vertical direction vector is calculated as the tilt angle, using the following formula: in, The tower's tilt angle, The direction vector of the tower's central axis. It is a vertical vector; The azimuth angle of the projection of the tower's central axis onto the horizontal plane is calculated as the tilt direction angle.

9. The real-time monitoring system for the verticality of wind turbine towers based on machine vision recognition as described in claim 8, characterized in that: The process of outputting warning levels and monitoring reports based on the relationship between tilt angle and warning threshold specifically includes: A three-tiered early warning threshold system is established, wherein the first-tier early warning threshold is lower than the second-tier early warning threshold, and the second-tier early warning threshold is lower than the third-tier early warning threshold; The tower is considered to be in normal condition when the tilt angle is less than the first-level warning threshold. When the tilt angle is not less than the Level 1 warning threshold and less than the Level 2 warning threshold, a Level 1 warning recommendation is issued, which is to increase the monitoring frequency. When the tilt angle is not less than the Level II warning threshold and less than the Level III warning threshold, a Level II warning suggestion is issued, which suggests conducting an on-site inspection. A Level 3 warning is issued when the tilt angle is not less than the Level 3 warning threshold, and the warning recommendation is to immediately stop the machine for maintenance. Monitoring reports are generated and stored in a database based on tilt angle, tilt direction angle, warning level, timestamp, and measurement data and quality scores from each camera.

10. A method for real-time monitoring of the verticality of wind turbine towers based on machine vision recognition, characterized in that: The system for real-time monitoring of wind turbine tower verticality based on machine vision recognition as described in any one of claims 1-9 includes the following steps: S1. Determine the locations of multiple cameras around the wind turbine tower, calculate the compensation pitch angle based on the terrain slope parameters and camera orientation relationship of each location, install cameras at each location according to the compensation pitch angle, obtain the intrinsic and extrinsic parameters of each camera, and establish a global coordinate system with the center of the bottom of the tower as the origin. S2. Control all cameras to synchronously acquire tower images, comprehensively evaluate the contrast, sharpness and brightness deviation of each camera image, obtain the initial tilt direction angle and tilt angle of the tower through edge detection and 3D reconstruction, and determine the set of master position camera and slave position camera based on image quality score, initial tilt direction angle and tilt angle. S3. Calculate the edge sharpness index of the main position camera image. When the edge sharpness index is lower than the preset threshold, the contour detected by the set of position cameras is projected onto the view of the main position camera to obtain contour guidance. The main position camera image is enhanced based on the contour guidance to obtain the enhanced main position camera image. S4. Divide the enhanced main position camera image into blocks along the height direction. For each block, adaptively calculate the edge detection threshold based on the gradient statistical features and the brightness difference between blocks. Use the corresponding threshold to perform edge detection on each block and merge them to obtain a complete edge point set. Fit the left and right contour lines using the least squares method and extract the contour pixel coordinates. S5. Convert the contour pixel coordinates into three-dimensional spatial coordinates using the intrinsic and extrinsic parameters of the main position camera and fit them to obtain the direction vector of the tower's central axis. Calculate the angle between the direction vector of the tower's central axis and the vertical direction vector as the tilt angle. Output the warning level and monitoring report based on the relationship between the tilt angle and the warning threshold.

Images