Wind turbine generator blade stress-strain monitoring method and device based on machine vision

Abstract

The invention provides a wind turbine generator blade stress-strain monitoring method and device based on machine vision, and the method comprises the steps: obtaining a wind turbine generator blade image of a target region through a mobile camera in a vehicle-mounted safety monitoring system; extracting contour features of the wind turbine generator blade in the wind turbine generator blade image; according to the contour features of the wind turbine generator blade, performing positioning compensation on the wind turbine generator blade in the wind turbine generator blade image in a visual guidance mode to obtain a target wind turbine generator blade image; and monitoring the wind turbine generator blades in the target wind turbine generator blade image according to a random sampling consistency algorithm to obtain a wind turbine generator blade stress-strain monitoring result. According to the method, positioning compensation is carried out on the wind turbine generator blades in a visual guidance mode, accurate positioning of the wind turbine generator blades is achieved, furthermore, wind turbine generator defects such as corrosion and fracture are recognized through the random sampling consistency technology, and therefore the wind turbine generator blade stress-strain monitoring result with higher accuracy is obtained, and the accuracy of the wind turbine generator blade stress-strain monitoring result is improved. The high-precision monitoring of the stress-strain condition of the wind turbine generator blade is realized, and more reliable data is provided for the health state evaluation of the wind turbine generator.

Description

Technical Field

[0001] The present invention relates to the technical field of wind turbine blade monitoring, and in particular to a method and device for monitoring wind turbine blade stress and strain based on machine vision. Background Art

[0002] Wind power generation is one of the clean energy sources with great prospects for large-scale development and has important social and economic benefits in today's society.

[0003] The blade is a key component of a wind turbine, and whether it can work stably is directly related to the working condition of the wind turbine. The working environment of the wind turbine directly determines that the blade needs to be exposed to the harsh outdoor environment of the wind farm for a long time, and the blade is the first component in the whole machine to be subjected to alternating loads.

[0004] In recent years, due to the aging and damage of gel coat, transverse cracks and longitudinal cracks in blades have frequently appeared, and the length of the cracks is generally more than one meter. These cracks can cause abnormal noise and whistling of the blades. More seriously, they can cause uneven load on the blades, causing additional axial loads on the blade main shaft and bearings, and accelerating the wear of such mechanical parts. How to conduct real-time, effective and high-precision monitoring of the stress and strain of dynamic wind turbine blades is of vital importance.

[0005] The existing technology mainly obtains stress and strain information by analyzing the image of the blade surface. This method can provide certain help for the stress and strain conditions visible on the surface, but it has the following limitations (1)-(5).

[0006] (1) Limited by visual information: It is impossible to obtain stress and strain information inside or hidden parts of the blade.

[0007] (2) Limited sensitivity: It is easily affected by factors such as image quality, lighting conditions, and camera resolution, which may limit the sensitivity and accuracy of the algorithm, especially in harsh environmental conditions such as strong wind, rain and snow.

[0008] (3) Modeling and algorithm complexity: It is necessary to model the blades and develop image processing algorithms, which involves complex image processing and computer vision technologies and requires a lot of time and resources for model training, algorithm optimization, and parameter adjustment.

[0009] (4) Challenges of real-time and scalability: Real-time performance is one of the important requirements for wind turbine blade monitoring. However, existing methods may face challenges in real-time performance, especially in large-scale wind farms where multiple blades need to be monitored simultaneously. In addition, the scalability of the algorithm is also an issue, and it is necessary to consider how to effectively expand to multiple wind turbines and a larger range of monitoring.

[0010] (5) Wind turbine blade collection problem: The blades of wind turbines are usually at a very high position. Conventional stress testing requires workers to climb to a very high position to collect relevant data and make judgments with the naked eye.

[0011] It is also worth mentioning that the blade length of large offshore and onshore wind turbines can reach 200 meters. Among the various non-destructive testing technologies for composite blades (acoustic emission, infrared, fiber grating, ultrasonic, guided wave, thermal imaging, terahertz imaging, mechanical, electromagnetic and vibration non-destructive testing technologies), there is no non-contact, remote monitoring and detection method for defects such as corrosion and cracks on the surface of wind turbine blades without stopping the wind turbine, without stopping production, and without normal power generation and high-speed rotation of the blades.

[0012] Therefore, how to solve the problems that the monitoring accuracy of existing wind turbine blade stress and strain monitoring methods is easily affected by the environment, and the real-time and scalability are not high, and the time-consuming is an important issue that needs to be urgently solved in the field of wind turbine blade monitoring. Summary of the invention

[0013] The present invention provides a method and device for monitoring stress and strain of wind turbine blades based on machine vision, so as to overcome the defects of existing wind turbine blade stress and strain monitoring methods, such as the monitoring accuracy is easily affected by the environment, the real-time performance and scalability are not high, and the time consumption is long, so as to achieve high-precision monitoring of the stress and strain of wind turbine blades and provide more reliable data for the health status assessment of wind turbines.

[0014] On the one hand, the present invention provides a method for monitoring stress and strain of wind turbine blades based on machine vision, comprising: using a mobile camera in a vehicle-mounted safety monitoring system to obtain an image of wind turbine blades in a target area; extracting contour features of the wind turbine blades in the wind turbine blade image; based on the contour features of the wind turbine blades, positioning and compensating the wind turbine blades in the wind turbine blade image in a visually guided manner to obtain a target wind turbine blade image; and monitoring the wind turbine blades in the target wind turbine blade image according to a random sampling consistency algorithm to obtain a wind turbine blade stress and strain monitoring result.

[0015] Furthermore, the extracting of contour features of the wind turbine blades in the wind turbine blade image includes: performing edge detection on the wind turbine blade image to obtain blade edges in the wind turbine blade image; based on the blade edges, using a color filter to select a color range of the wind turbine blades, separating the wind turbine blades from the image background, and obtaining initial contour features of the wind turbine blades; performing morphological operations on the initial contour features to obtain contour features of the wind turbine blades.

[0016] Furthermore, the positioning compensation of the wind turbine blades in the wind turbine blade image is performed in a visually guided manner according to the contour features of the wind turbine blades, including: analyzing the combined peak signals from two wind turbine blades and the top wind turbine blade according to the contour features of the wind turbine blades to identify the positions of all wind turbine blades; wherein the combined peak signal characterizes the intensity change of the blade edge in the wind turbine blade image; determining the vertical axis representation of one group of wind turbine blades according to the positions of all wind turbine blades; calculating the attitude of the wind turbine by analyzing the vertical axis representation of the wind turbine blades, the attitude including the roll angle, the pitch angle and the yaw angle; and adjusting the wind turbine blade image according to the attitude of the wind turbine to compensate for the visual deviation caused by the change in the attitude of the wind turbine, so as to obtain the target wind turbine blade image.

[0017] Furthermore, the monitoring of the wind turbine blades in the target wind turbine blade image according to the random sampling consistency algorithm includes: fitting a wind turbine blade feature model according to key blade features in the target wind turbine blade image; monitoring the wind turbine blades in the target wind turbine blade image based on the wind turbine blade feature model to obtain stress and strain monitoring results of the wind turbine blades; wherein the wind turbine blade stress and strain monitoring results include monitoring positions and their corresponding stress values ​​and stress values.

[0018] Further, the vehicle-mounted safety monitoring system comprises a first layer, a second layer and a third layer; wherein the first layer is provided with a magnetic sensor for monitoring defects and corrosion of the bottom plate; the second layer is provided with a control unit for coordinating the operation of the entire system, including data collection, processing and transmission; the third layer is provided with a mobile camera for capturing images of wind turbine blades of wind turbine blades in a target area; The vehicle-mounted safety monitoring system moves by driving wheels with motors to approach different positions of the wind turbine generator set.

[0019] Furthermore, the wheel includes a driving wheel, and the driving wheel is equipped with a pulse sensor connected to the control unit; wherein, when the driving wheel rotates one circle, the pulse sensor generates a pulse signal accordingly; accordingly, the wind turbine blade stress and strain monitoring method also includes: obtaining the number of pulse signals generated by the pulse sensor; according to the number of pulse signals, calculating the distance between the monitored corrosion position and / or fracture position and the starting point, so as to locate the corrosion position and / or fracture position.

[0020] Furthermore, it also includes: evaluating the structural integrity of the wind turbine blades according to the stress and strain monitoring results of the wind turbine blades; and / or predicting the remaining service life of the wind turbine blades according to the stress and strain monitoring results of the wind turbine blades; and / or determining maintenance and overhaul decisions according to the stress and strain monitoring results of the wind turbine blades; and / or performing a safety assessment on the wind turbine blades according to the stress and strain monitoring results of the wind turbine blades.

[0021] In a second aspect, the present invention also provides a wind turbine blade stress and strain monitoring device based on machine vision, comprising: a wind turbine blade image acquisition module, used to use a mobile camera in a vehicle-mounted safety monitoring system to acquire an image of a wind turbine blade in a target area; a wind turbine blade contour feature extraction module, used to extract contour features of the wind turbine blades in the wind turbine blade image; a wind turbine blade positioning compensation module, used to perform positioning compensation on the wind turbine blades in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blades, to obtain a target wind turbine blade image; a wind turbine blade stress and strain monitoring module, used to monitor the wind turbine blades in the target wind turbine blade image according to a random sampling consistency algorithm, to obtain a wind turbine blade stress and strain monitoring result.

[0022] In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for monitoring wind turbine blade stress and strain based on machine vision as described above is implemented.

[0023] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for monitoring wind turbine blade stress and strain based on machine vision.

[0024] The wind turbine blade stress and strain monitoring method based on machine vision provided by the present invention obtains the wind turbine blade image of the target area by using the mobile camera in the vehicle-mounted safety monitoring system, and extracts the contour features of the wind turbine blade in the wind turbine blade image, and then performs positioning compensation on the wind turbine blade in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blade, and obtains the target wind turbine blade image, thereby, according to the random sampling consistency algorithm, the wind turbine blade in the target wind turbine blade image is monitored to obtain the wind turbine blade stress and strain monitoring result. The method performs positioning compensation on the wind turbine blade in a visually guided manner, realizes the precise positioning of the wind turbine blade, and further, by using the random sampling consistency technology to identify wind turbine defects such as corrosion and fracture, so as to obtain a wind turbine blade stress and strain monitoring result with higher accuracy, realizes the high-precision monitoring of the stress and strain of the wind turbine blade, and provides more reliable data for the health status assessment of the wind turbine. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a flow chart of a method for monitoring stress and strain of wind turbine blades based on machine vision provided in an embodiment of the present invention.

[0027] Figure 2 It is a structural schematic diagram of a wind turbine blade stress and strain monitoring device based on machine vision provided in an embodiment of the present invention.

[0028] Figure 3 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the existing technology mainly obtains stress and strain information by analyzing the image of the blade surface. This method can provide certain help for the stress and strain conditions visible on the surface, but it has the following limitations (1)-(5).

[0031] (1) Limited by visual information: It is impossible to obtain stress and strain information inside or hidden parts of the blade.

[0032] (2) Limited sensitivity: It is easily affected by factors such as image quality, lighting conditions, and camera resolution, which may limit the sensitivity and accuracy of the algorithm, especially in harsh environmental conditions such as strong wind, rain and snow.

[0033] (3) Modeling and algorithm complexity: It is necessary to model the blades and develop image processing algorithms, which involves complex image processing and computer vision technologies and requires a lot of time and resources for model training, algorithm optimization, and parameter adjustment.

[0034] (4) Challenges of real-time and scalability: Real-time performance is one of the important requirements for wind turbine blade monitoring. However, existing methods may face challenges in real-time performance, especially in large-scale wind farms where multiple blades need to be monitored simultaneously. In addition, the scalability of the algorithm is also an issue, and it is necessary to consider how to effectively expand to multiple wind turbines and a larger range of monitoring.

[0035] (5) Wind turbine blade collection problem: The blades of wind turbines are usually at a very high position. Conventional stress testing requires workers to climb to a very high position to collect relevant data and make judgments with the naked eye.

[0036] In view of this, the present invention proposes a method for monitoring the stress and strain of wind turbine blades based on machine vision. Specifically, Figure 1 A schematic flow chart of a method for monitoring stress and strain of wind turbine blades based on machine vision provided in an embodiment of the present invention is shown.

[0037] like Figure 1 As shown, the method includes steps S110-S140, and steps S110-S140 and related steps will be described in detail below.

[0038] S110, using a mobile camera in the vehicle-mounted safety monitoring system to obtain an image of wind turbine blades in a target area.

[0039] It is easy to understand that an appropriate on-board safety monitoring system is installed on the vehicle, including but not limited to a mobile camera. Then, the angle, focal length and other parameters of the mobile camera are configured to ensure that the key parts of the wind turbine blades can be captured, and the image acquisition frame rate is set to ensure that sufficient image data can be obtained to analyze the status of the blades.

[0040] When collecting images of wind turbine blades in the target area, the vehicle or the on-board safety monitoring system installed on the vehicle will plan the driving route according to the current actual situation to ensure that the mobile camera can approach the wind turbine and cover all visible parts of the wind turbine blades. Therefore, the mobile camera moves with the vehicle or the on-board safety monitoring system installed on the vehicle, and starts the image acquisition program during the movement to obtain images of wind turbine blades in the target area in real time.

[0041] The target area refers to the area where the blades of the wind turbine are located, which can be determined according to actual needs and is not specifically limited here.

[0042] The wind turbine blade image refers to an image that at least includes the wind turbine blade.

[0043] It is worth mentioning that this step uses a mobile camera to collect images of wind turbine blades in the target area, which has many advantages. The following will further describe the three aspects of flexibility, full coverage, efficiency and cost-effectiveness.

[0044] Regarding flexibility, (1) mobility: the vehicle can be moved between different wind turbines without having to re-set up the monitoring equipment; (2) adaptability: it can more easily adapt to different terrain and environmental conditions, such as hills, valleys and other areas with complex terrain.

[0045] Regarding all-round coverage, (1) multi-angle observation: the vehicle can approach the wind turbine from multiple angles to obtain different side images of the wind turbine blades; (2) no blind spot monitoring: the blind spot can be reduced by changing the position and angle of the vehicle, thereby improving the comprehensiveness of image acquisition.

[0046] Regarding efficiency, (1) rapid deployment: it can quickly arrive at the designated location to start performing image acquisition tasks, reducing preparation time; (2) real-time feedback: it can obtain images of wind turbine blades in real time and detect potential problems in a timely manner.

[0047] Regarding cost-effectiveness, (1) Reduction of fixed facilities: There is no need to install fixed monitoring equipment for each wind turbine, which reduces the overall cost; (2) Convenient maintenance: The on-board safety monitoring system is easy to maintain and upgrade, reducing maintenance costs.

[0048] It is also worth mentioning that this embodiment uses a mobile camera for monitoring, and the vehicle-mounted safety monitoring system can follow the movement of the wind turbine and obtain image data of the blades in real time. This deployment method makes all objects in the scene move, so it can better solve the problem of how to accurately locate the blades, blade tips and tower centerline of the wind turbine.

[0049] S120: extracting contour features of the wind turbine blades in the wind turbine blade image.

[0050] It is easy to understand that based on obtaining the wind turbine blade image in the target area using the mobile camera in the vehicle-mounted safety monitoring system in step S110, the contour features of the wind turbine blade in the wind turbine blade image are further extracted.

[0051] Specifically, the contour feature extraction of wind turbine blades includes but is not limited to image preprocessing, edge detection, contour extraction and contour feature extraction.

[0052] First, about image preprocessing.

[0053] After obtaining the wind turbine blade image, it can be grayed out to convert the color image into a grayscale image to simplify subsequent processing. At the same time, filters (such as Gaussian filtering, median filtering, etc.) can be used to remove noise from the wind turbine blade image. In addition, the contrast of the wind turbine blade image can be enhanced by techniques such as histogram equalization to highlight the outline of the wind turbine blade.

[0054] Next, about edge detection.

[0055] An edge detection algorithm, such as Canny edge detection, Sobel operator, etc., may be used to detect the edge of the wind turbine blade in the wind turbine blade image, and a threshold segmentation technique may be used to further extract the edge.

[0056] Next, let’s talk about contour extraction.

[0057] You can use contour extraction functions in libraries such as OpenCV to obtain the contours of specific targets (wind turbine blades) in the image. This step usually includes finding connected areas and extracting boundary pixels.

[0058] Finally, about contour feature extraction.

[0059] After extracting the contour of the wind turbine blade, it is relatively easy to further obtain the corresponding contour features.

[0060] Among them, the contour features of the wind turbine blades include but are not limited to contour area, contour perimeter, contour center, contour shape description, contour strength, and contour sharpness.

[0061] It is worth mentioning that by accurately extracting and analyzing these contour features, the accuracy and reliability of wind turbine blade monitoring can be improved, thereby helping operation and maintenance personnel to promptly discover and solve problems and ensure the safe operation of wind turbines.

[0062] S130 , performing positioning compensation on the wind turbine blades in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blades to obtain a target wind turbine blade image.

[0063] It is easy to understand that, based on the extraction of the contour features of the wind turbine blades in the wind turbine blade image in step S120, step S130 will further perform positioning compensation on the wind turbine blades in the wind turbine blade image by visual guidance to improve the accuracy of subsequent wind turbine blade stress and strain monitoring.

[0064] Specifically, based on the contour features of the wind turbine blades, the current position of the wind turbine blades in the wind turbine blade image can be determined, and then the offset of the current position of the wind turbine blades compared to the expected position, that is, the position deviation of the wind turbine blades, can be calculated. Therefore, coordinate transformation (translation, rotation, scaling, etc.) is applied to correct the position deviation of the wind turbine blades.

[0065] Specifically, the position of the mobile camera is adjusted according to the image of the wind turbine blades and the position deviation of the wind turbine blades to achieve the effect of positioning compensation for the wind turbine blades.

[0066] After completing the positioning compensation of the wind turbine blades in the wind turbine blade image, the image of the wind turbine blades is recaptured or cropped as the target wind turbine blade image.

[0067] S140, monitoring the wind turbine blades in the target wind turbine blade image according to a random sampling consistency algorithm to obtain wind turbine blade stress and strain monitoring results.

[0068] It can be understood that in step S130, the wind turbine blades in the wind turbine blade image are positioned and compensated by visual guidance to obtain the target wind turbine blade image. Further, step S140 will use random sampling consistency technology to perform defect monitoring on the wind turbine blades in the target wind turbine blade image to obtain the wind turbine blade stress and strain monitoring results.

[0069] Among them, the Random Sample Consensus (RANSAC) algorithm is an iterative method for estimating the parameters of a mathematical model from a set of observations while identifying and excluding outliers (also known as "outliers"). This algorithm is particularly suitable for situations when the data set contains outliers.

[0070] The basic principle of RANSAC technology is to fit the model by randomly selecting a subset of data samples, then check which data points are consistent with the model (i.e., inlier points), and refit the model based on these inlier points. This process is repeated many times until the best model estimate is found.

[0071] An important property of RANSAC is that it does not rely on all data points being valid. Even if most of the data points are outliers, RANSAC can find the correct model as long as not all of the randomly selected samples in each iteration are outliers.

[0072] In this embodiment, in the monitoring problem of wind turbine blades, RANSAC technology can be used to detect feature points from the target wind turbine blade image and estimate the change or defect of the wind turbine blade shape. By matching the feature points in the wind turbine blade images taken at different time points, it is possible to detect which points are inner group points (representing normal changes in the wind turbine blades) and which points are outer group points (possibly representing defects or damage on the wind turbine blades), thereby further inferring the stress and strain conditions corresponding to these feature points, and obtaining the wind turbine blade stress and strain monitoring results.

[0073] Among them, the stress and strain monitoring results of wind turbine blades include but are not limited to monitoring positions, strain values, stress values, time series data and environmental conditions.

[0074] The monitoring location refers to the specific location of the monitoring point, which usually includes key locations such as the root, middle and tip of the wind turbine blade.

[0075] The strain value includes linear strain and shear strain. Linear strain is the ratio of the length change of the wind turbine blade to the original length when it is subjected to external force. Shear strain is the relative sliding between adjacent layers when the blade is subjected to shear force.

[0076] The stress value includes normal stress and shear stress. Normal stress is the stress perpendicular to the blade cross section, and shear stress is the stress parallel to the blade cross section.

[0077] Time series data refers to the data of strain and stress changes over time obtained by real-time monitoring.

[0078] Environmental conditions include, but are not limited to, environmental parameters such as wind speed, wind direction, and temperature during the monitoring period, which can affect the stress-strain state of the wind turbine blades.

[0079] Regarding the stress and strain monitoring of wind turbine blades, it should be noted that due to the complex long-term working conditions, wind turbine blades often experience a lot of internal and external damage, including material fatigue failure, wear, corrosion, erosion and cracks caused by system degradation or deterioration.

[0080] Environmental conditions (rain / hail / ice, lightning, wave impact and gusty winds) can cause damage to the blades. For example, rain can cause erosion, which reduces the annual energy production and eventually damages the blades themselves. Lightning can also cause the blades to split from the tip inwards.

[0081] Blade failures account for a large proportion of all wind turbine failures. According to statistics, the percentage of damage location and type, most of the damage is located on the coating surface and adhesive, while the main blade structure damage modes are transverse cracks, peeling, leading edge adhesive bond failure, delamination in the load-bearing laminate, interlayer / core debonding and trailing edge adhesive bond failure.

[0082] Among all damage types, leading edge erosion is a highly complex problem that involves the removal of material due to continuous exposure to rain, ice, insects and dust, which reduces wind turbine performance. Therefore, regular inspection, cleaning and repair of blades are required.

[0083] It is worth mentioning that wind turbines are often located in remote areas, such as mountainous areas and deep sea areas that are not conducive to long-term inspections by personnel. In the intelligent operation and maintenance system of wind turbines based on the SCADA system, video cameras and other equipment are often used to monitor the safety of wind turbines. Traditional means are to form an Internet of Things, and personnel are concentrated in the control room to monitor the camera equipment 24 hours a day. The fixed and vehicle-mounted machine vision monitoring devices provided in this embodiment are used to automatically extract the surface features of the high-speed rotating wind turbine blades using a random sampling consistency algorithm, and diagnose the stress and strain state of the wind turbine blades. This can help operation and maintenance personnel understand the working status of the blades, discover potential problems in a timely manner, and formulate reasonable maintenance plans, thereby improving the safety and reliability of wind turbines.

[0084] It is also worth mentioning that, compared with the prior art, this embodiment applies machine vision technology, which has obvious advantages, especially highlighting the potential role of machine vision and imaging technology in monitoring the safety of large wind turbines and saving wind energy costs.

[0085] In this embodiment, by using the mobile camera in the vehicle-mounted safety monitoring system, the wind turbine blade image in the target area is obtained, and the contour features of the wind turbine blade in the wind turbine blade image are extracted, and then the wind turbine blade in the wind turbine blade image is positioned and compensated by visual guidance according to the contour features of the wind turbine blade, and the target wind turbine blade image is obtained, so that the wind turbine blade in the target wind turbine blade image is monitored according to the random sampling consistency algorithm to obtain the wind turbine blade stress and strain monitoring results. The method performs positioning compensation for the wind turbine blade in a visually guided manner, thereby achieving accurate positioning of the wind turbine blade. Furthermore, by using the random sampling consistency technology to identify wind turbine defects such as corrosion and fracture, a wind turbine blade stress and strain monitoring result with higher accuracy is obtained, thereby achieving high-precision monitoring of the stress and strain of the wind turbine blade, and providing more reliable data for the health status assessment of the wind turbine.

[0086] On the basis of the above-mentioned embodiment, the process of extracting the contour features of the wind turbine blades in the wind turbine blade image will be described in detail below.

[0087] Extracting contour features of wind turbine blades in a wind turbine blade image includes: performing edge detection on the wind turbine blade image to obtain blade edges in the wind turbine blade image; based on the blade edges, using a color filter to select a color range of the wind turbine blades, separating the wind turbine blades from the image background, and obtaining initial contour features of the wind turbine blades; performing morphological operations on the initial contour features to obtain contour features of the wind turbine blades.

[0088] It can be understood that this embodiment uses advanced image processing techniques, such as edge detection, morphological operations, etc., on the wind turbine blade images to extract the shape features of the wind turbine blades in the image and distinguish other objects, and distinguish the wind turbine blades from the background through color filtering or texture analysis, ensuring that only the information of the wind turbine blades is used for monitoring and analysis.

[0089] Specifically, for the acquired wind turbine blade image, an image preprocessing operation is first performed on it, and the image preprocessing operation includes but is not limited to grayscale, denoising and contrast enhancement.

[0090] Then, edge detection is performed on the preprocessed wind turbine blade image, and an edge detection algorithm (such as Canny edge detection, Sobel operator, etc.) is used to detect the edge of the wind turbine blade, that is, the blade edge in the wind turbine blade image.

[0091] Next, select a suitable color space, such as HSV, Lab, etc., to more easily separate the color of the wind turbine blades from other colors in the image. Then, define the color range of the wind turbine blades, and use a color filter to select the color range of the wind turbine blades, separate the wind turbine blades from the image background, and ensure that only the information of the wind turbine blades is used for monitoring and analysis, thereby obtaining the initial contour features of the wind turbine blades.

[0092] Then, morphological operations are performed on the initial contour features. Specifically, morphological operations such as dilation and erosion are used to optimize the contour, and opening and closing operations are applied to remove noise and fill holes in the image.

[0093] Finally, the contour extraction function in OpenCV and other libraries is used to obtain the contour features of the wind turbine blades in the wind turbine blade image.

[0094] Among them, the contour features of the wind turbine blades include but are not limited to contour area, contour perimeter, contour center, contour shape description, contour strength, and contour sharpness.

[0095] In this embodiment, edge detection is performed on the wind turbine blade image to obtain the blade edge in the wind turbine blade image, and based on the blade edge, a color filter is used to select the color range of the wind turbine blade, and the wind turbine blade is separated from the image background to obtain the initial contour features of the wind turbine blade. Morphological operations are then performed on the initial contour features to obtain the contour features of the wind turbine blade, which can help improve the accuracy and reliability of wind turbine blade monitoring, thereby helping operation and maintenance personnel to promptly discover and solve problems and ensure the safe operation of wind turbines.

[0096] On the basis of the above-mentioned embodiment, the process of performing positioning compensation on the wind turbine blades in the wind turbine blade image by visual guidance will be described in detail below.

[0097] According to the contour features of the wind turbine blades, positioning compensation is performed on the wind turbine blades in the wind turbine blade image in a visually guided manner, including: according to the contour features of the wind turbine blades, analyzing the combined peak signals from two wind turbine blades and the uppermost wind turbine blade to identify the positions of all the wind turbine blades; wherein the combined peak signal characterizes the intensity change of the blade edge in the wind turbine blade image; according to the positions of all the wind turbine blades, determining the vertical axis representation of one group of wind turbine blades; calculating the attitude of the wind turbine by analyzing the vertical axis representation of the wind turbine blades, the attitude including the roll angle, the pitch angle and the yaw angle; according to the attitude of the wind turbine, adjusting the wind turbine blade image to compensate for the visual deviation caused by the change in the attitude of the wind turbine, so as to obtain the target wind turbine blade image.

[0098] It is easy to understand that, first, after obtaining the contour features of the wind turbine blades, the intensity change of the edge of the wind turbine blades, i.e., the combined peak signal, can be calculated based on the contour features of the wind turbine blades (the contour features include the contour strength). According to the combined peak signal of the edge of each wind turbine blade, the positions of all wind turbine blades in the current wind turbine can be identified.

[0099] Then, in order to better position the blades, a set of vertical axis representations of the wind turbine blades can be set according to the position of the wind turbine blades. This means that the vertical position of the wind turbine blades relative to the ground or camera coordinate system needs to be determined. This step is crucial to eliminate the interference of other objects (such as the centerline of the tower, background boundaries, etc.).

[0100] Next, by analyzing the vertical axis representation of the wind turbine blades, the roll angle, pitch angle and yaw angle of the wind turbine can be calculated.

[0101] Finally, the roll angle, pitch angle and yaw angle of the wind turbine are used to adjust the mobile camera to compensate for the visual deviation caused by the change of the wind turbine posture. Therefore, after adjusting the mobile camera, the mobile camera is used to capture the image of the wind turbine blades at a new angle and position. This wind turbine blade image is the target wind turbine blade image.

[0102] In this embodiment, the accuracy and robustness of the vehicle-mounted safety monitoring system can be improved through vision-guided positioning compensation, ensuring that the monitoring results are not affected by changes in the posture of the wind turbine. This method is crucial to ensuring the safe operation and maintenance of the wind turbine.

[0103] On the basis of the above-mentioned embodiment, the process of monitoring the wind turbine blades in the target wind turbine blade image by using the random sampling consistency technology will be described in detail below.

[0104] According to a random sampling consistency algorithm, the wind turbine blades in the target wind turbine blade image are monitored, including: fitting a wind turbine blade feature model according to key blade features in the target wind turbine blade image; based on the wind turbine blade feature model, the wind turbine blades in the target wind turbine blade image are monitored to obtain wind turbine blade stress and strain monitoring results; wherein the wind turbine blade stress and strain monitoring results include monitoring positions and their corresponding stress values ​​and stress values.

[0105] It is understandable that this embodiment uses RANSAC technology to identify wind turbine defects such as corrosion and fracture. RANSAC technology fits the characteristic model of the blade surface, that is, the wind turbine blade characteristic model, through random sampling consistency, so as to accurately locate the wind turbine (blade) defects.

[0106] Specifically, first, like the wind turbine blade image, a preprocessing operation is performed on the target wind turbine image, and key blade features are extracted therein. The key blade features here may be the contour features mentioned above.

[0107] Then, a portion of the extracted key blade features is randomly selected as samples, and a wind turbine blade feature model is fitted based on these sample points, such as a straight line model, a curve model or a plane model, depending on the type of defects that need to be detected.

[0108] Then, the fitting degree between the remaining key blade features and the wind turbine blade feature model is detected.

[0109] Repeat the above steps (random sampling, model fitting, consistency test) multiple times, using a different set of random sample points each time, until the optimal model is found. The optimal model here is the model with the highest degree of consistency in multiple consistency tests, the model that best represents the normal state of the wind turbine blades, and is also the final wind turbine blade feature model.

[0110] Finally, based on the fitted characteristic model of wind turbine blades, all key blade features (points) are detected, and the characteristic points that do not conform to the model are regarded as potential defect areas. It can also be inferred that the stress and strain values ​​of these characteristic points that do not conform to the model are large. Then, the strain can be measured by the strain gauge pasted on the surface of the wind turbine blade, and then the stress can be calculated according to the elastic modulus of the material, thereby obtaining the stress and strain monitoring results of the wind turbine blades.

[0111] It should be noted that the wind turbine blade stress and strain monitoring results herein particularly refer to the stress and strain monitoring results corresponding to the wind turbine blade regions that may have defects.

[0112] In this embodiment, a wind turbine blade feature model is fitted based on key blade features in the target wind turbine blade image, and based on the wind turbine blade feature model, the wind turbine blade in the target wind turbine blade image is monitored to obtain the wind turbine blade stress and strain monitoring results. This method uses random sampling consistency technology to identify wind turbine defects such as corrosion and fracture to obtain more accurate wind turbine blade stress and strain monitoring results, thereby achieving high-precision monitoring of the stress and strain conditions of wind turbine blades and providing more reliable data for the health status assessment of wind turbines.

[0113] On the basis of the above-mentioned embodiment, the vehicle safety monitoring system in step S110 will be described in detail below.

[0114] The on-board safety monitoring system includes the first layer, the second layer and the third layer; the first layer is arranged with magnetic sensors for monitoring defects and corrosion of the base plate; the second layer is equipped with a control unit for coordinating the operation of the entire system, including data collection, processing and transmission; the third layer is equipped with a mobile camera for capturing images of wind turbine blades of wind turbine blades in the target area; the on-board safety monitoring system moves by driving the wheels with motors to approach different positions of the wind turbine.

[0115] It is understandable that the vehicle-mounted safety monitoring system used when using a mobile camera to obtain images of wind turbine blades in the target area is innovatively designed. It is a multi-level detection vehicle system that integrates multiple sensors and technologies for monitoring wind turbine blades. The vehicle-mounted safety monitoring system integrates multiple components such as magnetic sensors, control units, signal acquisition and transmission modules, battery modules, Wi-Fi modules, cameras and alarm modules. These modules will be described in detail below.

[0116] The on-board safety monitoring system consists of the first, second and third layers, of which the first layer is the bottom layer and is equipped with magnetic sensors to detect defects and corrosion on the bottom plate of the wind turbine. The magnetic sensors can detect changes in the magnetic properties of the material, thereby identifying potential structural problems.

[0117] The second layer is the middle layer, which is equipped with a control unit. The control unit is responsible for coordinating the operation of the entire monitoring system, including data collection, processing and transmission.

[0118] In addition to the control unit, the second layer is also equipped with a signal acquisition and transmission module, a battery module and a WIFI module. The signal acquisition and transmission module is used to collect sensor data and send it to a remote server or monitoring center through a WIFI module or other wireless technologies. The battery module is used to power the entire monitoring system, and the WIFI module is used for wireless data transmission.

[0119] The third layer is the top layer, which is equipped with a mobile camera and an alarm module. The mobile camera is used to capture images of wind turbine blades, and the alarm module is used to send a warning signal when an abnormal situation is detected.

[0120] The vehicle-mounted safety monitoring system of this embodiment moves by means of motor-driven wheels, so that it can flexibly approach different positions of the wind turbine generator set.

[0121] The wheels here include a driving wheel, and the driving wheel is equipped with a pulse sensor connected to the control unit; wherein, when the driving wheel rotates one circle, the pulse sensor will correspondingly generate a pulse signal.

[0122] Correspondingly, the wind turbine blade stress and strain monitoring method also includes: obtaining the number of pulse signals generated by the pulse sensor; calculating the distance between the monitored corrosion position and / or fracture position and the starting point based on the number of pulse signals to locate the corrosion position and / or fracture position.

[0123] In this embodiment, the image of the wind turbine blades is obtained by a mobile camera installed on an innovatively designed on-board safety monitoring system, and the number of pulse signals generated by the pulse sensor in the on-board safety monitoring system is obtained. Then, based on the number of pulse signals, the distance between the monitored corrosion position and / or fracture position and the starting point is calculated to locate the corrosion position and / or fracture position, thereby achieving comprehensive monitoring of the wind turbine blades and the wind turbine base plate, improving the accuracy and efficiency of monitoring, and helping to ensure the safe operation of the wind turbine.

[0124] On the basis of the above-mentioned embodiments, the subsequent application of the wind turbine blade stress and strain monitoring results will be described in detail below.

[0125] It is understandable that after obtaining the stress-strain monitoring results of the wind turbine blades, if the strain value at a certain position increases abnormally, it means that cracks or other defects may have appeared at that position, thereby achieving the purpose of evaluating the structural integrity of the wind turbine blades.

[0126] At the same time, the fatigue life of wind turbine blades can be predicted using stress-life curves based on the stress-strain monitoring results of wind turbine blades and the fatigue properties of wind turbine materials. Historical data can also be collected and statistical methods can be used to analyze the failure mode and failure time of wind turbine blades to predict the remaining service life of wind turbine blades.

[0127] In addition, the stress and strain monitoring results of wind turbine blades can also be used to decide when to perform maintenance or overhaul. Specifically, maintenance or overhaul is triggered when the stress and strain value exceeds the set threshold.

[0128] Finally, the stress and strain monitoring results of wind turbine blades can also be used to determine whether the wind turbine blades meet safety standards in their current state. Specifically, the stress and strain values ​​are compared with the preset safety range to evaluate the safety risk level of the wind turbine blades.

[0129] In this embodiment, the structural integrity and safety of the wind turbine blades can be ensured through the stress and strain monitoring results of the wind turbine blades, while the ability to predict maintenance can be improved and the risk of unexpected downtime can be reduced.

[0130] Corresponding to the wind turbine blade stress and strain monitoring method based on machine vision described in the above embodiments, the present invention also provides a wind turbine blade stress and strain monitoring device based on machine vision. Specifically, Figure 2 A schematic structural diagram of a wind turbine blade stress and strain monitoring device based on machine vision provided by an embodiment of the present invention is shown.

[0131] like Figure 2 As shown, the device includes: a wind turbine blade image acquisition module 210, which is used to use a mobile camera in a vehicle-mounted safety monitoring system to acquire a wind turbine blade image in a target area; a wind turbine blade contour feature extraction module 220, which is used to extract contour features of the wind turbine blades in the wind turbine blade image; a wind turbine blade positioning compensation module 230, which is used to perform positioning compensation on the wind turbine blades in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blades, so as to obtain a target wind turbine blade image; a wind turbine blade stress and strain monitoring module 240, which is used to monitor the wind turbine blades in the target wind turbine blade image according to a random sampling consistency algorithm, so as to obtain a wind turbine blade stress and strain monitoring result.

[0132] In this embodiment, the wind turbine blade image acquisition module 210 uses the mobile camera in the vehicle-mounted safety monitoring system to acquire the wind turbine blade image of the target area, and the wind turbine blade contour feature extraction module 220 extracts the contour features of the wind turbine blade in the wind turbine blade image, and then the wind turbine blade positioning compensation module 230 performs positioning compensation on the wind turbine blade in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blade, and obtains the target wind turbine blade image, thereby, the wind turbine blade stress and strain monitoring module 240 monitors the wind turbine blade in the target wind turbine blade image according to the random sampling consistency algorithm, and obtains the wind turbine blade stress and strain monitoring result. The device performs positioning compensation on the wind turbine blade in a visually guided manner, realizes the precise positioning of the wind turbine blade, and further, by using the random sampling consistency technology to identify wind turbine defects such as corrosion and fracture, so as to obtain a wind turbine blade stress and strain monitoring result with higher accuracy, realizes the high-precision monitoring of the stress and strain of the wind turbine blade, and provides more reliable data for the health status assessment of the wind turbine.

[0133] It should be noted that the wind turbine blade stress and strain monitoring device based on machine vision provided in this embodiment and the wind turbine blade stress and strain monitoring method based on machine vision described in the above embodiments can correspond to each other and will not be described in detail here.

[0134] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a wind turbine blade stress and strain monitoring method based on machine vision, the method comprising: using a mobile camera in a vehicle-mounted safety monitoring system to obtain a wind turbine blade image in a target area; extracting the contour features of the wind turbine blade in the wind turbine blade image; according to the contour features of the wind turbine blade, positioning and compensating the wind turbine blade in the wind turbine blade image in a visually guided manner to obtain a target wind turbine blade image; monitoring the wind turbine blade in the target wind turbine blade image according to a random sampling consistency algorithm to obtain a wind turbine blade stress and strain monitoring result.

[0135] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the machine vision-based wind turbine blade stress and strain monitoring method provided by the above-mentioned methods. The method includes: using a mobile camera in a vehicle-mounted safety monitoring system to obtain an image of a wind turbine blade in a target area; extracting contour features of the wind turbine blade in the wind turbine blade image; based on the contour features of the wind turbine blade, positioning and compensating the wind turbine blade in the wind turbine blade image in a visually guided manner to obtain a target wind turbine blade image; and monitoring the wind turbine blade in the target wind turbine blade image according to a random sampling consistency algorithm to obtain a wind turbine blade stress and strain monitoring result.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring stress and strain of wind turbine blades based on machine vision, characterized in that: include: Using the mobile camera in the vehicle-mounted safety monitoring system, the wind turbine blade images in the target area are acquired; Extracting contour features of the wind turbine blades in the wind turbine blade image; According to the contour features of the wind turbine blade, positioning compensation is performed on the wind turbine blade in the wind turbine blade image in a visually guided manner to obtain a target wind turbine blade image; The wind turbine blades in the target wind turbine blade image are monitored according to a random sampling consistency algorithm to obtain wind turbine blade stress and strain monitoring results.

2. The method for monitoring wind turbine blade stress and strain based on machine vision according to claim 1, characterized in that: The extracting of the contour features of the wind turbine blades in the wind turbine blade image comprises: Performing edge detection on the wind turbine blade image to obtain blade edges in the wind turbine blade image; Based on the blade edge, a color filter is used to select a color range of the wind turbine blade, the wind turbine blade is separated from the image background, and an initial contour feature of the wind turbine blade is obtained; A morphological operation is performed on the initial contour feature to obtain the contour feature of the wind turbine blade.

3. The method for monitoring wind turbine blade stress and strain based on machine vision according to claim 1, characterized in that: The method of performing positioning compensation on the wind turbine blade in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blade comprises: According to the contour features of the wind turbine blades, the combined peak signals from the two wind turbine blades and the uppermost wind turbine blade are analyzed to identify the positions of all the wind turbine blades; wherein the combined peak signal represents the intensity change of the blade edge in the wind turbine blade image; According to the positions of all the wind turbine blades, a vertical axis representation of one group of wind turbine blades is determined; By analyzing the vertical axis representation of the wind turbine blades, the attitude of the wind turbine is calculated, wherein the attitude includes a roll angle, a pitch angle, and a yaw angle; According to the posture of the wind turbine generator set, the wind turbine generator set blade image is adjusted to compensate for the visual deviation caused by the change of the wind turbine generator set posture, so as to obtain the target wind turbine generator set blade image.

4. The method for monitoring wind turbine blade stress and strain based on machine vision according to claim 1, characterized in that: The monitoring of the wind turbine blades in the target wind turbine blade image according to the random sampling consistency algorithm includes: According to the key blade features in the target wind turbine blade image, a wind turbine blade feature model is obtained by fitting; Based on the wind turbine blade feature model, the wind turbine blade in the target wind turbine blade image is monitored to obtain a stress and strain monitoring result of the wind turbine blade; The wind turbine blade stress and strain monitoring results include the monitoring position and its corresponding stress value and stress value.

5. The method for monitoring wind turbine blade stress and strain based on machine vision according to claim 1, characterized in that: The vehicle safety monitoring system includes a first layer, a second layer and a third layer; wherein, The first layer is provided with magnetic sensors for monitoring defects and corrosion of the bottom plate; The second layer is equipped with a control unit for coordinating the operation of the entire system, including data collection, processing and transmission; The third layer is equipped with a mobile camera for capturing wind turbine blade images of wind turbine blades in a target area; The vehicle-mounted safety monitoring system moves by driving wheels with motors to approach different positions of the wind turbine generator set.

6. The method for monitoring wind turbine blade stress and strain based on machine vision according to claim 5, characterized in that: The wheel comprises a driving wheel, and the driving wheel is equipped with a pulse sensor connected to the control unit; wherein, when the driving wheel rotates one circle, the pulse sensor generates a pulse signal accordingly; Correspondingly, the wind turbine blade stress and strain monitoring method further includes: Obtaining the number of pulse signals generated by the pulse sensor; According to the number of pulse signals, the distance between the monitored corrosion position and / or fracture position and the starting point is calculated to locate the corrosion position and / or fracture position.

7. The method for monitoring stress and strain of wind turbine blades based on machine vision according to any one of claims 1 to 6, characterized in that: Also includes: Evaluating the structural integrity of the wind turbine blades according to the wind turbine blade stress and strain monitoring results; and / or, predicting the remaining service life of the wind turbine blades according to the stress and strain monitoring results of the wind turbine blades; and / or, determining maintenance and repair decisions based on the wind turbine blade stress and strain monitoring results; And / or, performing a safety assessment on the wind turbine blades according to the stress and strain monitoring results of the wind turbine blades.

8. A wind turbine blade stress and strain monitoring device based on machine vision, characterized in that: include: A wind turbine blade image acquisition module is used to acquire wind turbine blade images in a target area using a mobile camera in a vehicle-mounted safety monitoring system; A wind turbine blade contour feature extraction module, used to extract contour features of the wind turbine blade in the wind turbine blade image; A wind turbine blade positioning compensation module is used to perform positioning compensation on the wind turbine blades in the wind turbine blade image in a visually guided manner according to the contour features of the wind turbine blades to obtain a target wind turbine blade image; The wind turbine blade stress and strain monitoring module is used to monitor the wind turbine blades in the target wind turbine blade image according to a random sampling consistency algorithm to obtain wind turbine blade stress and strain monitoring results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for monitoring stress and strain of wind turbine blades based on machine vision according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring stress and strain of a blade of a wind turbine generator system based on machine vision as claimed in any one of claims 1 to 7 is implemented.

Images