Fan blade damage state real-time monitoring method based on image recognition

Abstract

The invention discloses a fan blade damage state real-time monitoring method based on image recognition, and the method comprises the following steps: S1, carrying out the shooting of a fan blade through a camera device, and obtaining a blade sample; s2, carrying out frame extraction preprocessing on the samples, removing picture frames with similar states, and making a training set; s3, constructing a blade damage detection model according to the training set; and S4, deploying the blade damage detection model to the wind field for operation testing, performing real-time damage detection on the blades in the wind field by the blade damage detection model, and feeding back a detection result to the fan master control PLC for early warning processing. According to the method, the blade operation state video can be collected in real time through the pan-tilt camera, the damage state of the fan blade is analyzed in real time through the image processing technology of the backbone network YOLOv5, fan control is conducted in time, and potential safety hazards caused by blade damage to the fan are reduced to the maximum extent.

Description

Technical Field

[0001] This invention relates to the technical field of wind turbine blade damage monitoring, and in particular to a real-time monitoring method for wind turbine blade damage status based on image recognition. Background Technology

[0002] As a core component of wind turbine generators, the blades' operational status directly impacts the unit's availability and, consequently, the overall investment return of the wind farm. Statistics on wind farm equipment malfunctions show that blade failures have the longest average troubleshooting time, severely affecting the economic benefits of wind farms. The wind power industry has long lacked effective real-time blade condition monitoring methods in wind farm operation and maintenance. If minor bulges, cracks, or lightning strikes on blades are not detected and repaired promptly, they can range from affecting the unit's wind capture capacity and reducing power generation to potentially causing blade breakage, resulting in serious safety risks and significant economic losses.

[0003] Existing technologies mostly assess damage indirectly, such as by combining SCADA data from wind turbine operation with nacelle vibration data to train models and predict whether the unit is damaged. This method has some effectiveness, but it cannot accurately predict damage. False alarms often cause unnecessary shutdowns and loss of power generation.

[0004] In addition, most wind turbines currently rely on manual methods to determine whether the blades are damaged when facing damage issues. This can be achieved by regularly arranging personnel to conduct inspections or by arranging drones to conduct regular inspections and manually shutting down the turbines. While this can ensure the accuracy of manual judgment, it is difficult to guarantee timeliness. Therefore, it is not possible to immediately shut down the turbines when damage occurs.

[0005] The above methods all have certain drawbacks. False alarms may cause power generation losses, or failure to detect damage in a timely manner may result in the wind turbine blades operating normally, thus endangering the wind turbine's service life. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a real-time monitoring method for wind turbine blade damage based on image recognition. This method uses a PTZ camera to collect videos of blades during damage, normal operation, and foggy conditions. Then, valid frames are extracted from the videos to create eight training sets, which are used to train a damage detection model. Subsequently, the video data is organized to create more training sets and optimize the damage detection model. Finally, the damage detection model is deployed in the wind farm for 24 / 7 operation. The PTZ camera can be set to operate 24 / 7 or collect data periodically. Real-time damage detection is performed based on image recognition technology, and the final results are fed back to the main control PLC for further early warning processing.

[0007] The objective of this invention is achieved through the following technical solution: a real-time monitoring method for wind turbine blade damage status based on image recognition, comprising the following steps: S1. Take pictures at three locations on the wind turbine blades: the tip, the middle, and the root, using a camera to obtain blade damage samples, undamaged blade samples, blade samples taken throughout the day in foggy weather, and blade samples taken throughout the day in sunny weather. S2. Perform frame preprocessing on each sample obtained in step S1, remove image frames with similar states, and create eight training sets, including a blurry sample training set, a bladeless and undamaged sample training set, a bladeless and damaged sample training set, a bladeless and undamaged sample training set, a blade with bulging damage sample training set, a blade with cracked damage sample training set, a blade with lightning strike damage sample training set, and a blade with fracture damage sample training set. S3. Construct a blade damage detection model based on the eight training sets in step S2; S4. Deploy the blade damage detection model to the wind farm. The blade damage detection model performs real-time damage detection on the wind farm blades and feeds the detection results back to the wind turbine main control PLC for early warning processing.

[0008] Furthermore, step S1 includes: A gimbal camera was mounted on the top of the wind turbine nacelle to photograph each wind turbine blade from the left, middle, and right sides. The tip, middle, and root of each blade were also photographed separately to collect blade damage samples, undamaged blade samples, blade samples taken throughout the day in fog, and blade samples taken throughout the day in sunny weather. The gimbal camera was also set to run for a specific time, including both all-day operation and timed operation.

[0009] Furthermore, step S2 includes: The blade damage samples, blade undamaged samples, all-day foggy blade samples, and all-day sunny blade samples obtained in step S1 are preprocessed by frame extraction. Each frame of each sample is processed into a static image frame, and image frames with similar states are removed. Eight training sets are created, including a training set of blurred samples, a training set of samples without blades and without damage, a training set of samples without blades and with damage, a training set of samples with blades and with bulging damage, a training set of samples with blades and with cracking damage, a training set of samples with blades and with lightning strike damage, and a training set of samples with blades and with fracture damage.

[0010] Furthermore, step S3 includes the following steps: S3.1 Read eight types of training sets, namely, fuzzy sample training set, no blade and no damage sample training set, no blade and damaged sample training set, blade and no damage sample training set, blade and bulge damage sample training set, blade and crack damage sample training set, blade and lightning strike damage sample training set, and blade and fracture damage sample training set. S3.2. Deep learning is used to augment the data of eight training sets, simulating samples under different working conditions, environments and lighting conditions to increase sample diversity. S3.3, Set hyperparameters; S3.4. Set up the backbone network YOLOv5 for training; S3.5. Use the augmented sample training set to train the backbone network YOLOv5. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is completed and output the statistical results. S3.6 Set the output path and output the trained blade damage detection model .pth file.

[0011] Furthermore, step S3.2 includes: RandomHorizontal Flip is used for random horizontal flipping, RandomVertical Flip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image to grayscale.

[0012] Furthermore, step S4 includes the following steps: S4.1 Deploy the blade damage detection model in the wind farm for testing; S4.2 The real-time wind turbine blade video collected is stored in the specified folder of the video controller through the wind farm ring network, and the output save folder of the analysis results is defined to store the video and result files after analysis and processing; S4.3. Use os.listdir() to monitor the storage files of captured videos in real time. If it is a new video that has not been analyzed, perform video analysis and prediction. S4.4 Convert the leaf damage detection model to an RKNN model, set the parameters for YOLOv5 network loading and training, modify the network's output layer, load the trained leaf damage detection model, and transfer the leaf damage detection model to the GPU to speed up its operation. S4.5 Set the parameters for the output video, reduce the video resolution, and analyze and predict the newly acquired videos using the loaded blade damage detection model. S4.6 For blade damage detection, obtain the prediction results for each frame in the video: 0, 1, 2, 3, 4 and 5, where 0 represents no damage, 1 represents blurry, 2 represents bulging damage, 3 represents cracking damage, 4 represents lightning strike damage, and 5 represents fracture damage. Write the prediction results into the output video and the txt file containing the statistical prediction results, respectively. S4.7 For blade damage detection, if the prediction result does not contain 1, the frequencies of 0, 2, 3, 4, and 5 are calculated respectively. If the frequency of 2 exceeds the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 3 exceeds the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 4 exceeds the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 5 exceeds the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. If the prediction result contains 1, the frequencies of 0, 1, 2, 3, 4, and 5 are counted respectively. If the frequency of 1 is less than 0.3 and the frequency of 2 is greater than the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 1 is less than 0.3 and the frequency of 3 is greater than the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 1 is less than 0.3 and the frequency of 4 is greater than the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 1 is less than 0.3 and the frequency of 5 is greater than the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. S4.8 Record the analysis results obtained in step S4.7 in a txt document and write the results in the file name of the output video to facilitate the extraction of intelligent analysis results.

[0013] A real-time monitoring system for wind turbine blade damage status based on image recognition, used to implement the aforementioned real-time monitoring method for wind turbine blade damage status based on image recognition, includes: The camera module is used to capture images at three locations on the wind turbine blades: the tip, the middle, and the root, to obtain video samples of the blades. The sample acquisition module is used to read leaf video samples captured by the camera module; the leaf video samples include leaf damage samples, leaf undamaged samples, leaf samples taken throughout the day in foggy weather, and leaf samples taken throughout the day in sunny weather. The frame extraction preprocessing module is used to perform frame extraction preprocessing on the video samples of each blade obtained by the sample acquisition module, remove image frames with similar states, and create a training set. The training set includes a blurred sample training set, a sample training set without blades and without damage, a sample training set without blades and with damage, a sample training set with blades and without damage, a sample training set with blades and with bulging damage, a sample training set with blades and with cracked damage, a sample training set with blades and with lightning strike damage, and a sample training set with blades and with broken damage. The blade damage detection module constructs and trains a blade damage detection model based on the training set for real-time damage detection of wind turbine blades. The detection and analysis module analyzes the detection results from the blade damage detection module to obtain the final detection results.

[0014] Furthermore, the blade damage detection module performs the following operations: a. Read eight types of training sets, namely, fuzzy sample training set, bladeless and undamaged sample training set, bladeless and damaged sample training set, bladeless and undamaged sample training set, blade and bulging damaged sample training set, blade and cracked damaged sample training set, blade and lightning strike damaged sample training set, and blade and fracture damaged sample training set. b. Deep learning is used to augment the data of eight training sets to simulate samples under different working conditions, environments and lighting conditions, thereby increasing sample diversity; RandomHorizontalFlip is used for random horizontal flipping, RandomVerticalFlip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image grayscale. c. Set hyperparameters; d. Configure the training backbone network YOLOv5; e. Use the augmented sample training set to train the backbone network YOLOv5. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is completed and output the statistical results. f. Set the output path and output the trained blade damage detection model .pth file.

[0015] A non-transitory computer-readable medium storing instructions that, when executed by a processor, perform the steps of the above-described image recognition-based real-time monitoring method for wind turbine blade damage status.

[0016] A computing device includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the above-described image recognition-based real-time monitoring method for wind turbine blade damage status.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. This invention can collect real-time video of blade operation status through a gimbal camera and analyze it in real time through the image processing technology of YOLOv5 backbone network, which improves timeliness compared with traditional manual maintenance. 2. Based on the results of image judgment, the present invention can analyze the damage status of wind turbine blades in real time, and control the wind turbine in a timely manner to minimize the safety hazards caused by blade damage to the wind turbine. Attached Figure Description

[0018] Figure 1 This is a schematic diagram showing the installation orientation of the gimbal camera.

[0019] Figure 2 Flowchart for training a blade damage detection model.

[0020] Figure 3 Flowchart for real-time monitoring of wind turbine blade damage status. Detailed Implementation

[0021] The present invention will be further described below with reference to specific embodiments.

[0022] Example 1 The real-time monitoring method for wind turbine blade damage based on image recognition provided in this embodiment includes the following steps: S1, see also Figure 1 As shown, a gimbal camera was installed on the top of the wind turbine nacelle to photograph each wind turbine blade from left position 1, middle position 2, and right position 3. The tip, middle, and root of each wind turbine blade were photographed separately to collect blade damage samples, undamaged blade samples, blade samples taken throughout the day in foggy weather, and blade samples taken throughout the day in sunny weather. At the same time, the gimbal camera was set to run for a period of time, including running all day and running at a set time.

[0023] S2. Perform frame-sampling preprocessing on the blade damage samples, undamaged blade samples, all-day foggy blade samples, and all-day sunny blade samples obtained in step S1. Process each frame of each sample into a static image frame, remove image frames with similar states, and create eight training sets. The training sets include: fuzzy sample training set, no-blade undamaged sample training set, no-blade damaged sample training set, blade undamaged sample training set, blade bulging damaged sample training set, blade cracked damaged sample training set, blade lightning strike damaged sample training set, and blade fractured damaged sample training set.

[0024] S3, see also Figure 2 As shown, a blade damage detection model is constructed based on the eight training sets in step S2. S3.1 Read eight types of training sets, namely, fuzzy sample training set, no blade and no damage sample training set, no blade and damaged sample training set, blade and no damage sample training set, blade and bulging damage sample training set, blade and cracking damage sample training set, blade and lightning strike damage sample training set, and blade and fracture damage sample training set.

[0025] S3.2. Deep learning is used to augment the data of eight training sets, simulating samples under different working conditions, environments and lighting conditions to increase sample diversity; RandomHorizontalFlip is used for random horizontal flipping, RandomVerticalFlip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image grayscale.

[0026] S3.3, Set hyperparameters.

[0027] S3.4. Set up the backbone network YOLOv5 for training.

[0028] S3.5. Use the augmented sample training set to train the backbone network YOLOv5. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is complete and output the statistical results.

[0029] S3.6 Set the output path and output the trained blade damage detection model .pth file.

[0030] S4. Deploy the blade damage detection model to the wind farm for testing. The blade damage detection model performs real-time damage detection on the wind farm blades and feeds the detection results back to the wind turbine main control PLC for early warning processing.

[0031] S4.1 Deploy the blade damage detection model in the wind field for testing.

[0032] S4.2 The real-time wind turbine blade video collected is stored in the specified folder of the video controller through the wind farm ring network, and the output save folder of the analysis results is defined to store the video and result files after analysis and processing.

[0033] S4.3. Use os.listdir() to monitor and collect video storage files in real time. If it is a new video that has not been analyzed, perform video analysis and prediction.

[0034] S4.4 Convert the leaf damage detection model to an RKNN model, set the parameters for YOLOv5 network loading and training, modify the network's output layer, load the trained leaf damage detection model, and transfer the leaf damage detection model to the GPU to speed up its operation.

[0035] S4.5 Set the parameters for the output video, reduce the video resolution, and analyze and predict the newly acquired video using the loaded blade damage detection model and blade icing detection model respectively.

[0036] S4.6 For blade damage detection, obtain the prediction results for each frame in the video: 0, 1, 2, 3, 4 and 5, where 0 represents no damage, 1 represents blurry, 2 represents bulging damage, 3 represents cracking damage, 4 represents lightning strike damage, and 5 represents fracture damage. Write the prediction results into the output video and the txt file containing the statistical prediction results, respectively.

[0037] S4.7 For blade damage detection, if the prediction result does not contain the number 1, the frequencies of 0, 2, 3, 4, and 5 are calculated respectively. If the frequency of 2 exceeds the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 3 exceeds the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 4 exceeds the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 5 exceeds the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. If the prediction result contains the number 1, the frequencies of 0, 1, 2, 3, 4, and 5 are counted respectively. If the frequency of 1 is less than 0.3 and the frequency of 2 is greater than the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 1 is less than 0.3 and the frequency of 3 is greater than the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 1 is less than 0.3 and the frequency of 4 is greater than the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 1 is less than 0.3 and the frequency of 5 is greater than the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage.

[0038] S4.8 Record the analysis results obtained in step S4.7 in a txt document and write the results in the file name of the output video to facilitate the extraction of intelligent analysis results.

[0039] Example 2 The real-time monitoring method for wind turbine blade damage based on image recognition provided in this embodiment differs from Embodiment 1 in that step S4 includes: Preferably, see Figure 3 As shown, the blade damage detection model is deployed in the wind farm and tested together with the wind farm's pre-set blade icing detection model trained on the EfficientNet-B0 network. The blade icing detection model performs real-time icing detection on the wind farm blades, and the blade damage detection model performs real-time damage detection on the wind farm blades. The detection results are fed back to the wind turbine's main control PLC for early warning processing.

[0040] S4.1 Deploy the blade damage detection model in the wind field and run it together with the blade icing detection model trained on the EfficientNet-B0 network, which is pre-set in the wind field.

[0041] S4.2 The real-time wind turbine blade video collected is stored in the specified folder of the video controller through the wind farm ring network, and the output save folder of the analysis results is defined to store the video and result files after analysis and processing.

[0042] S4.3. Use os.listdir() to monitor and collect video storage files in real time. If it is a new video that has not been analyzed, perform video analysis and prediction.

[0043] S4.4 Convert the leaf damage detection model and the leaf icing detection model into RKNN models. Set the loading and training parameters for the YOLOv5 and EfficientNet-B0 networks. The YOLOv5 network is used to load the leaf damage detection model, and the EfficientNet-B0 network is used to load the leaf icing detection model. Modify the output layer of the network to load the trained leaf damage detection model, and transfer the leaf damage detection model to the GPU to speed up its operation.

[0044] S4.5 Set the parameters for the output video, reduce the video resolution, and analyze and predict the newly acquired video using the loaded blade damage detection model and blade icing detection model respectively.

[0045] S4.6 For blade icing detection, obtain the prediction results for each frame in the video: a, b, and c, where a represents no icing, b represents blurry, and c represents icing; For blade damage detection, obtain the prediction results for each frame in the video: 0, 1, 2, 3, 4, and 5, where 0 represents no damage, 1 represents blurry, 2 represents bulging damage, 3 represents cracking damage, 4 represents lightning strike damage, and 5 represents fracture damage. Write the prediction results into the output video and the txt file containing the statistical prediction results, respectively.

[0046] S4.7 For leaf icing detection, if b is not in the prediction result, the frequencies of a and c are calculated separately. If the frequency of c exceeds the preset icing detection threshold, it is determined to be icing; otherwise, it is not icing. If b is in the prediction result, the frequencies of a, b, and c are counted separately. If the frequency of b is less than 0.3 and the frequency of c is greater than the icing threshold, it is determined to be icing; otherwise, it is not icing.

[0047] For blade damage detection, if the prediction result does not contain the number 1, the frequencies of 0, 2, 3, 4, and 5 are calculated respectively. If the frequency of 2 exceeds the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 3 exceeds the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 4 exceeds the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 5 exceeds the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. If the prediction result contains the number 1, the frequencies of 0, 1, 2, 3, 4, and 5 are counted respectively. If the frequency of 1 is less than 0.3 and the frequency of 2 is greater than the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 1 is less than 0.3 and the frequency of 3 is greater than the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 1 is less than 0.3 and the frequency of 4 is greater than the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 1 is less than 0.3 and the frequency of 5 is greater than the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage.

[0048] S4.8 Record the analysis results obtained in step S4.7 in a txt document and write the results in the file name of the output video to facilitate the extraction of intelligent analysis results.

[0049] Combining the blade damage detection model with the blade icing detection model trained on the EfficientNet-B0 network based on the wind field preset can better monitor the real-time status of the blades.

[0050] Example 3 The image recognition-based real-time monitoring system for wind turbine blade damage provided in this embodiment is used to implement the image recognition-based real-time monitoring method for wind turbine blade damage described in Embodiment 1, including: The camera module is used to capture images at three locations on the wind turbine blades: the tip, the middle, and the root, to obtain video samples of the blades.

[0051] The sample acquisition module is used to read leaf video samples captured by the camera module; the leaf video samples include leaf damage samples, leaf undamaged samples, leaf samples taken throughout the day in foggy weather, and leaf samples taken throughout the day in sunny weather.

[0052] The frame extraction preprocessing module is used to perform frame extraction preprocessing on the video samples of each blade obtained by the sample acquisition module, remove image frames with similar states, and create a training set. The training set includes a blurred sample training set, a bladeless and undamaged sample training set, a bladeless and damaged sample training set, a bladeless and undamaged sample training set, a blade with bulging damage sample training set, a blade with cracked damage sample training set, a blade with lightning strike damage sample training set, and a blade with broken damage sample training set.

[0053] The blade damage detection module constructs and trains a blade damage detection model based on the training set for real-time damage detection of wind turbine blades, including: a. Read eight types of training sets, namely, fuzzy sample training set, bladeless and undamaged sample training set, bladeless and damaged sample training set, bladeless and undamaged sample training set, bladed and bulging damaged sample training set, bladed and cracked damaged sample training set, bladed and lightning-damaged sample training set, and bladed and fractured damaged sample training set.

[0054] b. Deep learning is used to augment the data of eight training sets, simulating samples under different working conditions, environments and lighting conditions to increase sample diversity; RandomHorizontalFlip is used for random horizontal flipping, RandomVerticalFlip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image grayscale.

[0055] c. Set hyperparameters.

[0056] d. Set up the backbone network YOLOv5 for training.

[0057] e. Train the backbone network YOLOv5 using the augmented sample training set. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is complete and output the statistical results.

[0058] f. Set the output path and output the trained blade damage detection model .pth file.

[0059] The detection and analysis module analyzes the detection results from the blade damage detection module and the blade icing detection module to obtain the final detection results.

[0060] Example 4 This embodiment discloses a non-transitory computer-readable medium storing instructions that, when executed by a processor, perform the steps of the real-time monitoring method for wind turbine blade damage status based on image recognition as described in Embodiment 1.

[0061] In this embodiment, the non-transitory computer-readable medium can be a disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, etc.

[0062] Example 5 This embodiment discloses a computing device, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, it implements the real-time monitoring method for wind turbine blade damage status based on image recognition as described in Embodiment 1.

[0063] The computing device described in this embodiment may be a desktop computer, laptop computer, smartphone, PDA handheld terminal, tablet computer, programmable logic controller (PLC), or other terminal device with processor function.

[0064] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for real-time monitoring of wind turbine blade damage status based on image recognition, characterized in that, Includes the following steps: S1. Take pictures at three locations on the wind turbine blades: the tip, the middle, and the root, using a camera to obtain blade damage samples, undamaged blade samples, blade samples taken throughout the day in foggy weather, and blade samples taken throughout the day in sunny weather. S2. Perform frame preprocessing on each sample obtained in step S1, remove image frames with similar states, and create eight training sets, including a blurry sample training set, a bladeless and undamaged sample training set, a bladeless and damaged sample training set, a bladeless and undamaged sample training set, a blade with bulging damage sample training set, a blade with cracked damage sample training set, a blade with lightning strike damage sample training set, and a blade with fracture damage sample training set. S3. Construct a blade damage detection model based on the eight training sets in step S2; S4. Deploy the blade damage detection model to the wind farm. The blade damage detection model performs real-time damage detection on the wind farm blades and feeds the detection results back to the wind turbine main control PLC for early warning processing.

2. The method for real-time monitoring of wind turbine blade damage status based on image recognition according to claim 1, characterized in that, Step S1 includes: A gimbal camera was mounted on the top of the wind turbine nacelle to photograph each wind turbine blade from the left, middle, and right sides. The tip, middle, and root of each blade were also photographed separately to collect blade damage samples, undamaged blade samples, blade samples taken throughout the day in fog, and blade samples taken throughout the day in sunny weather. The gimbal camera was also set to run for a specific time, including both all-day operation and timed operation.

3. The method for real-time monitoring of wind turbine blade damage status based on image recognition according to claim 1, characterized in that, Step S2 includes: The blade damage samples, blade undamaged samples, all-day foggy blade samples, and all-day sunny blade samples obtained in step S1 are preprocessed by frame extraction. Each frame of each sample is processed into a static image frame, and image frames with similar states are removed. Eight training sets are created, including a training set of blurred samples, a training set of samples without blades and without damage, a training set of samples without blades and with damage, a training set of samples with blades and with bulging damage, a training set of samples with blades and with cracking damage, a training set of samples with blades and with lightning strike damage, and a training set of samples with blades and with fracture damage.

4. The method for real-time monitoring of wind turbine blade damage status based on image recognition according to claim 1, characterized in that, Step S3 includes the following steps: S3.1 Read eight types of training sets, namely, fuzzy sample training set, no blade and no damage sample training set, no blade and damaged sample training set, blade and no damage sample training set, blade and bulge damage sample training set, blade and crack damage sample training set, blade and lightning strike damage sample training set, and blade and fracture damage sample training set. S3.

2. Deep learning is used to augment the data of eight training sets, simulating samples under different working conditions, environments and lighting conditions to increase sample diversity. S3.3, Set hyperparameters; S3.

4. Set up the backbone network YOLOv5 for training; S3.

5. Use the augmented sample training set to train the backbone network YOLOv5. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is completed and output the statistical results. S3.6 Set the output path and output the trained blade damage detection model .pth file.

5. The method for real-time monitoring of wind turbine blade damage status based on image recognition according to claim 1, characterized in that, Step S3.2 includes: RandomHorizontal Flip is used for random horizontal flipping, RandomVertical Flip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image to grayscale.

6. The method for real-time monitoring of wind turbine blade damage status based on image recognition according to claim 1, characterized in that, Step S4 includes the following steps: S4.1 Deploy the blade damage detection model in the wind farm for testing; S4.2 The real-time wind turbine blade video collected is stored in the specified folder of the video controller through the wind farm ring network, and the output save folder of the analysis results is defined to store the video and result files after analysis and processing; S4.

3. Use os.listdir() to monitor the storage files of captured videos in real time. If it is a new video that has not been analyzed, perform video analysis and prediction. S4.4 Convert the leaf damage detection model to an RKNN model, set the parameters for YOLOv5 network loading and training, modify the network's output layer, load the trained leaf damage detection model, and transfer the leaf damage detection model to the GPU to speed up its operation. S4.5 Set the parameters for the output video, reduce the video resolution, and analyze and predict the newly acquired videos using the loaded blade damage detection model. S4.6 For blade damage detection, obtain the prediction results for each frame in the video: 0, 1, 2, 3, 4 and 5, where 0 represents no damage, 1 represents blurry, 2 represents bulging damage, 3 represents cracking damage, 4 represents lightning strike damage, and 5 represents fracture damage. Write the prediction results into the output video and the txt file containing the statistical prediction results, respectively. S4.7 For blade damage detection, if the prediction result does not contain 1, the frequencies of 0, 2, 3, 4, and 5 are calculated respectively. If the frequency of 2 exceeds the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 3 exceeds the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 4 exceeds the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 5 exceeds the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. If the prediction result contains 1, the frequencies of 0, 1, 2, 3, 4, and 5 are counted respectively. If the frequency of 1 is less than 0.3 and the frequency of 2 is greater than the preset damage detection threshold, it is determined to be bulging damage; if the frequency of 1 is less than 0.3 and the frequency of 3 is greater than the preset damage detection threshold, it is determined to be cracking damage; if the frequency of 1 is less than 0.3 and the frequency of 4 is greater than the preset damage detection threshold, it is determined to be lightning strike damage; if the frequency of 1 is less than 0.3 and the frequency of 5 is greater than the preset damage detection threshold, it is determined to be fracture damage; otherwise, it is considered no damage. S4.8 Record the analysis results obtained in step S4.7 in a txt document and write the results in the file name of the output video to facilitate the extraction of intelligent analysis results.

7. A real-time monitoring system for wind turbine blade damage status based on image recognition, characterized in that, The method for real-time monitoring of wind turbine blade damage status based on image recognition as described in any one of claims 1-6 includes: The camera module is used to capture images at three locations on the wind turbine blades: the tip, the middle, and the root, to obtain video samples of the blades. The sample acquisition module is used to read leaf video samples captured by the camera module; the leaf video samples include leaf damage samples, leaf undamaged samples, leaf samples taken throughout the day in foggy weather, and leaf samples taken throughout the day in sunny weather. The frame extraction preprocessing module is used to perform frame extraction preprocessing on the video samples of each blade obtained by the sample acquisition module, remove image frames with similar states, and create a training set. The training set includes a blurred sample training set, a sample training set without blades and without damage, a sample training set without blades and with damage, a sample training set with blades and without damage, a sample training set with blades and with bulging damage, a sample training set with blades and with cracked damage, a sample training set with blades and with lightning strike damage, and a sample training set with blades and with broken damage. The blade damage detection module constructs and trains a blade damage detection model based on the training set for real-time damage detection of wind turbine blades. The detection and analysis module analyzes the detection results from the blade damage detection module to obtain the final detection results.

8. The real-time monitoring system for wind turbine blade damage status based on image recognition according to claim 7, characterized in that, The blade damage detection module performs the following operations: a. Read eight types of training sets, namely, fuzzy sample training set, bladeless and undamaged sample training set, bladeless and damaged sample training set, bladeless and undamaged sample training set, blade and bulging damaged sample training set, blade and cracked damaged sample training set, blade and lightning strike damaged sample training set, and blade and fracture damaged sample training set. b. Deep learning is used to augment the data of eight training sets to simulate samples under different working conditions, environments and lighting conditions, thereby increasing sample diversity; RandomHorizontalFlip is used for random horizontal flipping, RandomVerticalFlip is used for random vertical flipping, ColorJitter is used to transform the contrast, saturation and zero degree of the image color, and ToTensor and Normalize are used to transform the image grayscale. c. Set hyperparameters; d. Configure the training backbone network YOLOv5; e. Use the augmented sample training set to train the backbone network YOLOv5. After each training cycle, run the preset test set once and output the accuracy to facilitate real-time monitoring of the training effect until the model training is completed and output the statistical results. f. Set the output path and output the trained blade damage detection model .pth file.

9. A non-transitory computer-readable medium storing instructions, characterized in that, When the instruction is executed by the processor, the steps of the real-time monitoring method for wind turbine blade damage status based on image recognition according to any one of claims 1-6 are performed.

10. A computing device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the real-time monitoring method for wind turbine blade damage status based on image recognition as described in any one of claims 1-6.

Images