Fan blade inner cavity defect detection method based on image recognition and deep learning

Abstract

The application discloses a fan blade inner cavity defect detection method and system based on image recognition and deep learning, which comprises the following steps: an imaging device is used to collect images of the inner cavity of a fan blade from multiple angles and positions; the collected fan blade inner cavity image dataset is preprocessed; a first layer generative adversarial network model is used to generate primary defect samples; a second layer generative adversarial network model is used to generate complex defect images; a convolutional neural network model is used for training; the preprocessed fan blade inner cavity image dataset is input into the trained convolutional neural network model; a defect report is generated based on the detection result of the defect; and when a new defect type appears, the recognition ability for the new defect is continuously optimized. Compared with existing single detection technology, the multiple network combination method of the application significantly improves the defect detection efficiency and classification ability.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade internal cavity technology, and in particular to a method for detecting defects in wind turbine blade internal cavities based on image recognition and deep learning. Background Technology

[0002] In existing technologies, defect detection in the internal cavity of wind turbine blades mainly relies on manual inspection and conventional physical inspection methods. Manual inspection methods typically involve professional inspectors using flaw detection equipment or manually inspecting the external and internal structures of wind turbine blades to detect common defects such as cracks, pores, and material spalling. However, manual inspection is inefficient, depends on the experience and skill of the inspectors, and cannot ensure comprehensive coverage of the entire blade cavity. In addition, manual inspection is prone to errors, easily resulting in missed detections and misjudgments, especially when dealing with complex and minute defects, where its detection accuracy is poor and cannot meet the requirements of the modern wind power industry for efficient and accurate inspection.

[0003] On the other hand, existing physical inspection methods typically include ultrasonic, X-ray, or infrared imaging technologies. While these can improve the accuracy and coverage of defect detection to some extent, their equipment is expensive, operation is complex, and comprehensive inspection of the internal cavities of large wind turbine blades requires a significant amount of time. More importantly, physical inspection technologies have limitations in identifying complex cracks and material degradation defects, often only able to detect specific types of defects and unable to address diverse and complex defect characteristics.

[0004] The shortcomings of existing technologies are mainly concentrated in the following aspects: First, manual inspection methods are not only time-consuming and costly, but also difficult to guarantee the accuracy of inspection when faced with complex or minute defects; second, although traditional physical inspection equipment has high accuracy, the equipment is expensive and the operation is complicated, making it difficult to achieve large-scale application; third, existing inspection technologies can often only identify single types of defects, and show great limitations when faced with compound or complex defects. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] Therefore, this invention provides a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning. Compared with existing single detection technologies, the multi-network combination method of this invention significantly improves the efficiency and classification ability of defect detection.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a method for detecting internal defects in wind turbine blades based on image recognition and deep learning, comprising:

[0008] By acquiring images of the inner cavity of the wind turbine blade from multiple angles and positions using imaging equipment, a dataset of images of the inner cavity of the wind turbine blade containing details of the inner cavity is generated.

[0009] The collected images of the wind turbine blade interior were processed for noise removal and image enhancement, and edge enhancement and contrast enhancement were performed on key areas.

[0010] A primary defect sample is generated using a first-layer generative adversarial network model. The first-layer generator generates virtual defect images of various types, including cracks, pores, and material spalling, based on the physical characteristics of the wind turbine blade's inner cavity. The first-layer discriminator is used to distinguish between the generated virtual defect images and the actual defect images.

[0011] Based on the virtual defect image generated by the first-layer generative adversarial network model, the second-layer generator of the second-layer generative adversarial network model generates complex defect images including composite cracks and material degradation. The second-layer discriminator performs fine adversarial discrimination on the complex defect images.

[0012] The convolutional neural network model is trained by combining complex defect images generated by a two-layer generative adversarial network model and virtual defect images of actual blades. The convolutional neural network model learns the features of various types and complexities of defects, thereby optimizing the defect recognition ability.

[0013] The pre-processed image dataset of the wind turbine blade cavity is input into the trained convolutional neural network model to automatically identify and locate defects in the blade cavity, including the type, location, size and shape of the defects.

[0014] A defect report is generated based on the defect detection results. The report includes the specific location, size, shape, and type of the defect.

[0015] When new defect types emerge, the two-layer generative adversarial network model and the convolutional neural network model adaptively learn through a feedback mechanism, continuously optimizing their ability to identify new defects.

[0016] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the step of generating a wind turbine blade internal cavity image dataset containing details of the wind turbine blade internal cavity includes acquiring images of the wind turbine blade internal cavity from multiple angles using an imaging device, wherein the angle range is θ, θ∈[θ...]. min ,θ max The step size of the angle θ is Δθ, and multiple images of the blade cavity from different perspectives are generated by adjusting θ.

[0017] Images are acquired at multiple locations within the inner cavity of the wind turbine blade, with coordinates P(x,y,z). The acquisition locations include the blade tip, the middle section of the blade, and the blade tail, generating image data of the inner cavity of the blade at different spatial locations.

[0018] Generate a dataset of wind turbine blade internal cavity images based on angle θ and position coordinates P(x,y,z):

[0019] I = {I1, I2, ..., I} n}

[0020] Among them, I n This represents the nth image of the wind turbine blade's internal cavity, captured at a specific angle and position.

[0021] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the noise removal and image enhancement processing includes: performing noise removal processing on the acquired wind turbine blade internal cavity image dataset using an adaptive median filtering algorithm, with a filtering window size of W. f The size W of the filtering window is dynamically adjusted based on the uniformity of noise distribution in the image of the wind turbine blade cavity. f Remove random noise while preserving edge details;

[0022] Image enhancement processing was performed on the internal cavity of the wind turbine blades to optimize the contrast of key areas in the image, with a contrast limiting factor set to α. c ;

[0023] Edge enhancement processing is performed on key regions of the wind turbine blade internal cavity image to extract edge features of the wind turbine blade internal cavity, and a gradient threshold τ is set. g ;

[0024] We obtained a dataset of wind turbine blade internal cavity images after comprehensive noise removal, contrast enhancement, and edge enhancement:

[0025] I′={I′1,I′2,…,I′ n}

[0026] Among them, I′ n This represents the nth image of the wind turbine blade's internal cavity after preprocessing.

[0027] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the virtual defect image includes: constructing a first-layer generative adversarial network model, including a first-layer generator G1 and a first-layer discriminator D1. The first-layer generator G1 generates primary virtual defect image data based on the input physical characteristics of the wind turbine blade internal cavity. The primary virtual defect image data includes cracks, pores, and material spalling. The output of the first-layer generator is the primary virtual defect image data.

[0028]

[0029] in, This is the nth virtual defect image generated;

[0030] The first layer generator G1 generates different types of virtual defects based on the physical characteristic parameters of the wind turbine blade cavity, including thickness t, material density ρ, material uniformity coefficient u, and cavity structure complexity C. Virtual defects of cracks are generated by changes in thickness t and material density ρ, virtual defects of pores are generated based on the complexity of cavity structure C, and virtual defects of material spalling are determined by material uniformity coefficient u.

[0031] The first-layer discriminator D1 receives the virtual defect image data V1 generated by the first-layer generator G1 and the actually acquired defect images R = {R1, R2, ..., R...} n The first-layer discriminator D1 maximizes the log probability function. Distinguish between virtual defect images and actual defect images, and output the judgment result:

[0032]

[0033] in, Let R be the log-probability loss function of the first-layer discriminator D1. i This is the i-th image of the internal cavity defect of the wind turbine blade that was actually acquired. For the i-th virtual defect image generated by the first-layer generator G1, α i The weight coefficients for the i-th image in the wind turbine blade internal cavity image dataset are automatically adjusted based on the defect type in the defect image, with cracks having a weight of α. crack The weight α of stomata pore And the weight α of material peeling peel Set according to the differences in the characteristics of the defects:

[0034]

[0035] By optimizing the parameters of the first-layer generator G1 and the first-layer discriminator D1 through adversarial training, the first-layer discriminator D1 is rendered unable to effectively distinguish between virtual defect images and actual defect images. The optimization objective of the first-layer discriminator is:

[0036]

[0037] in, The objective function for generating the adversarial network.

[0038] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the complex defect image includes:

[0039] A second-layer generative adversarial network (GAN) model is constructed, comprising a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer GAN model, the second-layer generator G2 further generates complex defect image data according to the input virtual defect image data. Complex defect image data includes composite cracks and material degradation;

[0040] The second-layer generator G2 combines the physical characteristics of composite cracks and material degradation, and generates complex defect images by adjusting physical parameters. The virtual defect image of the composite crack is determined based on the degree of crack overlap γ. crack The generation of virtual defect images of material degradation is based on the material structure change parameter λ. material The generated images of composite cracks and material degradation defects are as follows:

[0041] The second-layer discriminator D2 is used to compare the complex defect image V2 generated by the second-layer generator G2 with the actually acquired complex defect image. For refined adversarial discrimination, the loss function of the second-layer discriminator is:

[0042]

[0043] in, Let log be the probability loss function of the second-layer discriminator. Let i be the i-th actually acquired complex defect image. For the i-th complex defect image generated by the second-layer generator, β i β represents the weighting coefficient for complex defects in the image, and β represents the weighting coefficient for overlapping cracks. crack The weighting factor for material degradation is β. material The calculation formulas are as follows:

[0044]

[0045] The parameters of the second-layer generator G2 and the second-layer discriminator D2 are optimized through adversarial training until the second-layer discriminator D2 can no longer distinguish between complex defect images and actual defect images.

[0046]

[0047] in, The objective function for the second layer of the generative adversarial network.

[0048] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the step of training using a convolutional neural network model includes:

[0049] The complex defect image data V2 generated by the two-layer generative adversarial network convolutional neural network model and the virtual defect image data of the wind turbine blade cavity actually collected are combined to construct the input dataset D for training the convolutional neural network.

[0050] The input dataset D is fed into a convolutional neural network for training. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers.

[0051] Convolutional neural networks are trained using the backpropagation algorithm, and the loss function is... Used to measure the accuracy of convolutional neural network models in identifying defect features:

[0052]

[0053] in, The mean square error between the actual defect image and its predicted value. γ represents the mean square error between the complex defect image and its predicted value. i These are the weighting coefficients for the actual defect image and the virtual defect image;

[0054] The convolutional neural network model gradually optimizes the weight parameters W through multiple iterations. c and bias term b c This allows the convolutional neural network model to learn different types and complexities of defect features, optimize its recognition capabilities, and ultimately output a well-trained convolutional neural network model.

[0055] As a preferred embodiment of the wind turbine blade internal cavity defect detection method based on image recognition and deep learning described in this invention, the generation of the defect report includes generating a defect report based on the detected defect image data and virtual defect image data, and the report records the specific location, size, shape and type of each detected defect;

[0056] The location of the defect is determined by its coordinates (x, y) in the image coordinate system.i ,y i ) and image depth z i This means that, based on the actual physical dimensions of the wind turbine blades, it is converted into actual spatial position coordinates (X). i ,Y i Z i ), where i is the i-th defect;

[0057] Defect size ΔL i It is calculated by the pixel distance in the image and converted by combining it with the actual physical size of the pixels;

[0058] The shape of a defect is determined by analyzing the geometric features of its edges. Shape parameters include crack length, pore diameter, or area of ​​material spalling.

[0059] The defect type is determined by the classification results of the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, which outputs the probability distribution of each defect belonging to crack, porosity, or material spalling.

[0060] p i =[p crack ,p pore ,p peel ]

[0061] The generated defect report includes the specific location, size, shape, and type of each defect, and the report content is output in the following format:

[0062] Defect number, (X) i ,Y i Z i ), ΔL i S i Defect type: argmax(p i ).

[0063] As a preferred embodiment of the wind turbine blade internal cavity defect detection system based on image recognition and deep learning described in this invention, the detection system includes: a camera device, a data transmission system, and a monitoring center; the camera device is a high-resolution, low-light, vibration-resistant industrial-grade camera capable of adapting to the complex environmental conditions inside the wind turbine blade; the camera device has an automatic start / stop function and is equipped with a large-capacity battery to ensure that the battery is only charged during the annual wind turbine maintenance period; the data transmission system uses 4G or 5G transmission technology to transmit the image data collected by the camera to the monitoring center in real time, ensuring the stability and reliability of data transmission and avoiding signal interference and data loss; the monitoring center is equipped with high-performance servers and storage devices for receiving, storing, and processing the image data transmitted from the camera; monitoring software is installed to realize image display, video playback, and fault alarm functions. The system is also deployed within the blade internal cavity.

[0064] A computer device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement the steps of a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning.

[0065] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning.

[0066] The beneficial effects of this invention are as follows: By constructing a two-layer generative adversarial network (GAN) model, the first layer of the GAN generates a primary virtual defect image, and the second layer of the GAN generates complex defects such as composite cracks and material degradation based on this. Through multi-layer adversarial training, this invention can not only simulate and generate more complex virtual defects, but also refine the distinction between complex defect images and actual defect images through the second-layer discriminator, ensuring the accuracy of complex defect identification. This effectively solves the problem of not being able to identify complex defects in the prior art, especially the difficult-to-detect defects such as composite cracks and material degradation, and greatly improves the comprehensiveness and accuracy of defect detection.

[0067] This invention trains a convolutional neural network (CNN) by inputting defect images generated by a two-layer generative adversarial network (GAN) and actual defect images into the CNN. The CNN extracts defect features through multiple convolutional layers, which can not only automatically learn the edge and morphological features of defects, but also continuously optimize its detection capabilities through the backpropagation algorithm. The design of the loss function enables the model to effectively balance the recognition accuracy of actual and virtual defects, ensuring that the model maintains a high detection accuracy when facing diverse defects. Compared with existing single detection technologies, the multi-network combination method of this invention significantly improves the efficiency and classification ability of defect detection. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic flowchart of a wind turbine blade internal cavity defect detection method based on image recognition and deep learning, provided as an embodiment of the present invention. Detailed Implementation

[0070] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0071] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0073] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0074] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0075] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integrated connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0076] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning, including:

[0077] S1. The internal cavity of the wind turbine blade is captured from multiple angles and positions using imaging equipment to generate a dataset of images of the internal cavity of the wind turbine blade containing details of the internal cavity.

[0078] S2. Perform noise removal and image enhancement processing on the collected wind turbine blade internal cavity image dataset, and perform edge enhancement and contrast enhancement on key areas;

[0079] S3. Generate primary defect samples using the first-layer generative adversarial network model. The first-layer generator generates virtual defect images of various types, including cracks, pores, and material spalling, based on the physical characteristics of the wind turbine blade's inner cavity. The first-layer discriminator is used to distinguish between the generated virtual defect images and the actual defect images.

[0080] S4. Based on the virtual defect image generated by the first-layer generative adversarial network model, the second-layer generator of the second-layer generative adversarial network model generates a complex defect image including composite cracks and material degradation. The second-layer discriminator performs fine adversarial discrimination on the complex defect image.

[0081] S5. Combine complex defect images generated by a two-layer generative adversarial network model with virtual defect images of actual blades and train a convolutional neural network model. The convolutional neural network model learns the features of various types and complexities of defects and optimizes the ability to identify defects.

[0082] S6. Input the preprocessed wind turbine blade cavity image dataset into the trained convolutional neural network model to automatically identify and locate defects in the blade cavity, including the type, location, size and shape of the defects;

[0083] S7. Generate a defect report based on the defect detection results. The report includes the specific location, size, shape, and type of the defect.

[0084] S8. When new defect types emerge, the two-layer generative adversarial network model and the convolutional neural network model adaptively learn through a feedback mechanism to continuously optimize their ability to identify new defects.

[0085] In this embodiment, S1 includes the following steps:

[0086] S11. Multi-angle data acquisition is performed on the inner cavity of the wind turbine blades using imaging equipment, where the angle range is θ, θ∈[θ]. min ,θ max The step size of the angle θ is Δθ, and multiple images of the blade cavity from different perspectives are generated by adjusting θ.

[0087] S12. Collect images of multiple locations within the inner cavity of the wind turbine blade, with coordinates P(x,y,z). The collection locations include the blade tip, the middle section of the blade, and the blade tail, generating image data of the inner cavity of the blade at different spatial locations.

[0088] S13. Generate a dataset of wind turbine blade internal cavity images based on angle θ and position coordinates P(x,y,z):

[0089] I = {I1, I2, ..., I} n}

[0090] Among them, I n This represents the nth image of the wind turbine blade's internal cavity, captured at a specific angle and position.

[0091] In this embodiment, S2 includes the following steps:

[0092] S21. An adaptive median filtering algorithm is used to remove noise from the acquired wind turbine blade internal cavity image dataset. The filtering window size is W. f The size W of the filtering window is dynamically adjusted based on the uniformity of noise distribution in the image of the wind turbine blade cavity. f Remove random noise while preserving edge details;

[0093] S22. Perform image enhancement processing on the inner cavity of the wind turbine blade, optimize the contrast of key areas in the image, and set the contrast limiting factor to α. c ;

[0094] S23. Perform edge enhancement processing on key areas of the wind turbine blade inner cavity image, extract edge features of the wind turbine blade inner cavity, and set a gradient threshold τ. g ;

[0095] S24. Obtain the image dataset of the wind turbine blade cavity after comprehensive noise removal, contrast enhancement, and edge enhancement:

[0096] I′={I′1,I′2,…,I′ n}

[0097] Among them, I′ n This represents the nth image of the wind turbine blade's internal cavity after preprocessing.

[0098] In this embodiment, S3 includes the following steps:

[0099] S31. Construct a first-layer generative adversarial network model, which includes a first-layer generator G1 and a first-layer discriminator D1. The first-layer generator G1 generates primary virtual defect image data based on the input physical characteristics of the wind turbine blade's internal cavity. The primary virtual defect image data includes cracks, pores, and material spalling. The output of the first-layer generator is the primary virtual defect image data.

[0100]

[0101] in, This is the nth virtual defect image generated;

[0102] S32. The first layer generator G1 generates different types of virtual defects based on the physical characteristic parameters of the wind turbine blade cavity, including thickness t, material density ρ, material uniformity coefficient u, and cavity structure complexity C. Virtual defects of cracks are generated by changes in thickness t and material density ρ, virtual defects of pores are generated based on the complexity of cavity structure C, and virtual defects of material spalling are determined by material uniformity coefficient u.

[0103] S33, The first-layer discriminator D1 is used to receive the virtual defect image data V1 generated by the first-layer generator G1 and the actual acquired defect image R = {R1, R2, ..., R...} n The first-layer discriminator D1 maximizes the log probability function. Distinguish between virtual defect images and actual defect images, and output the judgment result:

[0104]

[0105] in, Let R be the log-probability loss function of the first-layer discriminator D1. i This is the i-th image of the internal cavity defect of the wind turbine blade that was actually acquired. For the i-th virtual defect image generated by the first-layer generator G1, α i The weight coefficients for the i-th image in the wind turbine blade internal cavity image dataset are automatically adjusted based on the defect type in the defect image, with cracks having a weight of α. crack The weight α of stomata pore And the weight α of material peeling peel Set according to the differences in the characteristics of the defects:

[0106]

[0107]

[0108] S34. Optimize the parameters of the first-layer generator G1 and the first-layer discriminator D1 through adversarial training, so that the first-layer discriminator D1 cannot effectively distinguish between virtual defect images and actual defect images. The optimization objective of the first-layer discriminator is:

[0109]

[0110] in, The objective function for generating the adversarial network.

[0111] S4 includes the following steps:

[0112] S41. Construct a second-layer generative adversarial network (GAN) model, which includes a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer GAN model, the second-layer generator G2 further generates complex defect image data according to the input virtual defect image data. Complex defect image data includes composite cracks and material degradation;

[0113] S42, the second-layer generator G2 combines the physical characteristics of composite cracks and material degradation, and generates complex defect images by adjusting physical parameters. The virtual defect image of the composite crack is determined according to the degree of crack overlap γ. crack The generation of virtual defect images of material degradation is based on the material structure change parameter λ. material The generated images of composite cracks and material degradation defects are as follows:

[0114] S43, the second-layer discriminator D2 is used to compare the complex defect image V2 generated by the second-layer generator G2 with the actually acquired complex defect image. For refined adversarial discrimination, the loss function of the second-layer discriminator is:

[0115]

[0116] in, Let log be the probability loss function of the second-layer discriminator. Let i be the i-th actually acquired complex defect image. For the i-th complex defect image generated by the second-layer generator, β i β represents the weighting coefficient for complex defects in the image, and β represents the weighting coefficient for overlapping cracks. crack The weighting factor for material degradation is β. material The calculation formulas are as follows:

[0117]

[0118]

[0119] S44. Optimize the parameters of the second-layer generator G2 and the second-layer discriminator D2 through adversarial training until the second-layer discriminator D2 can no longer distinguish between complex defect images and actual defect images:

[0120]

[0121] in, The objective function for the second layer of the generative adversarial network.

[0122] In this embodiment, S5 includes the following steps:

[0123] S51. Combine the complex defect image data V2 generated by the two-layer generative adversarial network convolutional neural network model with the actual collected virtual defect image data of the wind turbine blade cavity to construct the input dataset D for training the convolutional neural network.

[0124] S52. Input the input dataset D into the convolutional neural network for training. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers.

[0125] S53. Convolutional neural networks are trained using the backpropagation algorithm, and the loss function... Used to measure the accuracy of convolutional neural network models in identifying defect features:

[0126]

[0127] in, The mean square error between the actual defect image and its predicted value. γ represents the mean square error between the complex defect image and its predicted value. i These are the weighting coefficients for the actual defect image and the virtual defect image;

[0128] S54. The convolutional neural network model gradually optimizes the weight parameter W through multiple iterations. c and bias term b c This allows the convolutional neural network model to learn different types and complexities of defect features, optimize its recognition capabilities, and ultimately output a well-trained convolutional neural network model.

[0129] In this embodiment, S7 includes the following steps:

[0130] S71. Generate a defect report based on the detected defect image data and virtual defect image data. The report records the specific location, size, shape and type of each detected defect.

[0131] S72, The defect location is determined by the coordinates (x, y) in the image coordinate system. i ,y i ) and image depth z iThis means that, based on the actual physical dimensions of the wind turbine blades, it is converted into actual spatial position coordinates (X). i ,Y i Z i ), where i is the i-th defect;

[0132] S73, Defect size ΔL i It is calculated by the pixel distance in the image and converted by combining it with the actual physical size of the pixels;

[0133] S74. The shape of the defect is determined by analyzing the geometric features of the defect edge. Shape parameters include crack length, pore diameter, or area of ​​material spalling.

[0134] S75. The defect type is determined by the classification results of the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, which outputs the probability distribution of each defect belonging to crack, porosity, or material spalling:

[0135] p i =[p crack ,p pore ,p peel ]

[0136] S76. The generated defect report includes the specific location, size, shape, and type of each defect, and the report content is output in the following format:

[0137] Defect number, (X) i ,Y i Z i ), ΔL i S i Defect type: argmax(p i ).

[0138] Example 2 is an embodiment of the present invention, which provides a method for detecting defects in the internal cavity of wind turbine blades based on image recognition and deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0139] The specific implementation of the wind turbine blade internal cavity defect detection will be verified based on a real case process. On August 15, 2023, a wind power plant carried out a batch of wind turbine blades for routine maintenance and inspection. In this maintenance task, the operation and maintenance team selected the main blades of 10 wind turbines with an average length of 75 meters. The detection objective was to find and repair possible internal cavity defects such as cracks, pores and material spalling.

[0140] At 9 a.m., the technical team began to photograph the inner cavity of the wind turbine blades using high-resolution camera equipment mounted on an autonomous mobile robot. The camera equipment comprehensively scanned various areas of the inner cavity from different angles, recording the structural features and position coordinates of the wind turbine blades and generating an image dataset of the inner cavity of each blade. By automatically adjusting the exposure time and aperture size of the equipment, the system was able to obtain clear images in both bright and shadowy areas.

[0141] Testing process:

[0142] Time: 10:30 AM, August 15, 2023;

[0143] Blade number: WFLP-001;

[0144] Using image recognition and deep learning methods, the system detected three areas of material spalling located in the inner cavity at the front end of the blade.

[0145] Defect number: #0001, Defect location: front end of the inner cavity (X=4.5m, Y=0.9m, Z=0.5m);

[0146] Defect size: 15mm x 30mm; Defect type: material peeling.

[0147] Time: 11:15 AM, August 15, 2023;

[0148] Blade number: WFLP-002;

[0149] The system detected a complex crack, with the cracks intersecting to form a complex damaged area, which was identified as a high-risk defect.

[0150] Defect number: #0002, Defect location: Middle inner cavity (X=12.3m, Y=2.1m, Z=0.8m);

[0151] Defect dimensions: Crack length 35mm, width 5mm; Defect type: Complex crack.

[0152] The system generates a defect report and sends it to the maintenance team's terminal in real time, recommending immediate shutdown for repair.

[0153] In this inspection task, the method of this invention can not only detect conventional cracks and material spalling, but also identify complex defects that are difficult to detect by traditional methods. Compared with traditional ultrasonic testing, the deep learning model can automatically identify and locate defects with complex morphologies, such as composite cracks and material deterioration. The following are comparative data between the two methods:

[0154] Table 1 Comparison data between the present invention and traditional ultrasonic testing

[0155]

[0156] As shown in Table 1 of Example 1, the traditional method takes 5 hours to inspect each blade, while the method of this invention, through the combination of automatic image acquisition and a deep learning model, reduces the inspection time to 2 hours. Since the system can quickly acquire data from multiple angles and analyze defects in real time without manual intervention, the entire inspection process is more efficient. Traditional ultrasonic methods perform poorly in identifying complex defects, with a high false negative rate. This invention generates complex defect samples through a two-layer generative adversarial network and trains them using actual blade images, greatly improving the accuracy of defect identification. Among the 66 defects detected, there were several complex cracks and material deterioration that were difficult to detect using traditional methods. After detecting a defect, the system of this invention can instantly generate a report containing the defect location, size, shape, and type, and send it to the maintenance team's terminal in real time, shortening the time from detection results to maintenance decisions. This automated process makes defect warnings faster, thereby reducing the potential risk of blade failure.

[0157] Table 2 Comparison of wind turbine blade test data

[0158]

[0159] By implementing the method of this invention, the efficiency and accuracy of detecting internal defects in wind turbine blades have been significantly improved. In this inspection task, the maintenance team inspected the main blades of 10 wind turbines and found several complex defects that were difficult to detect using traditional methods. The system can automatically generate defect reports and push them to relevant personnel in real time, thereby greatly shortening the inspection time and reducing maintenance costs.

[0160] The method of this invention is not only applicable to the daily maintenance of wind turbine blades, but also to the detection and monitoring of large-scale wind farms, ensuring the safety and reliability of wind turbine blades during long-term operation. Compared with traditional methods, this invention not only improves the detection speed, but also reduces the missed detection rate, and performs excellently in the identification of complex defects, making it highly valuable for practical applications.

[0161] This invention constructs a two-layer generative adversarial network (GAN) model. The first layer of the GAN generates a primary virtual defect image, while the second layer generates complex defects such as composite cracks and material degradation. Through multi-layer adversarial training, this invention can not only simulate and generate more complex virtual defects, but also use a second-layer discriminator to finely distinguish between complex defect images and actual defect images, ensuring the accuracy of complex defect identification. This effectively solves the problem of not being able to identify complex defects in existing technologies, especially composite cracks and material degradation defects that are difficult to detect, greatly improving the comprehensiveness and accuracy of defect detection.

[0162] This invention trains a convolutional neural network (CNN) by inputting defect images generated by a two-layer generative adversarial network (GAN) and actual defect images into the CNN. The CNN extracts defect features through multiple convolutional layers, which can not only automatically learn the edge and morphological features of defects, but also continuously optimize its detection capabilities through the backpropagation algorithm. The design of the loss function enables the model to effectively balance the recognition accuracy of actual and virtual defects, ensuring that the model maintains a high detection accuracy when facing diverse defects. Compared with existing single detection technologies, the multi-network combination method of this invention significantly improves the efficiency and classification ability of defect detection.

[0163] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0164] Example 3

[0165] A third embodiment of the present invention provides a wind turbine blade internal cavity defect detection system based on image recognition and deep learning, comprising: the detection system includes a camera device, a data transmission system, and a monitoring center;

[0166] The camera device is a high-resolution, low-light, and vibration-resistant industrial-grade camera, which can adapt to the complex environmental conditions inside the wind turbine blades. The camera device also has an automatic start-stop function and is equipped with a large-capacity battery to ensure that the battery is charged only during the annual wind turbine maintenance period.

[0167] The data transmission system uses 4G or 5G transmission technology to transmit the image data captured by the camera to the monitoring center in real time, ensuring the stability and reliability of data transmission and avoiding signal interference and data loss.

[0168] The monitoring center is equipped with high-performance servers and storage devices to receive, store, and process image data transmitted from cameras; it also installs monitoring software to enable image display, video playback, and fault alarm functions. Additionally, it incorporates internal blade cavities.

[0169] The camera should be installed in a suitable location inside the wind turbine blades to ensure comprehensive monitoring of critical parts of the blades, such as the blade root, main beam, and web. The installation method should be secure and reliable to prevent loosening or detachment during wind turbine operation. Simultaneously, the installation should ensure unobstructed signal transmission from the camera, preventing interference from external factors.

[0170] During maintenance and management, cameras, data transmission equipment, and the monitoring center must be inspected and maintained regularly to ensure the system's normal operation. This includes checking the cleanliness of camera lenses, the clarity of image quality, and the stability of data transmission. System faults, such as camera damage or data transmission interruptions, should be addressed promptly. A troubleshooting mechanism should be established to ensure the system is restored to normal operation as quickly as possible. Simultaneously, the monitoring software should be upgraded regularly to improve system performance and functionality. Image recognition algorithms and fault diagnosis models should be updated promptly to adapt to constantly changing monitoring needs.

[0171] By implementing a camera monitoring system inside wind turbine blades, real-time monitoring and early warning of the internal condition of the blades can be achieved, improving the reliability and availability of wind turbines and reducing maintenance costs and downtime. At the same time, this system also provides strong technical support for the intelligent operation and maintenance of wind turbines, contributing to the sustainable development of the wind power industry.

[0172] Example 4

[0173] The fourth embodiment of the present invention differs from the first three embodiments in that:

[0174] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0179] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting internal defects in wind turbine blades based on image recognition and deep learning, characterized in that: include, By acquiring images of the inner cavity of the wind turbine blade from multiple angles and positions using imaging equipment, a dataset of images of the inner cavity of the wind turbine blade containing details of the inner cavity is generated. The collected images of the wind turbine blade interior were processed for noise removal and image enhancement, and edge enhancement and contrast enhancement were performed on key areas. A primary defect sample is generated using a first-layer generative adversarial network model. The first-layer generator generates virtual defect images of various types, including cracks, pores, and material spalling, based on the physical characteristics of the wind turbine blade's inner cavity. The first-layer discriminator is used to distinguish between the generated virtual defect images and the actual defect images. Based on the virtual defect image generated by the first-layer generative adversarial network model, the second-layer generator of the second-layer generative adversarial network model generates complex defect images including composite cracks and material degradation. The second-layer discriminator performs fine adversarial discrimination on the complex defect images. The convolutional neural network model is trained by combining complex defect images generated by a two-layer generative adversarial network model and virtual defect images of actual blades. The convolutional neural network model learns the features of various types and complexities of defects, thereby optimizing the defect recognition ability. The pre-processed image dataset of the wind turbine blade cavity is input into the trained convolutional neural network model to automatically identify and locate defects in the blade cavity, including the type, location, size and shape of the defects. A defect report is generated based on the defect detection results. The report includes the specific location, size, shape, and type of the defect. When new defect types emerge, the two-layer generative adversarial network model and the convolutional neural network model adaptively learn through a feedback mechanism to continuously optimize their ability to identify new defects. The virtual defect image includes constructing a first-layer generative adversarial network model, which includes a first-layer generator G1 and a first-layer discriminator D1. The first-layer generator G1 generates primary virtual defect image data based on the input physical characteristics of the wind turbine blade's internal cavity. The primary virtual defect image data includes cracks, pores, and material spalling. The output of the first-layer generator is the primary virtual defect image data. in, This is the nth virtual defect image generated; The first layer generator G1 generates different types of virtual defects based on the physical characteristic parameters of the wind turbine blade cavity, including thickness t, material density ρ, material uniformity coefficient u, and cavity structure complexity C. Virtual defects of cracks are generated by changes in thickness t and material density ρ, virtual defects of pores are generated based on the complexity of cavity structure C, and virtual defects of material spalling are determined by material uniformity coefficient u. The first-layer discriminator D1 receives the virtual defect image data V1 generated by the first-layer generator G1 and the actually acquired defect images R = {R1, R2, ..., R...} n The first-layer discriminator D1 maximizes the log probability function. Distinguish between virtual defect images and actual defect images, and output the judgment result: in, Let R be the log-probability loss function of the first-layer discriminator D1. i This is the i-th image of the internal cavity defect of the wind turbine blade that was actually acquired. For the i-th virtual defect image generated by the first-layer generator G1, α i The weight coefficients for the i-th image in the wind turbine blade internal cavity image dataset are automatically adjusted based on the defect type in the defect image, with cracks having a weight of α. crack The weight α of stomata pore And the weight α of material peeling peel Set according to the differences in the characteristics of the defects: By optimizing the parameters of the first-layer generator G1 and the first-layer discriminator D1 through adversarial training, the first-layer discriminator D1 is rendered unable to effectively distinguish between virtual defect images and actual defect images. The optimization objective of the first-layer discriminator is: in, The objective function for generating the adversarial network; The complex defect image involves constructing a second-layer generative adversarial network (GAN) model, which includes a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer GAN model, the second-layer generator G2 further generates complex defect image data according to the input virtual defect image data. Complex defect image data includes composite cracks and material degradation; The second-layer generator G2 combines the physical characteristics of composite cracks and material degradation, and generates complex defect images by adjusting physical parameters. The virtual defect image of the composite crack is determined based on the degree of crack overlap γ. crack The generation of virtual defect images of material degradation is based on the material structure change parameter λ. material The generated images of composite cracks and material degradation defects are as follows: The second-layer discriminator D2 is used to compare the complex defect image V2 generated by the second-layer generator G2 with the actually acquired complex defect image. For refined adversarial discrimination, the loss function of the second-layer discriminator is: in, Let log be the probability loss function of the second-layer discriminator. Let i be the i-th actually acquired complex defect image. For the i-th complex defect image generated by the second-layer generator, β i β represents the weighting coefficient for complex defects in the image, and β represents the weighting coefficient for overlapping cracks. crack The weighting factor for material degradation is β. material The calculation formulas are as follows: The parameters of the second-layer generator G2 and the second-layer discriminator D2 are optimized through adversarial training until the second-layer discriminator D2 can no longer distinguish between complex defect images and actual defect images. in, The objective function for the second layer of the generative adversarial network.

2. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning as described in claim 1, characterized in that: The process of generating a dataset of wind turbine blade internal cavity images containing details of the wind turbine blade's internal cavity includes acquiring images of the wind turbine blade's internal cavity from multiple angles using an imaging device, where the angle range is θ, θ∈θ. min ,θ max The step size of the angle θ is Δθ, and multiple images of the blade cavity from different perspectives are generated by adjusting θ. Images are acquired at multiple locations within the inner cavity of the wind turbine blade, with coordinates P(x,y,z). The acquisition locations include the blade tip, the middle section of the blade, and the blade tail, generating image data of the inner cavity of the blade at different spatial locations. Generate a dataset of wind turbine blade internal cavity images based on angle θ and position coordinates P(x,y,z): I={I1,I2,,I n } Among them, I n This represents the nth image of the wind turbine blade's internal cavity, captured at a specific angle and position.

3. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning as described in claim 2, characterized in that: The noise removal and image enhancement processing includes using an adaptive median filtering algorithm to remove noise from the acquired wind turbine blade internal cavity image dataset, with a filtering window size of W. f The size W of the filtering window is dynamically adjusted based on the uniformity of noise distribution in the image of the wind turbine blade cavity. f Remove random noise while preserving edge details; Image enhancement processing was performed on the internal cavity of the wind turbine blades to optimize the contrast of key areas in the image, with a contrast limiting factor set to α. c ; Edge enhancement processing is performed on key regions of the wind turbine blade internal cavity image to extract edge features of the wind turbine blade internal cavity, and a gradient threshold τ is set. g ; We obtained a dataset of wind turbine blade internal cavity images after comprehensive noise removal, contrast enhancement, and edge enhancement: I′={I′1,I′2,…,I′ n } Among them, I′ n This represents the nth image of the wind turbine blade's internal cavity after preprocessing.

4. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning as described in claim 3, characterized in that: The training using a convolutional neural network model includes, The complex defect image data V2 generated by the two-layer generative adversarial network convolutional neural network model and the virtual defect image data of the wind turbine blade cavity actually collected are combined to construct the input dataset D for training the convolutional neural network. The input dataset D is fed into a convolutional neural network for training. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers. Convolutional neural networks are trained using the backpropagation algorithm, and the loss function is... Used to measure the accuracy of convolutional neural network models in identifying defect features: in, The mean square error between the actual defect image and its predicted value. γ represents the mean square error between the complex defect image and its predicted value. i These are the weighting coefficients for the actual defect image and the virtual defect image; The convolutional neural network model gradually optimizes the weight parameters W through multiple iterations. c and bias term b c This allows the convolutional neural network model to learn different types and complexities of defect features, optimize its recognition capabilities, and ultimately output a well-trained convolutional neural network model.

5. The method for detecting internal defects in wind turbine blades based on image recognition and deep learning as described in claim 4, characterized in that: The generation of the defect report includes generating a defect report based on the detected defect image data and virtual defect image data. The report records the specific location, size, shape and type of each detected defect. The location of the defect is determined by its coordinates (x, y) in the image coordinate system. i ,y i ) and image depth z i This means that, based on the actual physical dimensions of the wind turbine blades, it is converted into actual spatial position coordinates (X). i ,Y i Z i ), where i is the i-th defect; Defect size ΔL i It is calculated by the pixel distance in the image and converted by combining it with the actual physical size of the pixels; The shape of a defect is determined by analyzing the geometric features of its edges. Shape parameters include crack length, pore diameter, or area of ​​material spalling. The defect type is determined by the classification results of the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, which outputs the probability distribution of each defect belonging to crack, porosity, or material spalling. p i =[p crack ,p pore ,p peel ] The generated defect report includes the specific location, size, shape, and type of each defect, and the report content is output in the following format: Defect number, (X) i ,Y i Z i ), ΔL i S i Defect type: argmax(p i ).

6. A system employing the image recognition and deep learning-based defect detection method for wind turbine blades as described in any one of claims 1 to 5, characterized in that: The detection system includes camera equipment, data transmission system, and monitoring center; The camera device is a high-resolution, low-light, and vibration-resistant industrial-grade camera, which can adapt to the complex environmental conditions inside the wind turbine blades. The camera device also has an automatic start-stop function and is equipped with a large-capacity battery to ensure that the battery is charged only during the annual wind turbine maintenance period. The data transmission system uses 4G or 5G transmission technology to transmit the image data captured by the camera to the monitoring center in real time, ensuring the stability and reliability of data transmission and avoiding signal interference and data loss. The monitoring center is equipped with high-performance servers and storage devices to receive, store, and process image data transmitted from cameras; it also installs monitoring software to enable image display, video playback, and fault alarm functions, and deploys the blade inner cavity.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images