Fan blade inner cavity defect detection method based on image recognition and deep learning

Abstract

The invention discloses a fan blade inner cavity defect detection method and system based on image recognition and deep learning. The method comprises the following steps: performing multi-angle and multi-position image acquisition on an inner cavity of a fan blade through imaging equipment; the collected fan blade inner cavity image data set is preprocessed; generating a primary defect sample by using the first-layer generative adversarial network model; generating a complex defect image by using the second-layer generative adversarial network model; a convolutional neural network model is used for training; inputting the preprocessed fan blade inner cavity image data set into the trained convolutional neural network model; generating a defect report based on a defect detection result; when a new defect type appears, the recognition capability of the new defect is continuously optimized. Compared with the existing single detection technology, the multi-network combination method disclosed by the invention has the advantage that the defect detection efficiency and the classification capability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of inner cavities of fan blades, and in particular to a method for detecting defects in the inner cavities of fan blades based on image recognition and deep learning. Background Art

[0002] In the prior art, defect detection of the inner cavity of wind turbine blades mainly relies on manual inspection and conventional physical inspection methods. Manual inspection methods usually involve professional inspectors using flaw detection equipment or manually inspecting the external and internal structures of wind turbine blades to find common defects such as cracks, pores, and material peeling. However, manual inspection is inefficient and relies on the experience and operational proficiency of inspectors, making it difficult to ensure full coverage of the entire blade cavity. In addition, manual inspection has large errors and is prone to missed detection and misjudgment. Especially when faced with complex and minor defects, its detection accuracy is poor, making it difficult to meet the requirements of the modern wind power industry for efficient and accurate detection.

[0003] On the other hand, existing physical detection methods usually include ultrasonic, X-ray or infrared imaging technology. Although they can improve the accuracy and coverage of defect detection to a certain extent, their equipment costs are high, the operation is complicated, and it takes a lot of time to fully detect the inner cavity of large wind turbine blades. More importantly, physical detection technology has the limitation of difficulty in identifying composite cracks and material deterioration defects. It can often only detect specific types of defects and cannot cope with diverse and complex defect characteristics.

[0004] The defects of existing technologies are mainly concentrated in the following aspects: First, manual inspection methods are not only time-consuming and costly, but also difficult to ensure detection accuracy when facing complex or minor defects; second, although traditional physical inspection equipment has higher accuracy, the equipment is expensive and complicated to operate, making it difficult to achieve large-scale application; third, existing inspection technologies can often only identify a single type of defect, and show great limitations when facing compound defects or complex defects. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a method for detecting defects in the inner cavity of a wind turbine blade based on image recognition and deep learning. Compared with the existing single detection technology, the multiple network combination method of the present invention significantly improves the efficiency and classification ability of defect detection.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning, comprising:

[0008] Capturing images of the inner cavity of the fan blade at multiple angles and multiple positions by using an imaging device to generate a fan blade inner cavity image data set containing details of the inner cavity of the fan blade;

[0009] Perform noise removal and image enhancement on the collected wind turbine blade inner cavity image dataset, and perform edge enhancement and contrast enhancement on key areas;

[0010] The first-layer generative adversarial network model is used to generate primary defect samples. The first-layer generator generates virtual defect images of various types, including cracks, pores, and material peeling, based on the physical characteristics of the inner cavity of the wind turbine blade. The first-layer discriminator is used to distinguish the generated virtual defect images from the actual defect images.

[0011] Based on the virtual defect image generated by the first-layer generative adversarial network model, the second-layer generator of the second-layer generative adversarial network model generates a complex defect image including compound cracks and material deterioration, and the second-layer discriminator performs fine adversarial discrimination on the complex defect image;

[0012] The convolutional neural network model is used to train the complex defect images generated by the two-layer generative adversarial network model and the actual blade virtual defect images. The convolutional neural network model learns the defect features of various types and complexities and optimizes the defect recognition capability.

[0013] The pre-processed wind turbine blade cavity image dataset is input into the trained convolutional neural network model to automatically identify and locate defects in the blade cavity, including the type, location, size and shape of the defects.

[0014] Generate a defect report based on the defect detection results, including the specific location, size, shape and type of the defect;

[0015] When new types of defects appear, the two-layer generative adversarial network model and convolutional neural network model perform adaptive learning through a feedback mechanism to continuously optimize the ability to recognize new types of defects.

[0016] As a preferred embodiment of the method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning described in the present invention, the generating of a fan blade inner cavity image data set containing the details of the inner cavity of the fan blade comprises: performing multi-angle acquisition of the inner cavity of the fan blade by an imaging device, wherein the angle range is θ, θ∈[θ min ,θ max ], the change step of angle θ is Δθ, and the blade cavity images under multiple viewing angles are generated by adjusting θ;

[0017] Capture images at multiple locations of the inner cavity of the fan blade, with the position coordinates being P(x, y, z). The acquisition locations include the front end, middle section and tail end of the blade, and generate blade inner cavity image data at different spatial locations.

[0018] Generate a fan blade cavity image dataset of the fan blade cavity details based on the angle θ and position coordinates P(x,y,z):

[0019] I={I1,I2,,I n}

[0020] Among them, I n Indicates the nth wind turbine blade inner cavity image acquired at a specific angle and position.

[0021] As a preferred solution of the fan blade inner cavity defect detection method based on image recognition and deep learning described in the present invention, the noise removal and image enhancement processing includes using an adaptive median filtering algorithm to perform noise removal processing on the collected fan blade inner cavity image data set, and the filter window size is W f , dynamically adjust the filter window size W according to the uniformity of noise distribution in the fan blade cavity image f Remove random noise and preserve edge details;

[0022] Perform fan blade inner cavity image enhancement processing, optimize the contrast of key areas in the fan blade inner cavity image, and set the contrast limit factor to α c ;

[0023] Perform edge enhancement on the key areas of the fan blade inner cavity image, extract the edge features of the fan blade inner cavity, and set the gradient threshold τ g ;

[0024] The wind turbine blade inner cavity image dataset is obtained after comprehensive noise removal, contrast enhancement and edge enhancement:

[0025] I′={I′1,I′2,…,I′ n}

[0026] Among them, I′ n Represents the nth fan blade inner cavity image after preprocessing.

[0027] As a preferred solution of the method for detecting inner cavity defects of a fan blade based on image recognition and deep learning according to the present invention, the virtual defect image includes: constructing a first-layer generative adversarial network model, which includes a first-layer generator G1 and a first-layer discriminator D1, and the first-layer generator G1 generates primary virtual defect image data according to the input physical characteristics of the inner cavity of the fan blade, and the primary virtual defect image data includes cracks, pores and material peeling, and the output of the first-layer generator is the primary virtual defect image data:

[0028]

[0029] in, is the nth virtual defect image generated;

[0030] The first layer generator G1 generates different types of virtual defects by adjusting parameters according to the physical characteristic parameters of the inner cavity of the fan blade, including thickness t, material density ρ, material uniformity coefficient u and inner cavity structure complexity C. The virtual defect of crack is generated by the change of thickness t and material density ρ, the virtual defect of pore is generated according to the complexity of the inner cavity structure complexity C, and the virtual defect of material peeling is determined by the material uniformity coefficient u;

[0031] The first layer discriminator D1 is used to receive the virtual defect image data V1 generated by the first layer generator G1 and the actual acquired defect image R = {R1, R2,, R n}, the first layer discriminator D1 maximizes the logarithmic probability function Distinguish virtual defect images from actual defect images and output the judgment results:

[0032]

[0033] in, is the logarithmic probability loss function of the first layer discriminator D1, R i is the i-th actually collected image of the inner cavity defect of the fan blade, is the i-th virtual defect image generated by the first layer generator G1, α i is the weight coefficient of the i-th image in the fan blade inner cavity image dataset, which is automatically adjusted based on the defect type in the defect image. The weight α of the crack crack , the weight of pores α pore and the material peeling weight α peel Set according to the characteristic differences of defects:

[0034]

[0035] The parameters of the first-layer generator G1 and the first-layer discriminator D1 are optimized through adversarial training, so that the first-layer discriminator D1 cannot effectively distinguish between virtual defect images and actual defect images. The optimization goal of the first-layer discriminator is:

[0036]

[0037] in, is the objective function of the generative adversarial network.

[0038] As a preferred solution of the fan blade inner cavity defect detection method based on image recognition and deep learning described in the present invention, wherein: the complex defect image includes:

[0039] Construct a second-layer generative adversarial network model, which includes a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer generative adversarial network model, the second-layer generator G2 further generates complex defect image data based on the input virtual defect image data. Complex defect image data includes compound cracks and material deterioration;

[0040] The second layer generator G2 combines the physical characteristics of composite cracks and material deterioration, and generates complex defect images by adjusting physical parameters. The virtual defect image of the composite crack is calculated according to the degree of crack overlap γ crack Generate virtual defect images of material deterioration according to the material structure change parameter λ material The generated composite crack and material deterioration defect images are

[0041] The second layer discriminator D2 is used to compare the complex defect image V2 generated by the second layer generator G2 with the actual collected complex defect image For fine adversarial discrimination, the loss function of the second-layer discriminator is:

[0042]

[0043] in, is the logarithmic probability loss function of the second layer discriminator, is the i-th complex defect image actually collected, is the i-th complex defect image generated by the second layer generator, β i is the weight coefficient of complex defects in the image, and the weight coefficient of crack overlap is β crack , the weight coefficient of material deterioration is β material , and the calculation formulas are:

[0044]

[0045] The parameters of the second-layer generator G2 and the second-layer discriminator D2 are optimized through adversarial training until the second-layer discriminator D2 cannot distinguish between complex defect images and actual defect images:

[0046]

[0047] in, Objective function for the second layer of the generative adversarial network.

[0048] As a preferred solution of the fan blade inner cavity defect detection method based on image recognition and deep learning described in the present invention, the training using the convolutional neural network model includes:

[0049] The complex defect image data V2 generated by the double-layer generative adversarial network convolutional neural network model is combined with the actual collected virtual defect image data of the inner cavity of the fan blade to construct the input data set D for convolutional neural network training;

[0050] Input the input data set D into the convolutional neural network for training. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers;

[0051] Convolutional neural networks are trained using the error back propagation algorithm, and the loss function Used to measure the recognition accuracy of the convolutional neural network model for defect features:

[0052]

[0053] in, is the mean square error between the actual defect image and its predicted value, is the mean square error between the complex defect image and its predicted value, γ i is the weight coefficient of the actual defect image and the virtual defect image;

[0054] The convolutional neural network model gradually optimizes the weight parameter W through multiple iterations. c and the bias term b c , so that the convolutional neural network model learns defect features of different types and complexities, optimizes recognition capabilities, and finally outputs a trained convolutional neural network model.

[0055] As a preferred solution of the method for detecting inner cavity defects of a fan blade based on image recognition and deep learning according to the present invention, wherein: generating a defect report includes generating a defect report based on detected defect image data and virtual defect image data, and recording the specific position, size, shape and type of each detected defect in the report;

[0056] The defect position is represented by the coordinate value (xi ,y i ) and image depth z i Indicates that, combined with the actual physical size of the wind turbine blade, it is converted into the actual spatial position coordinates (X i ,Y i ,Z i ), where i is the i-th defect;

[0057] Defect size ΔL i It is calculated by the pixel distance in the image and converted based on the actual physical size of the pixel;

[0058] The defect shape is determined by analyzing the geometric characteristics of the defect edge. The shape parameters include crack length, pore diameter or material peeling area.

[0059] The defect type is determined by the classification results of the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, which outputs the probability distribution of each defect belonging to cracks, pores or material peeling:

[0060] p i =[p crack ,p pore ,p peel ]

[0061] The generated defect report contains the specific location, size, shape and type of each defect. The report content is output in the following format:

[0062] Defect number, (X i ,Y i ,Z i ), ΔL i , S i , defect type: argmax(p i ).

[0063] As a preferred solution of the wind turbine blade cavity defect detection system based on image recognition and deep learning described in the present invention, the detection system includes a camera device, a data transmission system, and a monitoring center; the camera device uses a high-resolution, low-illumination, and vibration-resistant industrial-grade camera to adapt to the complex environmental conditions inside the wind turbine blade; and the camera device has an automatic start-stop function and is equipped with a large-capacity battery to ensure that the battery is only charged during the wind turbine regular inspection each year; the data transmission system uses 4G or 5G transmission technology to transmit the image data collected by the camera to the monitoring center in real time to ensure the stability and reliability of data transmission and avoid signal interference and data loss; the monitoring center is equipped with a high-performance server and storage device for receiving, storing and processing the image data transmitted by the camera; and monitoring software is installed to realize image display, video playback, and fault alarm functions. And deploy the blade cavity.

[0064] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, steps of a method for detecting defects in the inner cavity of a wind turbine blade are implemented based on image recognition and deep learning.

[0065] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of a method for detecting defects in the inner cavity of a wind turbine blade based on image recognition and deep learning are implemented.

[0066] The beneficial effects of the present invention are as follows: the present invention constructs a two-layer generative adversarial network model, in which the first-layer generative adversarial network is responsible for generating primary virtual defect images, and the second-layer generative adversarial network generates complex defects such as composite cracks and material deterioration on this basis. Through multi-level adversarial training, the present invention can not only simulate and generate more complex virtual defects, but also finely distinguish complex defect images from actual defect images through the second-layer discriminator, thereby ensuring the accuracy of complex defect recognition, and effectively solving the problem that complex defects cannot be recognized in the prior art, especially the defects that are difficult to detect such as composite cracks and material deterioration, thereby greatly improving the comprehensiveness and accuracy of defect detection.

[0067] The present invention inputs the defect images generated by the double-layer generative adversarial network and the actual defect images into the convolutional neural network for training. The convolutional neural network extracts defect features through multiple convolutional layers and can not only automatically learn the edge features and morphological features of the defects, but also continuously optimize its detection capability through the error back propagation algorithm. The design of the loss function enables the model to effectively balance the recognition accuracy of actual defects and virtual defects, ensuring that the model still maintains a high detection accuracy when facing diversified defects. Compared with the existing single detection technology, the multiple network combination method of the present invention significantly improves the efficiency and classification capability of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0069] Figure 1 A schematic flow chart of a method for detecting defects in the inner cavity of a wind turbine blade based on image recognition and deep learning provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0071] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0073] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0074] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0075] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0076] Example 1, reference Figure 1 , which is the first embodiment of the present invention, provides a method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning, comprising:

[0077] S1. Capturing images of the inner cavity of the fan blade at multiple angles and positions by using an imaging device to generate a fan blade inner cavity image dataset containing details of the inner cavity of the fan blade;

[0078] S2, performing noise removal and image enhancement processing on the collected wind turbine blade inner cavity image data set, and performing edge enhancement and contrast enhancement on key areas;

[0079] S3. Generate primary defect samples using the first-layer generative adversarial network model. The first-layer generator generates virtual defect images of various types including cracks, pores, and material peeling according to the physical characteristics of the inner cavity of the wind turbine blade. The first-layer discriminator is used to distinguish the generated virtual defect images from the actual defect images.

[0080] S4, based on the virtual defect image generated by the first-layer generative adversarial network model, a second-layer generator of the second-layer generative adversarial network model is used to generate a complex defect image including compound cracks and material deterioration, and the second-layer discriminator performs fine adversarial discrimination on the complex defect image;

[0081] S5. The complex defect images generated by the two-layer generative adversarial network model and the actual blade virtual defect images are combined and trained using a convolutional neural network model. The convolutional neural network model learns defect features of various types and complexities and optimizes the ability to identify defects.

[0082] S6, inputting the preprocessed wind turbine blade inner cavity image data set into the trained convolutional neural network model to automatically identify and locate defects in the blade inner cavity, including the type, location, size and shape of the defects;

[0083] S7. Generate a defect report based on the defect detection results, the report content includes the specific location, size, shape and type of the defect;

[0084] S8. When new defect types appear, the two-layer generative adversarial network model and the convolutional neural network model perform adaptive learning through a feedback mechanism to continuously optimize the ability to recognize new defects.

[0085] In this implementation, S1 includes the following steps:

[0086] S11. Use imaging equipment to collect the inner cavity of the fan blade at multiple angles, where the angle range is θ, θ∈[θ min ,θ max ], the change step of angle θ is Δθ, and the blade cavity images under multiple viewing angles are generated by adjusting θ;

[0087] S12, collecting images at multiple positions of the inner cavity of the fan blade, where the position coordinates are P(x, y, z), and the collection positions include the front end, middle section and tail end of the blade, and generating blade inner cavity image data at different spatial positions;

[0088] S13, generating a fan blade inner cavity image dataset of the fan blade inner cavity details based on the angle θ and the position coordinates P (x, y, z):

[0089] I={I1,I2,,I n}

[0090] Among them, I n Indicates the nth wind turbine blade inner cavity image acquired at a specific angle and position.

[0091] In this implementation, S2 includes the following steps:

[0092] S21, using the adaptive median filter algorithm to remove noise from the collected fan blade inner cavity image data set, the filter window size is W f , dynamically adjust the filter window size W according to the uniformity of noise distribution in the fan blade cavity image f Remove random noise and preserve edge details;

[0093] S22, perform fan blade inner cavity image enhancement processing, optimize the contrast of key areas in the fan blade inner cavity image, and set the contrast limit factor to α c ;

[0094] S23, edge enhancement processing is performed on the key area of ​​the fan blade inner cavity image to extract the edge features of the fan blade inner cavity and set the gradient threshold τ g ;

[0095] S24, obtain the wind turbine blade inner cavity image dataset after comprehensive noise removal, contrast enhancement and edge enhancement:

[0096] I′={I′1,I′2,…,I′ n}

[0097] Among them, I′ n Represents the nth fan blade inner cavity image after preprocessing.

[0098] In this implementation, S3 includes the following steps:

[0099] S31, constructing a first-layer generative adversarial network model, which includes a first-layer generator G1 and a first-layer discriminator D1. The first-layer generator G1 generates primary virtual defect image data according to the input physical characteristics of the inner cavity of the wind turbine blade. The primary virtual defect image data includes cracks, pores and material peeling. The output of the first-layer generator is the primary virtual defect image data:

[0100]

[0101] in, is the nth virtual defect image generated;

[0102] S32, the first layer generator G1 generates different types of virtual defects by adjusting parameters according to the physical characteristic parameters of the inner cavity of the fan blade, including thickness t, material density ρ, material uniformity coefficient u and inner cavity structure complexity C. The virtual defect of crack is generated by the change of thickness t and material density ρ, the virtual defect of pore is generated according to the complexity of the inner cavity structure complexity C, and the virtual defect of material peeling is determined by the material uniformity coefficient u;

[0103] S33, the first layer discriminator D1 is used to receive the virtual defect image data V1 generated by the first layer generator G1 and the actual collected defect image R = {R1, R2,, R n}, the first layer discriminator D1 maximizes the logarithmic probability function Distinguish virtual defect images from actual defect images and output the judgment results:

[0104]

[0105] in, is the logarithmic probability loss function of the first layer discriminator D1, R i is the i-th actually collected image of the inner cavity defect of the fan blade, is the i-th virtual defect image generated by the first layer generator G1, α i is the weight coefficient of the i-th image in the fan blade inner cavity image dataset, which is automatically adjusted based on the defect type in the defect image. The weight α of the crack crack , the weight of pores α pore and the material peeling weight α peel Set according to the characteristic differences of defects:

[0106]

[0107]

[0108] S34, optimizing the parameters of the first layer generator G1 and the first layer discriminator D1 through adversarial training, so that the first layer discriminator D1 cannot effectively distinguish the virtual defect image from the actual defect image. The optimization goal of the first layer discriminator is:

[0109]

[0110] in, is the objective function of the generative adversarial network.

[0111] S4 includes the following steps:

[0112] S41, constructing a second-layer generative adversarial network model, which includes a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer generative adversarial network model, the second-layer generator G2 further generates complex defect image data according to the input virtual defect image data. Complex defect image data includes compound cracks and material deterioration;

[0113] S42, the second layer generator G2 combines the physical characteristics of the composite crack and material deterioration, and generates a complex defect image by adjusting the physical parameters. The virtual defect image of the composite crack is calculated according to the degree of crack overlap γ crack Generate virtual defect images of material deterioration according to the material structure change parameter λ material The generated composite crack and material deterioration defect images are

[0114] S43, the second layer discriminator D2 is used to compare the complex defect image V2 generated by the second layer generator G2 with the complex defect image actually collected For fine adversarial discrimination, the loss function of the second-layer discriminator is:

[0115]

[0116] in, is the logarithmic probability loss function of the second layer discriminator, is the i-th complex defect image actually collected, is the i-th complex defect image generated by the second layer generator, β i is the weight coefficient of complex defects in the image, and the weight coefficient of crack overlap is β crack , the weight coefficient of material deterioration is β material , and the calculation formulas are:

[0117]

[0118]

[0119] S44, optimizing the parameters of the second-layer generator G2 and the second-layer discriminator D2 through adversarial training until the second-layer discriminator D2 cannot distinguish between complex defect images and actual defect images:

[0120]

[0121] in, Objective function for the second layer of the generative adversarial network.

[0122] In this implementation, S5 includes the following steps:

[0123] S51, combining the complex defect image data V2 generated by the double-layer generative adversarial network convolutional neural network model with the actually collected fan blade inner cavity virtual defect image data to construct an input data set D for convolutional neural network training;

[0124] S52, inputting the input data set D into a convolutional neural network for training, where the convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers;

[0125] S53, convolutional neural network is trained by error back propagation algorithm, loss function Used to measure the recognition accuracy of the convolutional neural network model for defect features:

[0126]

[0127] in, is the mean square error between the actual defect image and its predicted value, is the mean square error between the complex defect image and its predicted value, γ i is the weight coefficient of the actual defect image and the virtual defect image;

[0128] S54, the convolutional neural network model gradually optimizes the weight parameter W through multiple iterations c and the bias term b c , so that the convolutional neural network model learns defect features of different types and complexities, optimizes recognition capabilities, and finally outputs a trained convolutional neural network model.

[0129] In this implementation, S7 includes the following steps:

[0130] S71, generating a defect report based on the detected defect image data and the virtual defect image data, wherein the report records the specific position, size, shape and type of each detected defect;

[0131] S72, the defect position is determined by the coordinate value (x i ,y i ) and image depth z iIndicates that, combined with the actual physical size of the wind turbine blade, it is converted into the actual spatial position coordinates (X i ,Y i ,Z i ), where i is the i-th defect;

[0132] S73, defect size ΔL i It is calculated by the pixel distance in the image and converted based on the actual physical size of the pixel;

[0133] S74, the defect shape is determined by analyzing the geometric characteristics of the defect edge, and the shape parameters include crack length, pore diameter or area of ​​material peeling;

[0134] S75. The defect type is determined by the classification result of the defect by the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, and the probability distribution of each defect belonging to a crack, a pore or a material peeling is output:

[0135] p i =[p crack ,p pore ,p peel ]

[0136] S76. The generated defect report contains the specific location, size, shape and type of each defect. The report content is output in the following format:

[0137] Defect number, (X i ,Y i ,Z i ), ΔL i , S i , defect type: argmax(p i ).

[0138] Example 2 is an embodiment of the present invention, which provides a method for detecting defects in the inner cavity of a wind turbine blade based on image recognition and deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0139] The specific implementation method of wind turbine blade cavity defect detection will be verified based on the process of real cases. On August 15, 2023, a wind farm carried out regular maintenance and inspection on a batch of wind turbine blades. In this maintenance task, the operation and maintenance team selected the main blades of 10 wind turbines with an average length of 75 meters. The inspection goal was to discover and repair possible internal cavity defects such as cracks, pores and material peeling.

[0140] At 9 o'clock in the morning, the technical team began to shoot the inner cavity of the wind turbine blades using high-resolution cameras installed on an autonomous mobile robot. The cameras scanned all areas of the inner cavity from different angles. The system recorded the structural characteristics and position coordinates of the wind turbine blades, and generated an image data set of the inner cavity of each blade. By automatically adjusting the exposure time and aperture size of the equipment, the system can obtain clear images in both high-brightness areas and shadow areas.

[0141] Testing process:

[0142] Time: 10:30, August 15, 2023;

[0143] Blade number: WFLP-001;

[0144] Using image recognition and deep learning methods, the system detected three areas of material peeling located in the front inner cavity of the blade.

[0145] Defect number: #0001, defect location: front end of the inner cavity (X=4.5m, Y=0.9m, Z=0.5m);

[0146] Defect size: 15mm x 30mm, defect type: material peeling;

[0147] Time: 11:15, August 15, 2023;

[0148] Blade number: WFLP-002;

[0149] The system detected a composite crack, where cracks intersected to form a complex damaged area, and identified it as a high-risk defect.

[0150] Defect number: #0002, defect location: middle inner cavity (X=12.3m, Y=2.1m, Z=0.8m);

[0151] Defect size: crack length 35mm, width 5mm, defect type: compound crack;

[0152] The system generates a defect report and sends it to the maintenance team's terminal in real time, recommending immediate shutdown and maintenance.

[0153] In this inspection task, the method of the present invention can not only detect conventional cracks and material spalling, but also identify complex defects that are difficult to detect with traditional methods. Compared with traditional ultrasonic testing, the deep learning model can automatically identify and locate complex defects, such as compound cracks and material deterioration. The following is the comparison data of the two methods:

[0154] Table 1 Comparative data of the present invention and traditional ultrasonic testing

[0155]

[0156] It can be seen from Table 1 of Example 1 that the traditional method takes 5 hours to detect each blade, while the method of the present invention shortens the detection time to 2 hours by combining automatic image acquisition with a deep learning model. Since the system can quickly collect data at multiple angles and analyze defects in real time without manual intervention, the entire detection process is more efficient. The traditional ultrasonic method performs poorly in identifying complex defects and has a high missed detection rate. The present invention generates complex defect samples through a double-layer generative adversarial network and trains them in combination with actual blade images, which greatly improves the accuracy of defect identification. Among the 66 defects detected, there are many composite cracks and material deterioration that are difficult to find with traditional methods. After detecting a defect, the system of the present invention can instantly generate a report containing the defect location, size, shape and type, and send it to the terminal of the operation and maintenance team in real time, shortening the time from detection results to maintenance decisions. This automated process makes early warning of defects faster, thereby reducing the potential risk of blade failure.

[0157] Table 2 Comparison of fan blade test data

[0158]

[0159] By implementing the method of the present invention, the efficiency and accuracy of the inner cavity defect detection of wind turbine blades have been significantly improved. In this inspection task, the operation and maintenance team inspected the main blades of 10 wind turbines and found multiple complex defects that were difficult to detect with traditional methods. The system can automatically generate defect reports and push them to relevant personnel in real time, thereby greatly shortening the inspection time and reducing maintenance costs.

[0160] The method of the present invention is not only suitable for daily maintenance of wind turbine blades, but can also be used for detection and monitoring of large-scale wind farms to ensure the safety and reliability of wind turbine blades during long-term operation. Compared with traditional methods, the present invention not only improves the detection speed, but also reduces the missed detection rate, and performs well in complex defect identification, and has great practical application value.

[0161] The present invention constructs a two-layer generative adversarial network model. The first layer of the generative adversarial network is responsible for generating primary virtual defect images, and the second layer of the generative adversarial network generates complex defects such as composite cracks and material deterioration on this basis. Through multi-level adversarial training, the present invention can not only simulate and generate more complex virtual defects, but also finely distinguish complex defect images from actual defect images through the second layer of discriminator, thereby ensuring the accuracy of complex defect recognition. It effectively solves the problem that complex defects cannot be recognized in the prior art, especially the defects that are difficult to detect such as composite cracks and material deterioration, and greatly improves the comprehensiveness and accuracy of defect detection.

[0162] The present invention inputs the defect images generated by the double-layer generative adversarial network and the actual defect images into the convolutional neural network for training. The convolutional neural network extracts defect features through multiple convolutional layers and can not only automatically learn the edge features and morphological features of the defects, but also continuously optimize its detection capability through the error back propagation algorithm. The design of the loss function enables the model to effectively balance the recognition accuracy of actual defects and virtual defects, ensuring that the model still maintains a high detection accuracy when facing diversified defects. Compared with the existing single detection technology, the multiple network combination method of the present invention significantly improves the efficiency and classification capability of defect detection.

[0163] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0164] Example 3

[0165] A third embodiment of the present invention provides a wind turbine blade inner cavity defect detection system based on image recognition and deep learning, including: the detection system includes a camera device, a data transmission system, and a monitoring center;

[0166] The camera equipment uses a high-resolution, low-light, vibration-resistant industrial-grade camera that can adapt to the complex environmental conditions inside the wind turbine blades; the camera equipment has an automatic start-stop function and is equipped with a large-capacity battery to ensure that the battery is only charged during the annual wind turbine inspection;

[0167] The data transmission system adopts 4G or 5G transmission technology to transmit the image data collected by the camera to the monitoring center in real time, ensuring the stability and reliability of data transmission and avoiding signal interference and data loss;

[0168] The monitoring center is equipped with high-performance servers and storage devices for receiving, storing and processing image data transmitted by the camera; monitoring software is installed to realize image display, video playback and fault alarm functions. And the blade cavity is deployed.

[0169] Camera installation location Choose a suitable location inside the wind turbine blade to install the camera to ensure that the key parts of the blade, such as the blade root, main beam, web, etc., can be fully monitored. The installation method should be firm and reliable to avoid loosening or falling off during the operation of the wind turbine. At the same time, the installation should ensure that the camera signal transmission is unobstructed to prevent it from being affected by external factors.

[0170] In maintenance management, cameras, data transmission equipment and monitoring centers need to be regularly inspected and maintained to ensure the normal operation of the system. Check whether the camera lens is clean, the image quality is clear, and the data transmission is stable. Timely handle system failures, such as camera damage and data transmission interruption. Establish a troubleshooting mechanism to ensure that the normal operation of the system is restored in the shortest time. At the same time, regularly upgrade the monitoring software to improve the performance and function of the system. Timely update the image recognition algorithm and fault diagnosis model to adapt to the ever-changing monitoring needs.

[0171] By implementing the camera monitoring solution inside the wind turbine blades, real-time monitoring and early warning of the internal conditions of the wind turbine blades can be achieved, improving the reliability and availability of the wind turbines, and reducing maintenance costs and downtime. At the same time, the solution also provides strong technical support for the intelligent operation and maintenance of wind turbines, which helps promote the sustainable development of the wind power industry.

[0172] Example 4

[0173] The fourth embodiment of the present invention is different from the first three embodiments in that:

[0174] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0175] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning, characterized in that: include, Capturing images of the inner cavity of the fan blade at multiple angles and multiple positions by using an imaging device to generate a fan blade inner cavity image data set containing details of the inner cavity of the fan blade; Perform noise removal and image enhancement on the collected wind turbine blade inner cavity image dataset, and perform edge enhancement and contrast enhancement on key areas; The first-layer generative adversarial network model is used to generate primary defect samples. The first-layer generator generates virtual defect images of various types, including cracks, pores, and material peeling, based on the physical characteristics of the inner cavity of the wind turbine blade. The first-layer discriminator is used to distinguish the generated virtual defect images from the actual defect images. Based on the virtual defect image generated by the first-layer generative adversarial network model, the second-layer generator of the second-layer generative adversarial network model generates a complex defect image including compound cracks and material deterioration, and the second-layer discriminator performs fine adversarial discrimination on the complex defect image; The convolutional neural network model is used to train the complex defect images generated by the two-layer generative adversarial network model and the actual blade virtual defect images. The convolutional neural network model learns the defect features of various types and complexities and optimizes the defect recognition capability. The pre-processed wind turbine blade cavity image dataset is input into the trained convolutional neural network model to automatically identify and locate defects in the blade cavity, including the type, location, size and shape of the defects. Generate a defect report based on the defect detection results, including the specific location, size, shape and type of the defect; When new types of defects appear, the two-layer generative adversarial network model and convolutional neural network model perform adaptive learning through a feedback mechanism to continuously optimize the ability to recognize new types of defects.

2. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning according to claim 1, characterized in that: The generating of the fan blade inner cavity image data set containing the fan blade inner cavity details comprises: performing multi-angle acquisition of the fan blade inner cavity by an imaging device, wherein the angle range is θ, θ∈θ min ,θ max ], the change step of angle θ is Δθ, and the blade cavity images under multiple viewing angles are generated by adjusting θ; Capture images at multiple locations of the inner cavity of the fan blade, with the position coordinates being P(x, y, z). The acquisition locations include the front end, middle section and tail end of the blade, and generate blade inner cavity image data at different spatial locations. Generate a fan blade cavity image dataset of the fan blade cavity details based on the angle θ and position coordinates P(x,y,z): I={I1,I2,,I n } Among them, I n Indicates the nth wind turbine blade inner cavity image acquired at a specific angle and position.

3. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning as claimed in claim 2, characterized in that: The noise removal and image enhancement processing includes using an adaptive median filtering algorithm to perform noise removal on the collected fan blade inner cavity image data set, and the filter window size is W. f , dynamically adjust the filter window size W according to the uniformity of noise distribution in the fan blade cavity image f Remove random noise and preserve edge details; Perform fan blade inner cavity image enhancement processing, optimize the contrast of key areas in the fan blade inner cavity image, and set the contrast limit factor to α c ; Perform edge enhancement on the key areas of the fan blade inner cavity image, extract the edge features of the fan blade inner cavity, and set the gradient threshold τ g ; The wind turbine blade inner cavity image dataset is obtained after comprehensive noise removal, contrast enhancement and edge enhancement: I′={I′1,I′2,…,I′ n } Among them, I′ n Represents the nth fan blade inner cavity image after preprocessing.

4. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning as claimed in claim 3, characterized in that: The virtual defect image includes constructing a first-layer generative adversarial network model, which includes a first-layer generator G1 and a first-layer discriminator D1. The first-layer generator G1 generates primary virtual defect image data according to the input physical characteristics of the inner cavity of the fan blade. The primary virtual defect image data includes cracks, pores and material peeling. The output of the first-layer generator is the primary virtual defect image data: in, is the nth virtual defect image generated; The first layer generator G1 generates different types of virtual defects by adjusting parameters according to the physical characteristic parameters of the inner cavity of the fan blade, including thickness t, material density ρ, material uniformity coefficient u and inner cavity structure complexity C. The virtual defect of crack is generated by the change of thickness t and material density ρ, the virtual defect of pore is generated according to the complexity of the inner cavity structure complexity C, and the virtual defect of material peeling is determined by the material uniformity coefficient u; The first layer discriminator D1 is used to receive the virtual defect image data V1 generated by the first layer generator G1 and the actual acquired defect image R = {R1, R2,, R n }, the first layer discriminator D1 maximizes the logarithmic probability function Distinguish virtual defect images from actual defect images and output the judgment results: in, is the logarithmic probability loss function of the first layer discriminator D1, R i is the i-th actually collected image of the inner cavity defect of the fan blade, is the i-th virtual defect image generated by the first layer generator G1, α i is the weight coefficient of the i-th image in the fan blade inner cavity image dataset, which is automatically adjusted based on the defect type in the defect image. The weight α of the crack crack , the weight of pores α pore and the material peeling weight α peel Set according to the characteristic differences of defects: The parameters of the first-layer generator G1 and the first-layer discriminator D1 are optimized through adversarial training, so that the first-layer discriminator D1 cannot effectively distinguish between virtual defect images and actual defect images. The optimization goal of the first-layer discriminator is: in, is the objective function of the generative adversarial network.

5. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning according to claim 4, characterized in that: The complex defect image includes: Construct a second-layer generative adversarial network model, which includes a second-layer generator G2 and a second-layer discriminator D2. Based on the virtual defect image data generated by the first-layer generative adversarial network model, the second-layer generator G2 further generates complex defect image data based on the input virtual defect image data. Complex defect image data includes compound cracks and material deterioration; The second layer generator G2 combines the physical characteristics of composite cracks and material deterioration, and generates complex defect images by adjusting physical parameters. The virtual defect image of the composite crack is calculated according to the degree of crack overlap γ crack Generate virtual defect images of material deterioration according to the material structure change parameter λ material The generated composite crack and material deterioration defect images are The second layer discriminator D2 is used to compare the complex defect image V2 generated by the second layer generator G2 with the actual collected complex defect image For fine adversarial discrimination, the loss function of the second-layer discriminator is: in, is the logarithmic probability loss function of the second layer discriminator, is the i-th complex defect image actually collected, is the i-th complex defect image generated by the second layer generator, β i is the weight coefficient of complex defects in the image, and the weight coefficient of crack overlap is β crack , the weight coefficient of material deterioration is β material , and the calculation formulas are: The parameters of the second-layer generator G2 and the second-layer discriminator D2 are optimized through adversarial training until the second-layer discriminator D2 cannot distinguish between complex defect images and actual defect images: in, Objective function for the second layer of the generative adversarial network.

6. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning as claimed in claim 5, characterized in that: The training using the convolutional neural network model includes: The complex defect image data V2 generated by the double-layer generative adversarial network convolutional neural network model is combined with the actual collected virtual defect image data of the inner cavity of the fan blade to construct the input data set D for convolutional neural network training; Input the input data set D into the convolutional neural network for training. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers; Convolutional neural networks are trained using the error back propagation algorithm, and the loss function Used to measure the recognition accuracy of the convolutional neural network model for defect features: in, is the mean square error between the actual defect image and its predicted value, is the mean square error between the complex defect image and its predicted value, γ i is the weight coefficient of the actual defect image and the virtual defect image; The convolutional neural network model gradually optimizes the weight parameter W through multiple iterations. c and the bias term b c , so that the convolutional neural network model learns defect features of different types and complexities, optimizes recognition capabilities, and finally outputs a trained convolutional neural network model.

7. The method for detecting defects in the inner cavity of a fan blade based on image recognition and deep learning according to claim 6, characterized in that: Generating the defect report includes generating a defect report based on the detected defect image data and the virtual defect image data, wherein the report records the specific position, size, shape and type of each detected defect; The defect position is represented by the coordinate value (x i ,y i ) and image depth z i Indicates that, combined with the actual physical size of the wind turbine blade, it is converted into the actual spatial position coordinates (X i ,Y i ,Z i ), where i is the i-th defect; Defect size ΔL i It is calculated by the pixel distance in the image and converted based on the actual physical size of the pixel; The defect shape is determined by analyzing the geometric characteristics of the defect edge. The shape parameters include crack length, pore diameter or material peeling area. The defect type is determined by the classification results of the convolutional neural network model. The classification is based on the output layer of the convolutional neural network model, which outputs the probability distribution of each defect belonging to cracks, pores or material peeling: p i =[p crack ,p pore ,p peel ] The generated defect report contains the specific location, size, shape and type of each defect. The report content is output in the following format: Defect number, (X i ,Y i ,Z i ), ΔL i , S i , defect type: argmax(p i ).

8. A system using the fan blade inner cavity defect detection method based on image recognition and deep learning as described in claims 1 to 7, characterized in that: The detection system includes camera equipment, data transmission system, and monitoring center; The camera equipment uses a high-resolution, low-light, vibration-resistant industrial-grade camera that can adapt to the complex environmental conditions inside the wind turbine blades; the camera equipment has an automatic start-stop function and is equipped with a large-capacity battery to ensure that the battery is only charged during the annual wind turbine inspection; The data transmission system adopts 4G or 5G transmission technology to transmit the image data collected by the camera to the monitoring center in real time, ensuring the stability and reliability of data transmission and avoiding signal interference and data loss; The monitoring center is equipped with high-performance servers and storage devices for receiving, storing and processing image data transmitted by the camera; monitoring software is installed to realize image display, video playback, fault alarm functions, and blade cavity deployment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images