Wind power blade production defect analysis method based on image recognition

Abstract

The invention relates to the technical field of wind power blade manufacturing, and particularly discloses a wind power blade production defect analysis method based on image recognition. According to the analysis method, based on a blade production process image captured by a high-altitude arranged industrial camera in real time and a blade production process image captured by handheld photographing equipment in real time, an independent analysis process is executed on images of two scenes; the analysis process of the image comprises the following steps: S1, carrying out segmentation and normalization preprocessing on the image; s2, performing multi-scale feature extraction and key region attention enhancement processing on the image; s3, performing multi-scale feature complementation and screening processing on the image; s4, performing defect accurate positioning and classification processing on the image; and S5, judging the existence state of the defect and the category and position coordinates of the current defect. According to the invention, the quality detection efficiency of the wind power blade in the production process is improved, the quality detection precision is guaranteed, and the dependence on manpower is reduced.

Description

Technical Field

[0001] This invention relates to the field of wind turbine blade manufacturing technology, specifically a method for analyzing production defects in wind turbine blades based on image recognition. Background Technology

[0002] In the structure of a wind turbine, the wind turbine blade is one of the core components. It is the source of wind energy capture by the unit and plays a vital role in the operation of the unit.

[0003] The manufacturing process of wind turbine blades is relatively long, and can be summarized into seven stages: mold preparation, core material placement, pressure holding, infusion curing, mold closing and first bonding, mold closing and second bonding, and heat curing. Each stage is susceptible to defects caused by improper human operation. For example, in the core material placement and mold closing stages, wrinkles and foreign matter in the layup are often caused by improper operation; in the infusion curing and heat curing stages, whitening defects often occur due to improper resin ratios or temperature control. These defects directly affect the quality and production efficiency of the final blade product to varying degrees. Therefore, because the production process of wind turbine blades involves an irreversible thermosetting reaction, quality control during the production process directly determines the pass rate of wind turbine blades and the company's operating results in improving quality, reducing costs, and increasing efficiency.

[0004] Currently, quality control in the production of wind turbine blades is mainly done manually. Due to the limitations of operators' subjective experience and physiological fatigue, manual inspection suffers from low consistency and high missed detection rate in identifying defects such as wrinkles, small foreign objects, and hidden whitening. According to incomplete statistics, the missed detection rate is usually as high as 15% or more.

[0005] It is evident that current quality control methods, which rely primarily on manual labor, suffer from technical problems such as difficulty in quality control, low production efficiency, and poor precision. Summary of the Invention

[0006] The technical objective of this invention is to provide a method for analyzing wind turbine blade production defects by analyzing and recognizing images of the wind turbine blade production process based on the YOLOv5 algorithm, in order to improve the efficiency of quality inspection, ensure the accuracy of quality inspection, and reduce the reliance on manual labor, in view of the special characteristics of the wind turbine blade production process, the technical requirements for quality control, and the shortcomings of the existing technology.

[0007] The technical logic of this invention is to use the YOLOv5s algorithm as the baseline model and employ a one-stage detector architecture to achieve end-to-end defect localization and classification. The backbone network in the original YOLOv5 algorithm is a Cross-Stage Partial Network (CSPNet) used to extract deep semantic features from images. A Squeeze-and-Excitation (SE) Block is added after each convolutional layer to enhance the model's attention to key regions. The SE Block improves the model's expressive power and generalization performance by automatically learning which channels are more important. Since factors such as shooting angle and lighting conditions affect the quality of captured images, the neck network, based on the original FPN+PAN structure, adds a Convolutional Block Attention Module (CBAM) after each feature pyramid layer to enhance the model's sensitivity to defect features and defect locations, preventing feature loss.

[0008] The technical objective of this invention is achieved through the following technical solution: a method for analyzing production defects of wind turbine blades based on image recognition. The analysis method is based on images of the blade production process captured in real time by industrial cameras arranged at high altitudes and images of the blade production process captured in real time by handheld cameras, and performs independent analysis processes on the images of the two scenarios. The image parsing process involves the following steps: S1. Perform image segmentation and normalization preprocessing; S2. Using YOLOv5s as the baseline model, a Backbone network structure was constructed by combining the CSPDarknet53 network structure with the SE Block attention mechanism to perform multi-scale feature extraction and key region attention enhancement processing on the image. S3. The Neck module is constructed using the PAN module, FPN network structure and CBAM module to perform multi-scale feature complementation and filtering processing on the image; S4. Construct a multi-scale detection head module and optimize the loss function to accurately locate and classify defects in the image; S5. Using the bounding box coordinates, target confidence, and category confidence output by the Head module, combined with the threshold, determine the existence status of the defect and the current defect category and location coordinates; Output binary classification results of "large folds" or "production debris" from high-altitude scene images; Output multi-classification results, including "dry yarn", "wrinkles", "insufficient filling", "whitening", and "bubbles", based on handheld scene images.

[0009] Furthermore, in S1, the image preprocessing procedure is as follows: First, the original image is divided into multiple sub-regions using multi-size windows; Then, unify the resolution of the segmented sub-region images; Then convert the BGR format to RGB format; Then, the image is normalized. Finally, standardization is performed according to the following relationship: ; In the formula, I represents the image data before preprocessing; μ represents the mean; σ represents the standard deviation.

[0010] Furthermore, before preprocessing the handheld scene image, it is necessary to remove interfering content, including unit name watermarks and borders, from the image, retaining only the core area with defects.

[0011] Furthermore, in S2, multi-scale feature extraction and key region attention enhancement are performed on the image, specifically including the following process steps: Step 1. The input image feature map is divided into two parts using the CSPNet network structure. One part is processed by convolutional layers and residual blocks, and the other part is directly passed. The two sets of features are finally spliced ​​and fused to extract deep semantic features at three scales: P3, P4, and P5. Step 2. After each convolutional layer in the CSPDarknet53 network structure, add a Squeeze-and-Excitation Block module; First, perform global average pooling on the convolutional output feature map according to the following relationship to obtain the channel-level feature vector: ; In the formula, This represents the global average pooling value; Indicates the height of the feature map; Indicates the width of the feature map; Indicates the number of channels; This indicates that the feature map is in the height direction. Position index on; This indicates that the feature map is in the width direction. Position index on; Then, a two-layer fully connected network with a first layer of ReLU activation function and a second layer of Sigmoid activation function is used to generate channel weights. Finally, the weights are applied to the original feature map through element-wise multiplication to enhance the model's attention to defect regions, satisfying the following relationship: ; In the formula, This indicates that the feature map after channel attention weighting enhances the focus on defective areas; Represents the input feature map; This represents the Sigmoid activation function; Represents the ReLU activation function; This represents the weight matrix of the first fully connected layer; This represents the weight matrix of the second fully connected layer; Step 3. Freeze the parameters of the first 5 layers of the Backbone network structure during the initial training phase, and train only the subsequent network layers to reduce computational resource consumption and accelerate model convergence.

[0012] Furthermore, in step 1, the parameters of the CSPNet network structure are set as follows: Initial convolutional layer: 3 input channels, 64 output channels, 6×6 kernel, stride 2, padding 2; Subsequent convolutional layers are combined with C3 modules: a convolutional layer with 128 output channels is followed by 3 C3 modules, a convolutional layer with 256 output channels is followed by 6 C3 modules, a convolutional layer with 512 output channels is followed by 9 C3 modules, a convolutional layer with 1024 output channels is followed by 3 C3 modules, and finally, the SPPF module is used to enhance the feature extraction effect.

[0013] Furthermore, in S3, the image undergoes multi-scale feature complementation and filtering, specifically including the following process steps; Step A. Perform 1×1 convolutional dimensionality reduction on the P5 feature layer output by the Backbone network structure, and perform multi-channel data upsampling; Step B. After completing the upsampling in Step A, the sample is concatenated and fused with the P4 feature layer, which has also undergone 1×1 convolution dimensionality reduction. After processing, the fused feature layer is subjected to 1×1 convolution dimensionality reduction and upsampling again, and then spliced ​​and fused with the P3 feature layer that has undergone 1×1 convolution dimensionality reduction to enhance the small defect detection capability. Step C. Perform 3×3 convolution downsampling on the P3 feature layer after fusion of the FPN network structure, and then concatenate and fuse it with the feature layer after P4 processing in the FPN network structure process; Then, a 3×3 convolution downsampling is performed on this feature layer, and it is spliced ​​and fused with the P5 feature layer output by the Backbone network structure that has undergone 1×1 convolution dimensionality reduction to enhance feature flow and information transmission. Step D. After the feature layers P3, P4 and P5 are fused, the convolutional block attention module is introduced sequentially for attention filtering.

[0014] Furthermore, in step D, the attention filtering includes the following specific steps: Step a. Generate channel weights through the channel attention module, perform global average pooling and global max pooling on the input feature map respectively, concatenate the results, and then process them sequentially through 1×1 convolution, ReLU activation function, 1×1 convolution and Sigmoid activation function to finally output the weight coefficients of each channel. The following relationship must be satisfied: ; In the formula, Represents the channel weight matrix; Represents the feature map; Step b. Perform average pooling and max pooling on the feature maps in the channel dimension, and then concatenate the two resulting two-dimensional feature maps. Perform 7×7 convolution on the concatenated result, and finally activate it with the Sigmoid activation function to output the spatial weight matrix. The following relationship must be satisfied: ; In the formula, Represents the spatial weight matrix; Step c. By using element-wise multiplication, the channel weights and spatial weights are multiplied sequentially with the feature map to integrate the attention filtering effect; The following relationship must be satisfied: ; In the formula, This indicates the attention filtering result of the feature map.

[0015] Furthermore, in S4, the image undergoes precise defect localization and classification, specifically including the following process steps; Step I. Use the P3, P4, and P5 feature layers processed by the Neck module as input to the detection head. The output of each scale includes a 6-dimensional vector containing the coordinates of the bounding box center point, the width and height of the bounding box, the target confidence, and the class confidence. Among them, the P3 scale focuses on detecting small defects, including bubbles; The P4 scale focuses on detecting defects, including wrinkles; The P5 scale focuses on detecting large defects, including production debris. Step II. Calculate the bounding box regression loss using the Complete-IoU loss function, taking into account the overlapping area, center distance, and aspect ratio. The following relationship must be satisfied: ; In the formula, Represents the Complete-IoU loss function; , which represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box; A To predict the area of ​​the frame; B This represents the actual area of ​​the bounding box. This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; Indicates the length of the diagonal of the minimum bounding box; This represents the weighting coefficient used to balance the impact of the aspect ratio loss term; This measures the consistency between the aspect ratio of the predicted bounding box and the ground truth bounding box. This is the actual frame width; The actual bounding box height; This is the predicted bounding box width; To predict the bounding box height; Step III. The total loss function is composed of a weighted sum of the classification loss and the target confidence loss; The following relationship must be satisfied: ; In the formula, Represents the total loss function; The weighting coefficients represent the bounding box regression loss; The weighting coefficients representing the loss of target confidence; Represents the target confidence loss function; The weighting coefficients represent the classification loss. This represents the classification loss function.

[0016] Furthermore, the industrial cameras are multiple industrial-grade CMOS cameras, which are distributed above the blade production station, and there is an overlapping field of view between adjacent industrial cameras. Each industrial camera is connected to a high-altitude edge computer, which completes the recognition process of the corresponding input image. The handheld camera device is a mobile phone held by the construction worker. The mobile phone is connected to a mobile phone server, which performs the recognition process on the input corresponding image.

[0017] Furthermore, the blade production process includes process nodes before blade glue application, process nodes after blade glue application, and process nodes after blade mold closing.

[0018] The beneficial technical effects of this invention are as follows: The above-mentioned technical measures address the unique characteristics of the wind turbine blade production process and the technical requirements for quality control. By using industrial cameras arranged at high altitudes and handheld cameras (preferably mobile phones) operated manually, images of the wind turbine blade production process are acquired collaboratively. Based on the YOLOv5 algorithm, the images of the two scenarios are analyzed and identified separately. Defects such as "large wrinkles" or "production debris" in wind turbine blade production are detected through image analysis and identification of the high-altitude scene, while defects including "dry yarn," "wrinkles," "insufficient filling," "whitening," and "bubbles" are detected through image analysis and identification of the handheld scene. This improves the efficiency of quality inspection during the wind turbine blade production process, ensures the accuracy of quality inspection, and reduces reliance on manual labor. Therefore, a highly intelligent quality control process ensures the compliance rate of wind turbine blade production, thereby improving the enterprise's goals of quality improvement, cost reduction, and efficiency enhancement.

[0019] Specifically, the above technical measures include: 1. This invention achieves significant breakthroughs in detection performance in two typical application scenarios by introducing an attention mechanism into the YOLO v5s algorithm and combining it with loss function optimization; Tests showed that in high-altitude detection scenarios, the accuracy rate for recognizing large wrinkles reached approximately 96.2%, the recall rate for production debris improved to approximately 95.8%, and the mAP@0.5 index reached approximately 96.5%. In mobile device detection scenarios, the dry yarn detection accuracy is approximately 95.1%, the bubble recall rate is approximately 94.7%, and the mAP@0.5 is as high as approximately 95.3%. All key indicators exceeded the industry benchmark of 95%, which fully demonstrates its superior detection capabilities and stability in complex scenarios. Compared to traditional manual detection methods (which typically have a false negative rate exceeding 15%), this invention reduces the false negative rate by approximately 13 to 14 percentage points.

[0020] 2. This invention integrates multi-scale training, SE / CBAM attention mechanism and generative data augmentation technology, enabling the improved model to exhibit excellent robustness in complex industrial environments; Under extreme testing conditions, such as 50,000 lux strong light reflection, 500 lux nighttime supplemental lighting, and 45° oblique shooting scenarios, the fluctuation range of its detection accuracy is ≤3%; It is evident that the environmental adaptability and generalization ability of the present invention are significantly enhanced, enabling it to reliably cope with complex field conditions. Attached Figure Description

[0021] Figure 1 This is a process flow diagram of the present invention. Detailed Implementation

[0022] This invention relates to the field of wind turbine blade manufacturing technology, specifically a method for analyzing production defects in wind turbine blades based on image recognition. The following description, in conjunction with the accompanying drawings, illustrates this method. Figure 1 The technical solution of this invention will be clearly and thoroughly explained.

[0023] It should be noted that the accompanying drawings of this invention are schematic, and unnecessary details have been simplified to clarify the technical objectives of this invention, so as to avoid obscuring the technical solutions contributed by this invention to the prior art. Furthermore, the terms "approximately" or "basically" used below to refer to quantities or fit relationships mean that reasonable assembly and processing errors are allowed in the industry, and do not literally describe absolute quantities or fit relationships.

[0024] See Figure 1 As shown, the present invention is a method for analyzing and recognizing images in the production of wind turbine blades based on the YOLOv5 algorithm to determine the defects in the production of wind turbine blades. It is based on images of the blade production process captured in real time by industrial cameras arranged at high altitudes, and images of the blade production process captured in real time by handheld camera devices - mobile phones, and performs independent analysis processes on the images of the two scenarios.

[0025] Specifically, the wind turbine blade production process targeted by this invention includes the process nodes before blade glue injection, the process nodes after blade glue injection, and the process nodes after blade mold assembly, basically covering the entire process of wind turbine blade production.

[0026] The present invention utilizes 18 industrial-grade CMOS cameras, each with a resolution of 5472×3648 pixels. These cameras are distributed approximately 18 meters above the blade production station, with overlapping fields of view between adjacent cameras. Each camera is connected to a high-altitude edge computer. These 18 cameras synchronously and in real-time capture images of the blade surface at various stages of wind turbine blade production, detecting two types of defects: large wrinkles (≥200×200 pixels) and production debris. The high-altitude edge computer then performs the corresponding image recognition process. Testing showed that, through the synchronous capture by the 18 industrial cameras and the real-time processing by the high-altitude edge computer according to the following analytical recognition process, the inspection time for a single blade was significantly reduced from approximately 30 minutes manually to approximately 4 minutes, improving efficiency by approximately 86.7%.

[0027] The preferred handheld camera device is a mobile phone held by the construction worker, connected to a mobile server. At each stage of wind turbine blade production, the worker's phone captures real-time images of the blade surface, detecting defects such as "dry yarn," "wrinkles," "insufficient filling," "whitening," and "bubbles." The mobile server then processes the corresponding images for identification. Testing showed that the mobile server, using the following real-time processing method, can detect a single image in as little as 20 milliseconds, allowing operators to capture images on-site and obtain results instantly without requiring specialized training on the equipment.

[0028] Image processing for both scenarios (high-altitude and handheld scenarios) involves the following parsing and recognition process, each performed independently by its respective server (high-altitude edge computer and mobile server). Based on this, the image parsing process is as follows: S1. Because the target of the defect is small, a sliding segmentation image algorithm must be used for preprocessing, that is, the image is segmented and normalized. S2. Using YOLOv5s as the baseline model, a Backbone network structure was constructed by combining the CSPDarknet53 network structure with the SE Block attention mechanism to perform multi-scale feature extraction and key region attention enhancement processing on the image. S3. The Neck module is constructed using the PAN module, FPN network structure and CBAM module to perform multi-scale feature complementation and filtering processing on the image; S4. Construct a multi-scale detection head module and optimize the loss function to accurately locate and classify defects in the image; S5. Using the bounding box coordinates, target confidence, and category confidence output by the Head module, combined with the threshold, determine the existence status of the defect and the current defect category and location coordinates; Output binary classification results of "large folds" or "production debris" from high-altitude scene images; Output multi-classification results, including "dry yarn", "wrinkles", "insufficient filling", "whitening", and "bubbles", based on handheld scene images.

[0029] More specifically, in S1, the image preprocessing process is as follows: The original image is first divided into multiple sub-regions using windows of different sizes (2852×1698 pixels, 1769×1003 pixels, 294×276 pixels) to facilitate accurate identification of minute defects. Then, the segmented sub-region images are uniformly sized to 1280×1280 pixels using the cv2.resize function; Then, the cv2.cvtColor function is used to convert the BGR format to RGB format. Then, the image data is converted to np.float32 type and divided by 255.0 to complete the normalization process of the image; Finally, standardization is performed according to the following relationship: ; In the formula, I represents the image data before preprocessing; μ represents the mean, μ = [0.485, 0.456, 0.406]; σ represents the standard deviation, and σ = [0.229, 0.224, 0.225] is used for standardization.

[0030] In addition, before preprocessing the images of handheld scenes, it is necessary to remove interfering content, including unit name watermarks and borders, from the images, retaining only the core areas with defects.

[0031] In S2, multi-scale feature extraction and key region attention enhancement are performed on the image, specifically including the following process steps: Step 1. The input image feature map is divided into two parts using the CSPNet network structure. One part is processed by convolutional layers and residual blocks, and the other part is directly passed. The two sets of features are finally spliced ​​and fused together to extract deep semantic features at three scales: P3 (80×80 pixels), P4 (40×40 pixels), and P5 (20×20 pixels). The parameters for the CSPNet network structure are set as follows: Initial convolutional layer: 3 input channels, 64 output channels, 6×6 kernel, stride 2, padding 2; Subsequent convolutional layers are combined with C3 modules: a convolutional layer with 128 output channels (3×3, stride 2) is followed by 3 C3 modules; a convolutional layer with 256 output channels (3×3, stride 2) is followed by 6 C3 modules; a convolutional layer with 512 output channels (3×3, stride 2) is followed by 9 C3 modules; a convolutional layer with 1024 output channels (3×3, stride 2) is followed by 3 C3 modules; finally, an SPPF module (5×5 convolutional kernel) is used to enhance the feature extraction effect. Step 2. After each convolutional layer in the CSPDarknet53 network structure, add a Squeeze-and-Excitation Block module; First, perform global average pooling on the convolutional output feature map according to the following relationship to obtain the channel-level feature vector: ; In the formula, This represents the global average pooling value; Indicates the height of the feature map; Indicates the width of the feature map; Indicates the number of channels; This indicates that the feature map is in the height direction. Position index on; This indicates that the feature map is in the width direction. Position index on; Then, a two-layer fully connected network with a first layer of ReLU activation function and a second layer of Sigmoid activation function is used to generate channel weights. Finally, the weights are applied to the original feature map through element-wise multiplication to enhance the model's attention to defect regions, satisfying the following relationship: ; In the formula, This indicates that the feature map after channel attention weighting enhances the focus on defective areas; This represents the input feature map, with dimension 1. × × ,in For height, For width, Number of channels; Indicates the feature map The channel-level feature vector obtained by global average pooling has a dimension of ; This represents the Sigmoid activation function, which maps weight values ​​to the (0,1) interval; Represents the ReLU activation function, introducing nonlinearity; This represents the weight matrix of the first fully connected layer, with the dimension changed from... Compress to ( The reduction ratio is usually taken as 16). This represents the weight matrix of the second fully connected layer, with the dimension changed from... Restore to ; Step 3. Freeze the parameters of the first 5 layers of the Backbone network structure (based on pre-trained weights) during the initial training phase, and train only the subsequent network layers to reduce computational resource consumption and accelerate model convergence.

[0032] In S3, the image undergoes multi-scale feature complementation and filtering, specifically including the following process steps; Step A. Perform 1×1 convolution dimensionality reduction on the P5 feature layer output by the Backbone network structure (output channels 512), and use the function torch.nn.Upsample in the PyTorch deep learning framework to perform multi-channel data upsampling, setting the function parameter to a scaling factor of 2 and selecting nearest interpolation as the interpolation method; Step B. After completing the upsampling in Step A, the sample is spliced ​​and fused with the P4 feature layer, which also undergoes 1×1 convolutional dimensionality reduction to 512 channels. After the fused feature layer is processed by the C3 module with 3 output channels of 512, it is subjected to 1×1 convolution dimensionality reduction again, with 256 output channels and upsampling. It is then spliced ​​and fused with the P3 feature layer which has been reduced to 256 channels by 1×1 convolution to enhance the small defect detection capability. Step C. The P3 feature layer after the fusion of the FPN network structure is downsampled by 3×3 convolution (stance 2, output channels 256), and then spliced ​​and fused with the feature layer after the P4 processing in the FPN network structure process, and processed by 3 C3 modules (output channels 512); Then, a 3×3 convolution (stance 2, output channels 512) is performed on the feature layer for downsampling. It is then spliced ​​and fused with the P5 feature layer output by the Backbone network structure, which has been reduced to 1024 channels by a 1×1 convolution. After processing by three C3 modules (output channels 1024), the feature flow and information transmission are enhanced. Step D. After the feature layers P3, P4, and P5 are fused, a Convolutional Block Attention Module (CBAM) is introduced sequentially for attention filtering. In step D above, the attention filtering includes the following specific steps: Step a. Generate channel weights through the channel attention module, perform global average pooling and global max pooling on the input feature map respectively, concatenate the results, and then process them sequentially through 1×1 convolution, ReLU activation function, 1×1 convolution and Sigmoid activation function to finally output the weight coefficients of each channel. The following relationship must be satisfied: ; In the formula, Represents the channel weight matrix; Represents the feature map; Step b. Perform average pooling and max pooling on the feature maps in the channel dimension, and then concatenate the two resulting two-dimensional feature maps. Perform 7×7 convolution on the concatenated result, and finally activate it with the Sigmoid activation function to output the spatial weight matrix. The following relationship must be satisfied: ; In the formula, Represents the spatial weight matrix; Step c. By using element-wise multiplication, the channel weights and spatial weights are multiplied sequentially with the feature map to integrate the attention filtering effect; The following relationship must be satisfied: ; In the formula, This indicates the attention filtering result of the feature map.

[0033] In S4, the image undergoes precise defect localization and classification, specifically including the following process steps; Step I. Using the P3, P4, and P5 feature layers processed by the Neck module as input to the detection head, each scale outputs a 6-dimensional vector (cx, cy, w, h, obj_conf, cls_conf), where cx and cy are the coordinates of the bounding box center point, w is the width of the bounding box, h is the height of the bounding box, obj_conf is the target confidence (determining whether it is a defect), and cls_conf is the category confidence (determining the defect category); among them, the P3 scale (80×80 pixels) focuses on detecting small defects, including bubbles; The P4 scale (40×40 pixels) focuses on detecting medium-sized defects, including wrinkles; The P5 scale (20×20 pixels) focuses on detecting large defects, including production debris; Step II. Calculate the bounding box regression loss using the Complete-IoU loss function, taking into account the overlapping area, center distance, and aspect ratio. The following relationship must be satisfied: ; In the formula, Represents the Complete-IoU loss function; , which represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box; A To predict the area of ​​the frame; B This represents the actual area of ​​the bounding box. This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; Indicates the length of the diagonal of the minimum bounding box; The weighting coefficient used to balance the impact of aspect ratio loss is expressed by the following formula: ; The aspect ratio of the predicted bounding box to the ground truth bounding box is a measure of consistency, and the formula is as follows: ; This is the actual frame width; The actual bounding box height; This is the predicted bounding box width; To predict the bounding box height; Step III. The total loss function is composed of a weighted sum of the classification loss and the target confidence loss; The following relationship must be satisfied: ; In the formula, Represents the total loss function; These represent the weighting coefficients of the bounding box regression loss, used to balance the impact of the bounding box localization loss. ; The weighting coefficients represent the target confidence loss. ; Represents the target confidence loss function; The weighting coefficients represent the classification loss. =0.5; Represents the classification loss function; As above , , A weighting coefficient of 0.5 is used to balance the impact of each loss term on model training.

[0034] The classification loss (Lcls) and target confidence loss (Lobj) mentioned above are both calculated using the binary cross-entropy loss function (specifically BCEWithLogitsLoss).

[0035] In S5, the model outputs bounding box coordinates with parameters named cx, cy, w, and h through the Head module, target confidence with parameter name obj_conf, and class confidence with parameter name cls_conf, and sets confidence thresholds. If not set, the default is ,when When a defect is found in the area, the defect category is determined based on cls_conf, and the category corresponding to the maximum value of cls_conf is taken as the final defect category. The system outputs binary classification results of "large folds" or "production debris" from high-altitude scene images, while simultaneously recording the coordinates of the defect locations. The system outputs multi-classification results, including "dry yarn," "wrinkles," "insufficient filling," "whitening," and "bubbles," from handheld scene images and uploads them synchronously to a central server for storage, supporting subsequent quality traceability.

[0036] As can be seen from the above process, the production of wind turbine blades requires a system consisting of the aforementioned hardware and intelligent agents (software) to perform the above processes, enabling intelligent detection of the corresponding blade production process. This system includes the aforementioned 18 industrial cameras, a high-altitude edge computer, mobile phones for construction workers (or mobile phones specifically configured for quality inspection), and a mobile server. The industrial cameras and mobile phones are responsible for real-time image acquisition, while the high-altitude edge computer and mobile server are responsible for running the above processes, analyzing and recognizing the images, and outputting the corresponding results.

[0037] Based on this, the high-altitude edge computer / mobile server performs the following model training and full-process defect identification process: 1. Dataset Construction and Allocation: The dataset consists of real collected data and generative augmented data. The dataset is divided into training and test sets in a 9:1 ratio to avoid sample overlap. 2. Model training parameter settings: The learning rate adopts piecewise constant decay (initially 0.01, decreasing to 0.001 in the 30th cycle), the batch size is 128 (can be adjusted to 64 depending on the GPU memory), and a total of 50 cycles are trained and the optimal weights are saved; Multi-scale inputs (640×640, 800×800, 1280×1280) are introduced during training to improve model adaptability; 3. Full-process defect identification: After training, the model is deployed to high-altitude edge computing and mobile servers for full-process defect identification of wind turbine blades: detecting wrinkles and debris before glue application; detecting glue overflow and bubbles after glue application; detecting pressure and mold-related defects after mold closing. 4. Model Iteration and Optimization: To continuously optimize performance, the system will collect missed / false detection samples a second time, and then re-perform data augmentation, labeling and training. The network structure or augmentation strategy can be adjusted until the model accuracy, mAP@0.5 and recall are all ≥95%, meeting the standards for industrial applications.

[0038] The above specific technical solutions are only used to illustrate the present invention, and are not intended to limit it.

[0039] Although the present invention has been described in detail with reference to the specific technical solutions described above, those skilled in the art should understand that modifications can still be made to the specific technical solutions described above, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the present invention.

Claims

1. A method for analyzing manufacturing defects in wind turbine blades based on image recognition, characterized in that: The analysis method is based on real-time images of the blade production process captured by high-altitude industrial cameras and real-time images of the blade production process captured by handheld cameras, and performs independent analysis processes on the images of the two scenarios. The image parsing process involves the following steps: S1. Perform image segmentation and normalization preprocessing; S2. Using YOLOv5s as the baseline model, a Backbone network structure was constructed by combining the CSPDarknet53 network structure with the SE Block attention mechanism to perform multi-scale feature extraction and key region attention enhancement processing on the image. S3. The Neck module is constructed using the PAN module, FPN network structure and CBAM module to perform multi-scale feature complementation and filtering processing on the image; S4. Construct a multi-scale detection head module and optimize the loss function to accurately locate and classify defects in the image; S5. Using the bounding box coordinates, target confidence, and category confidence output by the Head module, combined with the threshold, determine the existence status of the defect and the current defect category and location coordinates; Output binary classification results of "large folds" or "production debris" from high-altitude scene images; Output multi-classification results, including "dry yarn", "wrinkles", "insufficient filling", "whitening" and "bubbles", from handheld scene images.

2. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, In S1, the image preprocessing procedure is as follows: First, the original image is divided into multiple sub-regions using multi-size windows; Then, unify the resolution of the segmented sub-region images; Then convert the BGR format to RGB format; Then, the image is normalized. Finally, standardization is performed according to the following relationship: ； In the formula, I represents the image data before preprocessing; μ represents the mean; σ represents the standard deviation.

3. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1 or 2, characterized in that, Before preprocessing images of handheld scenes, it is necessary to remove interfering content, including unit name watermarks and borders, from the images, retaining only the core areas with defects.

4. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, In S2, multi-scale feature extraction and key region attention enhancement are performed on the image, specifically including the following process steps: Step 1. The input image feature map is divided into two parts using the CSPNet network structure. One part is processed by convolutional layers and residual blocks, and the other part is directly passed. The two sets of features are finally spliced ​​and fused to extract deep semantic features at three scales: P3, P4, and P5. Step 2. After each convolutional layer in the CSPDarknet53 network structure, add a Squeeze-and-ExcitationBlock module; First, perform global average pooling on the convolutional output feature map according to the following relationship to obtain the channel-level feature vector: ； In the formula, This represents the global average pooling value; Indicates the height of the feature map; Indicates the width of the feature map; Indicates the number of channels; This indicates that the feature map is in the height direction. Position index on; This indicates that the feature map is in the width direction. Position index on; Then, a two-layer fully connected network with a first layer of ReLU activation function and a second layer of Sigmoid activation function is used to generate channel weights. Finally, the weights are applied to the original feature map through element-wise multiplication to enhance the model's attention to defect regions, satisfying the following relationship: ； In the formula, This indicates that the feature map after channel attention weighting enhances the focus on defective areas; Represents the input feature map; This represents the Sigmoid activation function; Represents the ReLU activation function; This represents the weight matrix of the first fully connected layer; This represents the weight matrix of the second fully connected layer; Step 3. Freeze the parameters of the first 5 layers of the Backbone network structure during the initial training phase, and train only the subsequent network layers to reduce computational resource consumption and accelerate model convergence.

5. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 4, characterized in that, In step 1, the parameters of the CSPNet network structure are set as follows: Initial convolutional layer: 3 input channels, 64 output channels, 6×6 kernel, stride 2, padding 2; Subsequent convolutional layers are combined with C3 modules: a convolutional layer with 128 output channels is followed by 3 C3 modules, a convolutional layer with 256 output channels is followed by 6 C3 modules, a convolutional layer with 512 output channels is followed by 9 C3 modules, a convolutional layer with 1024 output channels is followed by 3 C3 modules, and finally, the SPPF module is used to enhance the feature extraction effect.

6. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, In S3, the image undergoes multi-scale feature complementation and filtering, specifically including the following process steps; Step A. Perform 1×1 convolutional dimensionality reduction on the P5 feature layer output by the Backbone network structure, and perform multi-channel data upsampling; Step B. After completing the upsampling in Step A, the sample is concatenated and fused with the P4 feature layer, which has also undergone 1×1 convolution dimensionality reduction. After processing, the fused feature layer is subjected to 1×1 convolution dimensionality reduction and upsampling again, and then spliced ​​and fused with the P3 feature layer that has undergone 1×1 convolution dimensionality reduction to enhance the small defect detection capability. Step C. Perform 3×3 convolution downsampling on the P3 feature layer after fusion of the FPN network structure, and then concatenate and fuse it with the feature layer after P4 processing in the FPN network structure process; Then, a 3×3 convolution downsampling is performed on this feature layer, and it is spliced ​​and fused with the P5 feature layer output by the Backbone network structure that has undergone 1×1 convolution dimensionality reduction to enhance feature flow and information transmission. Step D. After the feature layers P3, P4 and P5 are fused, the convolutional block attention module is introduced sequentially for attention filtering.

7. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 6, characterized in that, In step D, the attention filtering includes the following specific steps: Step a. Generate channel weights through the channel attention module, perform global average pooling and global max pooling on the input feature map respectively, concatenate the results, and then process them sequentially through 1×1 convolution, ReLU activation function, 1×1 convolution and Sigmoid activation function to finally output the weight coefficients of each channel. The following relationship must be satisfied: ； In the formula, Represents the channel weight matrix; Represents the feature map; Step b. Perform average pooling and max pooling on the feature maps in the channel dimension, and then concatenate the two resulting two-dimensional feature maps. Perform 7×7 convolution on the concatenated result, and finally activate it with the Sigmoid activation function to output the spatial weight matrix. The following relationship must be satisfied: ； In the formula, Represents the spatial weight matrix; Step c. By using element-wise multiplication, the channel weights and spatial weights are multiplied sequentially with the feature map to integrate the attention filtering effect; The following relationship must be satisfied: ； In the formula, This indicates the attention filtering result of the feature map.

8. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, In S4, the image undergoes precise defect localization and classification, specifically including the following process steps; Step I. Use the P3, P4, and P5 feature layers processed by the Neck module as input to the detection head. The output of each scale includes a 6-dimensional vector containing the coordinates of the bounding box center point, the width and height of the bounding box, the target confidence, and the class confidence. Among them, the P3 scale focuses on detecting small defects, including bubbles; The P4 scale focuses on detecting defects, including wrinkles; The P5 scale focuses on detecting large defects, including production debris. Step II. Calculate the bounding box regression loss using the Complete-IoU loss function, taking into account the overlapping area, center distance, and aspect ratio. The following relationship must be satisfied: ； In the formula, Represents the Complete-IoU loss function; , which represents the intersection-union ratio of the predicted bounding box and the ground truth bounding box; A To predict the area of ​​the frame; B This represents the actual area of ​​the bounding box. This represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box; Indicates the length of the diagonal of the minimum bounding box; This represents the weighting coefficient used to balance the impact of the aspect ratio loss term; This measures the consistency between the aspect ratio of the predicted bounding box and the ground truth bounding box. This is the actual frame width; The actual bounding box height; This is the predicted bounding box width; To predict the bounding box height; Step III. The total loss function is composed of a weighted sum of the classification loss and the target confidence loss; The following relationship must be satisfied: ； In the formula, Represents the total loss function; The weighting coefficients represent the bounding box regression loss; The weighting coefficients representing the loss of target confidence; Represents the target confidence loss function; The weighting coefficients represent the classification loss. This represents the classification loss function.

9. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, The industrial cameras are multiple industrial-grade CMOS cameras, which are distributed above the blade production station. There is an overlapping field of view between adjacent industrial cameras. Each industrial camera is connected to a high-altitude edge computer, which performs the recognition process on the corresponding input image. The handheld camera device is a mobile phone held by the construction worker. The mobile phone is connected to a mobile phone server, which performs the recognition process on the input corresponding image.

10. The method for analyzing manufacturing defects of wind turbine blades based on image recognition according to claim 1, characterized in that, The blade production process includes the process nodes before blade glue injection, the process nodes after blade glue injection, and the process nodes after blade mold closing.

Images