A multi-scale fusion blade defect detection method based on improved yolov3

Abstract

The present application relates to the technical field of target detection, and in particular to a multi-scale fusion blade defect detection method based on an improved yolov3. The method comprises the following steps: collecting defect sample data of an engine internal blade; pre-processing the defect sample data to obtain blade defect data; reducing the number of channels of a backbone network of a yolov3 model, sending features extracted by the backbone network into a feature fusion layer for multi-scale fusion to obtain an improved yolov3 model; pre-training the improved yolov3 model to obtain a pre-training model; loading the pre-training model, training the blade defect data until the pre-training model converges. The purpose of the multi-scale fusion blade defect detection method based on the improved yolov3 is to solve the problems of low detection efficiency and poor accuracy of existing blade defect detection.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and specifically to a multi-scale fusion blade defect detection method based on an improved YOLOv3. Background Technology

[0002] Engine failures caused by blade surface defects account for approximately 75% of all engine malfunctions. Therefore, rigorous detection of blade cracks and defects is crucial for ensuring flight safety and reliability. While numerous studies have been conducted both domestically and internationally on the detection of blade and surface cracks, including methods such as X-ray inspection, fluorescent magnetic particle inspection, borehole inspection, ultrasonic inspection, and infrared thermal imaging, these ultimately rely on manual visual inspection or simple computer image processing. These methods have limitations and heavily depend on the experience and skill level of the inspectors, leading to inefficiency and the occurrence of missed or incorrect detections. Therefore, research into automatic blade surface defect identification technology is urgently needed to improve the efficiency and accuracy of blade inspection.

[0003] Therefore, the inventors provide a multi-scale fusion blade defect detection method based on improved YOLOv3. Summary of the Invention

[0004] (1) Technical problems to be solved

[0005] This invention provides a multi-scale fusion blade defect detection method based on an improved YOLOv3, which solves the technical problems of low detection efficiency and poor accuracy in existing blade defect detection methods.

[0006] (2) Technical solution

[0007] This invention provides a multi-scale fusion blade defect detection method based on an improved YOLOv3, comprising the following steps:

[0008] Collect defect sample data of the internal blades of the engine;

[0009] The defect sample data is preprocessed to obtain blade defect data;

[0010] The number of channels in the backbone network of the YOLOv3 model is reduced, and the features extracted by the backbone network are fed into the feature fusion layer for multi-scale fusion to obtain the improved YOLOv3 model.

[0011] The improved YOLOv3 model was used for pre-training to obtain a pre-trained model;

[0012] Load the pre-trained model and train it on the blade defect data until the pre-trained model converges.

[0013] Furthermore, the acquisition of defect sample data of the internal blades of the engine specifically involves: acquiring the defect sample data through a probe device, and the acquisition type can be images or videos.

[0014] Furthermore, the preprocessing of the defect sample data to obtain blade defect data specifically involves performing data annotation, enhancement, and normalization on the defect sample data to obtain the blade defect data.

[0015] Furthermore, the step of sending the features extracted by the backbone network into the feature fusion layer for multi-scale fusion specifically involves the feature fusion layer extracting features from the feature layer output by the backbone network in at least two ways, and then concatenating the processed results by channel to achieve multi-scale feature fusion.

[0016] Furthermore, the feature fusion layer extracts features from the feature layer output by the backbone network in at least two ways, and concatenates the processed results by channel to achieve multi-scale feature fusion, specifically including the following steps:

[0017] The first method uses an m*m input convolution, followed by an n*n convolution.

[0018] The second method uses an m*m convolution as input, followed by a p*p convolution.

[0019] The first method and the second method have the same number of output channels, and feature fusion is performed according to layers;

[0020] The third approach involves connecting an n*n maximum pooling layer and then concatenating it with an m*m convolutional layer.

[0021] The fourth approach involves connecting a p*p maximum pooling layer and then concatenating it with an m*m convolutional layer.

[0022] The output feature maps of the third and fourth methods are the same size, and the blank areas at the edges of the feature maps are filled with 0. The outputs of these two methods are then spliced ​​together by channel.

[0023] The layer fusion result and the channel fusion result are then fused again to obtain the final output layer;

[0024] Where m, n, and p are all positive integers, and m < n < p.

[0025] Furthermore, when the number of layers in the layer fusion result is the same as the number of layers in the channel fusion result, the two fusion results are fused together to obtain the output layer.

[0026] Furthermore, the bounding box regression loss of the yolov3 model is adjusted to Smooth L1 loss.

[0027] Furthermore, the step of pre-training the improved YOLOv3 model to obtain a pre-trained model specifically involves using the COCO2017 dataset to pre-train the improved YOLOv3 model to obtain the pre-trained model.

[0028] (3) Beneficial effects

[0029] In summary, this invention reduces the number of channels and parameters by fine-tuning the YOLOv3 backbone network, thereby increasing the detection speed. It also provides rich multi-scale features for the blade defect detection process by utilizing a multi-scale feature fusion method that combines channel fusion and corresponding feature layer fusion, thus improving accuracy. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic flowchart of a multi-scale fusion blade defect detection method based on an improved YOLOv3 provided in an embodiment of the present invention;

[0032] Figure 2 This is a flowchart illustrating a multi-scale feature fusion method for multi-scale fusion blade defect detection based on an improved YOLOv3, provided in an embodiment of the present invention. Detailed Implementation

[0033] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present invention by way of example, but should not be used to limit the scope of the present invention. That is, the present invention is not limited to the described embodiments, and any modifications, substitutions and improvements to the parts, components and connection methods are covered without departing from the spirit of the present invention.

[0034] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] Figure 1 This is a flowchart illustrating a multi-scale fusion blade defect detection method based on an improved YOLOv3, as provided in an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0036] S100: Collect defect sample data of the internal blades of the engine.

[0037] Specifically, defect sample data is acquired using a probe device, and the acquisition type can be images or videos.

[0038] S200. Preprocess the defect sample data to obtain blade defect data.

[0039] Specifically, the defect sample data is labeled, enhanced, and normalized to obtain blade defect data.

[0040] S300. Reduce the number of channels in the backbone network of the YOLOv3 model, and feed the features extracted from the backbone network into the feature fusion layer for multi-scale fusion to obtain the improved YOLOv3 model.

[0041] Specifically, the feature fusion layer extracts features from the feature layer output by the backbone network in at least two ways, and concatenates the processed results by channel to achieve multi-scale feature fusion.

[0042] One way to reduce the number of parameters in the YOLOv3 model is to halve the number of channels in the backbone network. See Table 1 below for details:

[0043] Table 1. Parameter list of the fine-tuned darknet53 backbone network

[0044]

[0045]

[0046] S400. Pre-training is performed using the improved YOLOv3 model to obtain the pre-trained model.

[0047] Specifically, the improved YOLOv3 model was pre-trained using the COCO2017 dataset to obtain the pre-trained model.

[0048] S500: Load the pre-trained model and train it on the blade defect data until the pre-trained model converges.

[0049] As an optional implementation, in step S300, the feature fusion layer extracts features from the feature layer output by the backbone network in at least two ways, and concatenates the processed results by channel to achieve multi-scale feature fusion, specifically including the following steps:

[0050] Step S301: The first method uses an m*m input convolution, followed by an n*n convolution.

[0051] Step S302: The second method uses an input m*m convolution, followed by a concatenated p*p convolution;

[0052] Step S303: The first method and the second method have the same number of output channels, and feature fusion is performed according to the layers;

[0053] Step S304: The third method involves connecting an n*n maximum pooling layer and concatenating it with an m*m convolutional layer;

[0054] Step S305: The fourth method involves connecting a p*p maximum pooling layer and concatenating it with an m*m convolution.

[0055] Step S306: The output feature maps of the third and fourth methods have the same size. The blank areas on the edges of the feature maps are filled with 0s. The outputs of these two methods are then spliced ​​together by channel.

[0056] Step S307: Perform channel fusion again on the layer fusion result and the channel fusion result to obtain the final output layer;

[0057] Where m, n, and p are all positive integers, and m < n < p. It should be noted that m can be 1, mainly to change the dimension and incidentally perform feature transformation; n and p are both odd numbers because convolution kernels are always odd.

[0058] In the above implementation, the order of steps S301 to S302 is not specifically limited, as long as the layer fusion between the two methods can be completed; the order of steps S304 to S305 is not specifically limited, as long as the channel splicing between the two methods can be completed; the order of steps S303 and S306 is not specifically limited, as long as the channel fusion between the two fusion results can be completed.

[0059] like Figure 2 As shown, four methods are illustrated below. The first method inputs a 1x1 convolution and then concatenates it with a 3x3 convolution. The second method inputs a 1x1 convolution and then concatenates it with a 3x3 convolution. Both methods have the same number of output channels, and feature fusion is performed layer by layer. The third method inputs a 3x3 max-pooling layer and then concatenates it with a 1x1 convolution. The fourth method inputs a 3x3 max-pooling layer and then concatenates it with a 1x1 convolution. The output feature maps of the third and fourth methods have the same size, and the edges of the feature maps are padded with zeros. The outputs of these two methods are concatenated by channel. The results of layer fusion and channel fusion are then fused again by channel to obtain the final output layer.

[0060] For two input paths with the same number of channels and subsequent convolutions, since the convolution kernel for each output channel is independent, we can consider only the output of a single channel. Assume the two input paths have channels X1, X2, ..., X... cand Y1, Y2, ..., Y c The i-th convolution kernel uses K i express.

[0061] Z 通道融合 This represents a single channel calculated using the channel fusion method. Therefore, a single output channel obtained through channel fusion is (* indicates convolution):

[0062]

[0063] Z 层融合 This represents a single channel calculated using the layer-by-layer fusion method. Therefore, a single output channel of the layer-by-layer fusion method is (* indicates convolution):

[0064]

[0065] As an optional implementation, when the number of layers in the layer fusion result and the channel fusion result are the same, the two fusion results are fused to obtain the output layer.

[0066] As an optional implementation, the bounding box regression loss of the YOLOv3 model is adjusted to Smooth L1 loss. The error of Smooth L1 loss is L2 loss in the (-1,1) interval and L1 loss in the (-∞,-1] and [1,+∞) intervals, thus solving the gradient explosion problem during gradient update.

[0067] The functional expression for Smooth L1 loss is as follows:

[0068]

[0069] As an optional implementation, a pre-trained model is obtained by pre-training using the improved YOLOv3 model, specifically as follows:

[0070] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. The present invention is not limited to the specific steps and structures described above and shown in the figures. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here.

[0071] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art without departing from the scope of the invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A multi-scale fusion blade defect detection method based on improved YOLOv3, characterized in that, The method includes the following steps: Collect defect sample data of the internal blades of the engine; The defect sample data is preprocessed to obtain blade defect data; The number of channels in the backbone network of the YOLOv3 model is reduced, and the features extracted by the backbone network are fed into the feature fusion layer for multi-scale fusion to obtain the improved YOLOv3 model. The improved YOLOv3 model was used for pre-training to obtain a pre-trained model; Load the pre-trained model and train it on the blade defect data until the pre-trained model converges; The step of sending the features extracted by the backbone network into the feature fusion layer for multi-scale fusion specifically involves the feature fusion layer extracting features from the feature layer output by the backbone network in at least two ways, and then concatenating the processed results by channel to achieve multi-scale feature fusion. The feature fusion layer extracts features from the feature layer output by the backbone network in at least two ways, and concatenates the processed results by channel to achieve multi-scale feature fusion, specifically including the following steps: The first method uses an m*m input convolution, followed by an n*n convolution. The second method uses an m*m convolution as input, followed by a p*p convolution. The first method and the second method have the same number of output channels, and feature fusion is performed according to layers; The third approach involves connecting an n*n maximum pooling layer and then concatenating it with an m*m convolutional layer. The fourth approach involves connecting a p*p maximum pooling layer and then concatenating it with an m*m convolutional layer. The output feature maps of the third and fourth methods are the same size, and the blank areas at the edges of the feature maps are filled with 0. The outputs of these two methods are then spliced ​​together by channel. The layer fusion result and the channel fusion result are then fused again to obtain the final output layer; Where m, n, and p are all positive integers, and m < n < p.

2. The multi-scale fusion blade defect detection method based on improved YOLOv3 according to claim 1, characterized in that, The specific steps involved in collecting defect sample data from the engine's internal blades are as follows: The defect sample data is acquired using a probe device, and the acquisition type can be images or videos.

3. The multi-scale fusion blade defect detection method based on improved YOLOv3 according to claim 1, characterized in that, The preprocessing of the defect sample data to obtain blade defect data specifically involves: The defect sample data is labeled, enhanced, and normalized to obtain the blade defect data.

4. The multi-scale fusion blade defect detection method based on improved YOLOv3 according to claim 1, characterized in that, When the number of layers in the layer fusion result is the same as the number of layers in the channel fusion result, the two fusion results are fused to obtain the output layer.

5. The multi-scale fusion blade defect detection method based on improved YOLOv3 according to claim 1, characterized in that, The bounding box regression loss of the yolov3 model was adjusted to SmoothL1 loss.

6. The multi-scale fusion blade defect detection method based on improved YOLOv3 according to claim 1, characterized in that, The process of pre-training using the improved YOLOv3 model to obtain the pre-trained model is as follows: The improved YOLOv3 model was pre-trained using the COCO2017 dataset to obtain the pre-trained model.

Images