A method, device, medium and equipment for detecting damage to an aeroengine

Abstract

The present invention discloses a method, device, medium and equipment for detecting damage to an aeroengine, relating to the technical field of aeroengine damage detection. First, obtain aeroengine damage images as a sample data set and label each sample data. Then, replace the backbone network of the YOLOv5m model with the FasterNet network, apply depthwise separable convolution in the neck network of the YOLOv5m model and introduce the GS BottleNeck module to construct a damage detection model. Next, use the constructed damage detection model to detect damage to the sample data to obtain the predicted damage type and predicted damage location of each sample data. Finally, train the damage detection model according to the gap between the labeled true damage type and true damage location and the prediction result, and perform aeroengine damage detection through the trained damage detection model. The present invention effectively reduces the computational amount of the damage detection model, enabling the damage detection model to have high detection accuracy while ensuring lightweight.

Description

Technical Field

[0001] This application relates to the technical field of aero-engine damage detection, and particularly relates to a method, device, medium, and equipment for aero-engine damage detection. Background Art

[0002] Generally, as the heart of an aircraft, an aero-engine plays a decisive role in the safe operation of the aircraft. The core components of an aero-engine work in an environment of high temperature, high pressure, and high speed for a long time, and are prone to damages such as cracks, curled edges, and material loss. Early detection and accurate assessment of these damages are of great significance for ensuring the reliability of the engine, extending its life, and avoiding potential catastrophic accidents. At present, the detection of internal damages of an aero-engine mainly relies on borescope detection technology, but borescope detection highly depends on the professional skills and experience of personnel and has a slow detection speed. Automated damage detection methods can improve the detection efficiency and avoid the interference of human factors. Therefore, it is of great significance to achieve real-time and accurate automatic detection of aero-engine damages.

[0003] Deep learning is a machine learning method that simulates the human brain neural network. It obtains damage features by learning a large number of sample data, and extracts damage targets based on the features learned from the sample rules. The essence of deep learning is a neural network composed of multiple layers. Each layer processes the input data, and through non-linear transformation, converts the original image into high-level feature information. The network structure makes full use of the feature information of different layers, solving the problem of weak feature extraction ability of traditional methods.

[0004] In recent years, deep learning has gradually been applied to the automatic detection of aero-engine damages. The object detection models based on deep learning are mainly divided into two categories: two-stage object detection models and one-stage object detection models. In the two-stage object detection models, there is an existing method that uses Faster R-CNN to detect cracks and dents on turbine blades, with an average accuracy rate reaching 96.1%. There is also a method that combines an improved Mask R-CNN with a convolutional block attention module to achieve the classification and pixel-level segmentation of aero-engine blade damages, where the accuracy rates of cracks and material loss can reach 98.81%. There is also a method that proposes an improved Faster R-CNN model to detect damages on the blade surface, achieving high-precision automatic damage detection on the blade surface.

[0005] Although the two-stage model has high detection accuracy, its detection process requires generating candidate regions first and then classifying and performing location regression on these candidate regions. This sequential processing method results in a slow detection speed and cannot meet the working requirements of real-time detection. One-stage object detection models represented by YOLO abandon the candidate region generation process and directly perform classification and location regression on the objects, so the inference speed is faster. Based on this, an improved YOLOv3 model is proposed to detect aero-engine damage. A pyramid structure with richer feature scales and more levels is constructed, and the average detection accuracy is improved by about 18.75%, but the detection speed decreases. There is also the use of an improved YOLOv4 model to detect aero-engine damage. A convolutional structure is added to the output layer of the feature extraction network and the spatial pyramid pooling, and the detection accuracy is improved by 4.55%. However, the convolutional structure introduces additional parameters and reduces the detection speed. There is also an improvement on the YOLOv4 model. A low-level feature fusion layer is constructed in the path aggregation network to fuse shallower features with deeper features and introduce depthwise separable convolutions, which improves the detection accuracy of the model by 3.43% and increases the detection speed by 31.03%, with the FPS reaching 38. There is also the use of an improved YOLOv4 model to detect internal damage of aero-engines. The backbone feature extraction network is replaced with a lightweight backbone feature extraction network, which significantly improves the detection speed, with the FPS reaching 48.8. There is also the use of a lightweight improved YOLOv4 model to detect aero-engine damage. The backbone network is replaced with MobileNetv3, depthwise separable convolutions are introduced and a convolutional attention module is added, which increases the detection speed by 1.6 times, with the FPS reaching 37, but the detection accuracy decreases by 3.55%.

[0006] It can be seen that existing high-precision deep learning detection models have a large number of parameters, occupy a large amount of memory space, and have high requirements for device computing power, and cannot meet the lightweight requirements of aero-engine damage detection. Existing lightweight deep learning detection models have a fast detection speed, but the lightweight improvements are all at the cost of sacrificing part of the model detection accuracy, resulting in a low model detection accuracy and unable to meet the detection accuracy requirements in aero-engine damage detection. Summary of the Invention

[0007] Based on this, it is necessary to provide an aero-engine damage detection method, device, medium and equipment for the above technical problems.

[0008] This specification adopts the following technical solutions:

[0009] This specification provides an aero-engine damage detection method, including:

[0010] Obtain aero-engine damage images as a sample data set, and label the true damage types and true damage locations of each sample data;

[0011] Replace the backbone network of the YOLOv5m model with the FasterNet network, use depthwise separable convolutions within the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure of the neck network with the GS Bottleneck module to construct a damage detection model;

[0012] Input each sample data into the constructed damage detection model for damage detection to obtain the predicted damage type and predicted damage location of each sample data;

[0013] Take minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location as the optimization objective to train the damage detection model; and perform aero-engine damage detection through the trained damage detection model.

[0014] Optionally, the use of depthwise separable convolutions within the CBS structure in the neck network of the YOLOv5m model specifically includes:

[0015] Replace the convolutions in the neck network of the YOLOv5m model with depthwise separable convolutions.

[0016] Optionally, the FasterNet network includes: an embedding layer, a merging layer, and FasterNet blocks, with the embedding layer or the merging layer located before the FasterNet blocks; among them, the embedding layer contains a convolutional kernel with a size of 4×4 and a stride of 4 and a batch normalization layer, the merging layer contains a convolutional kernel with a size of 2×2 and a stride of 2 and a batch normalization layer, and the FasterNet blocks contain an inverted residual structure composed of a partial convolutional layer and two pointwise convolutional layers, and also contain a batch normalization layer and an activation function between the two pointwise convolutional layers.

[0017] Optionally, the replacement of the backbone network of the YOLOv5m model with the FasterNet network specifically includes:

[0018] Replace the CBS structure and the C3 structure in the backbone network of the YOLOv5m model with the FasterNet network, and concatenate the SPPF structure in the backbone network of the YOLOv5m model.

[0019] Optionally, the neck network of the YOLOv5m model includes an FPN network and a PAN network;

[0020] The use of depthwise separable convolutions within the CBS structure in the neck network of the YOLOv5m model, and the replacement of the Bottleneck module in the C3 structure of the neck network with the GS Bottleneck module specifically includes:

[0021] Replace the convolution in the CBS structure in the FPN network with the GSConv convolution to form the GBS structure, and replace the Bottleneck module in the G3 structure in the FPN network with the GS Bottleneck module to form the GS G3 structure;

[0022] Replace the convolution in the CBS structure in the PAN network with the GSConv convolution to form the GBS structure, and replace the Bottleneck module in the C3 structure in the PAN network with the GS Bottleneck module to form the GS C3 structure;

[0023] Among them, the GSConv convolution is formed by combining the convolution in the CBS structure and the depthwise separable convolution through the shuffle mixing strategy. The GS Bottleneck module convolves the input through two GSConv convolutions, and at the same time processes the input through a CBS structure, and combines the convolution result and the processing result for output.

[0024] Optionally, marking the true damage positions of each sample data specifically includes:

[0025] Marking the horizontal axis coordinate of the true damage center after normalizing each sample data, the vertical axis coordinate of the true damage center after normalizing, the true damage width after normalizing, and the true damage height after normalizing.

[0026] Optionally, the true damage types and predicted damage types specifically include:

[0027] Chunk damage, curled edge damage, material missing damage, crack damage, deformation damage, ablation damage, corrosion damage, and dent damage.

[0028] This specification provides an aero-engine damage detection device, including:

[0029] An acquisition module, configured to acquire aero-engine damage images as a sample data set, and mark the true damage types and true damage positions of each sample data;

[0030] A construction module, configured to replace the backbone network of the YOLOv5m model with the FasterNet network, use depthwise separable convolution in the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure in the neck network with the GS Bottleneck module to construct a damage detection model;

[0031] A damage prediction module, configured to input each sample data into the constructed damage detection model for damage detection, and obtain the predicted damage types and predicted damage positions of each sample data;

[0032] A training and detection module, which is used to train a damage detection model with the optimization objectives of minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location, and perform aero-engine damage detection through the trained damage detection model.

[0033] This specification provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the above-mentioned aero-engine damage detection method.

[0034] This specification provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, where the processor implements the above-mentioned aero-engine damage detection method when executing the program.

[0035] The above at least one technical solution adopted in this specification can achieve the following beneficial effects:

[0036] In the present invention, the backbone network of the YOLOv5m model is replaced with the FasterNet network, and the FasterNet network extracts engine damage features at different levels and scales of aero-engine damage images to represent engine damage of different complexities. At the same time, depthwise separable convolutions are used in the CBS in the neck network of the YOLOv5m model, and the Bottleneck module of the C3 structure in the neck network is replaced with the GS Bottleneck module to fuse engine damage features at different levels and scales through the neck network.

[0037] Since the FasterNet network includes partial convolutional layers and pointwise convolutional layers, where the partial convolutional layers only perform convolution on a part of the input channels, effectively reducing the computational complexity of the model while retaining spatial information, and the pointwise convolutional layers perform convolution on each pixel point of the input, being able to make full use of the information of all channels, the two can effectively reduce the computational complexity and improve the output quality at the same time. The neck network, through the application of depthwise separable convolutions in the two structures, compared with ordinary convolution operations that need to be performed on each input channel and each channel requires a set of convolutional kernels, resulting in a large number of redundant parameters during the convolution process, the depthwise separable convolution applies a convolutional kernel to each channel of the input respectively through one-stage channel-wise convolution without changing the number of channels, and adjusts the number of channels of the result of the channel-wise convolution using 1×1 convolution through two-stage pointwise convolution, effectively reducing the model's computational load.

[0038] In summary, by improving the YOLOv5m model from both the backbone network and the neck network to obtain a damage detection model, the damage detection model has high detection accuracy while ensuring light weight. Description of the Drawings

[0039] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0040] Figure 1 It is a schematic flow chart of a method for detecting damage to an aero-engine provided in this specification;

[0041] Figure 2 It is a schematic diagram of sample data of a damage image provided in this specification;

[0042] Figure 3 It is a schematic diagram of the specific structure of a YOLOv5m model provided in this specification;

[0043] Figure 4 It is a schematic diagram of the specific structure of a C3 structure provided in this specification;

[0044] Figure 5 It is a schematic diagram of an SPPF structure provided in this specification;

[0045] Figure 6 It is a schematic diagram of the backbone network of a damage detection model provided in this specification;

[0046] Figure 7 It is a schematic diagram of a GSConv structure provided in this specification;

[0047] Figure 8 It is a schematic diagram of a GS C3 structure provided in this specification;

[0048] Figure 9 It is a schematic diagram of an aero-engine damage detection model provided in this specification;

[0049] Figure 10 It is a schematic diagram of the process comparison between ordinary convolution and DSConv provided in this specification;

[0050] Figure 11 It is a schematic diagram of a DBS structure provided in this specification;

[0051] Figure 12 It is a schematic diagram of a mobile aero-engine damage detection model provided in this specification;

[0052] Figure 13aSchematic diagram of the detection result of a lightweight damage detection model for ablation damage type provided in this specification;

[0053] Figure 13b Schematic diagram of the detection result of a lightweight damage detection model for dent damage type provided in this specification;

[0054] Figure 13c Schematic diagram of the detection result of a lightweight damage detection model for crack damage type provided in this specification;

[0055] Figure 13d Schematic diagram of the detection result of a lightweight damage detection model for material loss damage type provided in this specification;

[0056] Figure 14a Schematic diagram of the detection result of a damage detection model for ablation damage type based on a mobile Internet terminal provided in this specification;

[0057] Figure 14b Schematic diagram of the detection result of a damage detection model for dent damage type based on a mobile Internet terminal provided in this specification;

[0058] Figure 14c Schematic diagram of the detection result of a damage detection model for crack damage type based on a mobile Internet terminal provided in this specification;

[0059] Figure 14d Schematic diagram of the detection result of a damage detection model for material loss damage type based on a mobile Internet terminal provided in this specification;

[0060] Figure 15 Schematic diagram of an aeroengine damage detection device provided in this specification;

[0061] Figure 16 Schematic diagram of a computer device for implementing the aeroengine damage detection method provided in this specification. Specific embodiments

[0062] To make the purpose, technical solutions and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0063] At present, deep learning models still have many inapplicabilities in the damage detection of aero-engines. (1) Existing deep learning models have a large number of parameters, occupy a large amount of memory space, and have high requirements for the computing power of devices, which cannot meet the lightweight requirements for aero-engine damage detection. (2) Existing lightweight improved deep learning models have a relatively small number of parameters and a relatively fast detection speed, but the lightweight improvement is achieved at the cost of sacrificing part of the model's detection accuracy, resulting in a relatively low detection accuracy of the model, which cannot meet the requirements for detection accuracy in aero-engine damage detection.

[0064] In view of the shortcomings of existing deep learning models in aero-engine damage detection, the present invention proposes a method for aero-engine damage detection, which can meet the lightweight requirements and maintain a high detection accuracy in aero-engine damage detection.

[0065] The following will, with reference to the accompanying drawings, detail the technical solutions provided by the embodiments of the present application.

[0066] Figure 1 It is a schematic flow diagram of a method for aero-engine damage detection in this specification, which specifically includes the following steps:

[0067] S101: Obtain aero-engine damage images as a sample data set, and label the true damage types and true damage positions of each sample data.

[0068] Generally, when the server of the business platform conducts aero-engine damage detection services, it can first establish a sample data set, label each sample data, then construct a damage detection model, and train the damage detection model with the labeled sample data. Finally, the trained damage detection model can be applied to conduct aero-engine damage detection services.

[0069] Based on this, in one or more embodiments of this specification, the server can obtain aero-engine damage images collected historically to form a sample data set. For example, usually, aero-engine borescope inspection reports track and record the damage conditions of aero-engines for a long time, accumulating a large number of typical aero-engine damage images and videos. Therefore, the server can choose to collect aero-engine damage images from aero-engine borescope inspection reports as sample data.

[0070] After obtaining the sample data set, when training the object detection network, the aero-engine damage images are input into the damage detection model as sample data, and corresponding label information needs to be provided to adjust the weights of the damage detection model, so as to continuously improve the detection accuracy of the damage detection model. The original damage images do not have label information. Therefore, the server can further annotate each sample data, and generally can annotate its true damage type and true damage location. In one or more embodiments of this specification, the main damage types of aero-engines include chip damage, curled edge damage, material loss damage, crack damage, deformation damage, ablation damage, corrosion damage, and dent damage, etc. How to specifically annotate the aero-engine damage images can be determined according to needs, and this specification does not limit this. For example, the Labelimg software can be used to annotate the selected aero-engine damage images.

[0071] Of course, for some of these damage types, due to their low occurrence frequency, it may be difficult to collect enough sample data for model training. Therefore, in one or more embodiments of this specification, the server can also select some common damages among them to establish a sample data set and perform subsequent damage detection.

[0072] For example, the server can select four damages that often occur in engines, namely material loss damage, crack damage, ablation damage, and dent damage, to establish a sample data set. Some sample data are as Figure 2 shown, Figure 2 is a schematic diagram of sample data of a damage image in this specification, Figure 2 in which, from left to right are ablation damage, dent damage, crack damage, and material loss damage in sequence.

[0073] When annotating the true damage location of the sample data, in one or more embodiments of this specification, the true damage location of the sample data includes: the horizontal axis coordinate of the true damage center after normalization of the sample data, the vertical axis coordinate of the true damage center after normalization, the true damage width after normalization, and the true damage height after normalization.

[0074] For example, the labels of aero-engine damage images can be stored in txt format. Each line represents a damage object. The data in the first column represents the true damage type, 0 represents ablation damage, 1 represents dent damage, 2 represents crack damage, and 3 represents material loss damage. The data in the second column represents the X coordinate of the true damage center after normalization, the data in the third column represents the Y coordinate of the true damage center after normalization, the data in the fourth column represents the true damage width after normalization, and the data in the fifth column represents the true damage height after normalization. The data in the second column and the fourth column are normalized using the width of the original image, and the data in the third column and the fifth column are normalized using the height of the original image.

[0075] Further, in one or more embodiments of this specification, the server can divide the aero-engine damage sample dataset into a training set, a validation set, and a test set according to a set ratio. The dataset can be composed of two folders, images and labels. The images folder is used to store damage images, and the labels folder is used to store label files. Both folders contain three sub-folders, train, val, and test. The train folder is used to store training data, the val folder is used to store validation data, and the test folder is used to store test data. Subsequently, the damage detection model can be trained using the training set, the damage detection accuracy of the damage detection model can be verified using the validation set, and the damage detection model can be tested using the test set.

[0076] The server mentioned in this specification can be a server set up on the business platform, or a device such as a desktop computer or a laptop that can execute the solution of this specification. For the convenience of description, only the server is used as the execution subject for description below.

[0077] S102: Replace the backbone network of the YOLOv5m model with the FasterNet network, use depthwise separable convolution in the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure of the neck network with the GS Bottleneck module to construct a damage detection model.

[0078] After obtaining the sample dataset and completing the annotation as described above, the server can construct a damage detection model and train the damage detection model.

[0079] For the construction of the damage detection model, YOLOv5 is an efficient and fast one-stage object detection model with a total of 5 versions. The models are arranged from small to large as YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. As the model becomes more complex, the detection accuracy will increase accordingly. Although YOLOv5n and YOLOv5s have fewer parameters, due to their small model size, their feature extraction ability is insufficient, resulting in low detection accuracy. YOLOv5l and YOLOv5x have high detection accuracy, but the model is too large to meet the requirement of model lightweight. Therefore, the YOLOv5m model, which balances detection accuracy and detection speed, is selected as the benchmark model.

[0080] The YOLOv5m model mainly consists of three parts: a backbone network, a neck network, and a detection head, as Figure 3 shown.

[0081] Figure 3 is a specific structural schematic diagram of a YOLOv5m model in this specification, byFigure 2 It can be seen that the backbone network mainly includes a CBS structure, a C3 structure, and an SPPF (Spatial Pyramid Pooling–Fast) structure.

[0082] Among them, the CBS structure consists of a common convolution, a batch normalization layer, and a SiLU activation function.

[0083] The C3 structure divides the input into two parts. One part undergoes convolution through a path containing N Bottleneck structures, and the other part passes through a residual edge. Then, the outputs of the two are combined by Concat, enabling information exchange and fusion between feature maps at different stages, increasing the model's expressive ability and receptive field, as Figure 4 shown. Figure 4 This is a schematic diagram of the specific structure of a C3 structure in this specification.

[0084] The SPPF is improved from the SPP (Spatial Pyramid Pooling) structure. The SPP structure applies max-pooling with different scales to the input feature map in parallel, capable of obtaining feature information from different scales and enhancing the model's perception ability for targets of different scales. The SPPF structure, on the other hand, uses a serial method, connecting three max-pooling operations with the same scale in series, while maintaining consistency with the output result of the SPP structure and improving the detection speed, as Figure 5 shown. Figure 5 This is a schematic diagram of an SPPF structure in this specification.

[0085] The damage detection model in the present invention is constructed based on the YOLOv5m model and consists of three parts: a backbone network, a neck network, and a detection head network connected in series. Among them, the server can replace the backbone network of the YOLOv5m model with the FasterNet network. Of course, in one or more embodiments of this specification, the server can also replace the CBS structure and the C3 structure in the backbone network of the YOLOv5m model with the FasterNet network and connect the SPPF structure in the backbone network of the YOLOv5m model. That is, the backbone network of the damage detection model uses the FasterNet network to replace the backbone network of YOLOv5m and retains the SPPF structure, as Figure 6 shown. Figure 6 This is a schematic diagram of the backbone network of a damage detection model in this specification, consisting of Figure 6 It can be seen that the FasterNet network includes an embedding layer, a merging layer, and FasterNet blocks.

[0086] Table 1 FasterNet model structure parameter table

[0087]

[0088] Table 1 is a table of the structural parameters of a FasterNet model in this specification, and Table 1 is the corresponding Figure 6 explanation. In Table 1, Conv_k_c_s represents a convolution with a kernel size of k, an output channel number of c, and a stride of s. BN represents a batch normalization layer (Batch Normalization, BN). PConv_k_c_s_r represents a partial convolution with a kernel size of k, an output channel number of c, a stride of s, and a partial ratio of r. r represents the ratio of the number of channels for which operations are performed in the partial convolution to the number of input channels.

[0089] Combined with Figure 6 and Table 1, it can be seen that the FasterNet network includes: an embedding layer, a merging layer, and a FasterNet block. The embedding layer or the merging layer is located before the FasterNet block and is used for spatial downsampling and channel number expansion.

[0090] Among them, the embedding layer contains a convolution kernel with a size of 4×4 and a stride of 4 and a batch normalization layer. The merging layer contains a convolution kernel with a size of 2×2 and a stride of 2 and a batch normalization layer. The FasterNet block contains an inverted residual structure composed of a partial convolution (PartialConvolution, PConv) layer and two pointwise convolution (Pointwise Convolution, PWConv) layers. A batch normalization layer and an activation function are also included between the two pointwise convolution layers, thus achieving lower latency and faster inference while ensuring feature diversity.

[0091] PConv only uses ordinary convolution to extract the spatial features of the engine damage image on only part of the input channels and keeps the remaining channels unchanged, ensuring that the input and output have the same number of channels, and regarding the first or last continuous channels as representatives of the entire feature map. By only performing convolution on a part of the input channels, PConv effectively reduces the computational complexity of the model while retaining spatial information.

[0092] For example, assuming that r is the ratio of the number of channels for which operations are performed in the PConv to the number of input channels, h is the height of the input feature map, w is the width of the input feature map, k is the kernel size, and c p is the number of channels for which operations are performed in the partial convolution. When r = c p / c = 1 / 4, the number of floating-point operations (Giga Floating-point Operations, GFLOPs) of PConv is:

[0093]

[0094] Only 1 / 16 of that of ordinary convolution.

[0095] The memory access volume of PConv is as follows:

[0096]

[0097] It is only 1 / 4 of that of ordinary convolution. It can be seen that compared with ordinary convolution, PConv effectively reduces GFLOPs and memory access volume, and realizes the reduction of the model's computational complexity by introducing PConv.

[0098] PWConv uses a 1×1 convolutional kernel to perform convolution on each pixel of the input, and can make full use of the information of all channels. The combined structure of PConv and PWConv has a T-shaped effective receptive field on the input feature map, and emphasizes the central position more than the ordinary convolution with unified processing. Using the combination of PConv and PWConv can not only effectively reduce the computational complexity, but also improve the output quality.

[0099] SPPF is improved from SPP (Spatial Pyramid Pooling, SPP). It changes the maximum pooling of three different scales in parallel in the SPP structure to the maximum pooling of three identical scales in series, and improves the detection speed while ensuring the same output result as SPP.

[0100] At the same time, when establishing the damage detection model, the server can also use depthwise separable convolution in the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure in the neck network with the GSBottleNeck module.

[0101] Specifically, in one or more embodiments of this specification, the neck network of the YOLOv5m model includes an FPN network and a PAN network. The neck network of the damage detection model in the present invention also includes an FPN network and a PAN network, which are used to fuse engine damage features of different scales and multiple levels, generate an engine damage feature map with multi-scale information, and improve the detection accuracy of the damage detection model.

[0102] The server can replace the convolution in the CBS structure in the FPN network with the GSConv convolution to form the GBS structure, and replace the Bottleneck module in the G3 structure in the FPN network with the GS Bottleneck module to form the GS G3 structure. Similarly, for the FPN network, the server can replace the convolution in the CBS structure in the PAN network with the GSConv convolution to form the GBS structure, and replace the Bottleneck module in the C3 structure in the PAN network with the GS Bottleneck module to form the GS C3 structure. Among them, the GSConv convolution is formed by combining the convolution in the CBS structure and the depthwise separable convolution (DSConv) through the shuffle mixing strategy. The GS Bottleneck module performs convolution on the input through two GSConv convolutions, and at the same time processes the input through a CBS structure, and combines and outputs the convolution result and the processing result.

[0103] The FPN network constructs an engine damage feature pyramid through top-down feature fusion. The top-down engine damage feature fusion fuses low-level features with high-level features through upsampling and fusion operations to obtain richer semantic information and improve the detection performance of the damage detection model.

[0104] In one or more embodiments of this specification, the FPN network in the neck network of the damage detection model in the present invention mainly may include the GBS structure, upsampling, Concat, and the GS C3 structure. The PAN network mainly may include the GBS structure and the GS C3 structure.

[0105] The GBS structure is improved from the CBS structure by replacing the ordinary convolution in the CBS structure with the GSConv convolution. Each time the engine damage feature map compresses in space (width and height) and expands the channel feature map, it will cause the loss of some feature information. The ordinary convolution maximally retains the hidden connections between each channel, and the DSConv convolution completely cuts off these connections. The GSConv convolution combines the ordinary convolution and the DSConv convolution by introducing the shuffle mixing strategy. The shuffle mixing strategy completely mixes the information extracted by the ordinary convolution into the output of the DSConv by uniformly exchanging local feature information on different channels, retaining the hidden connections between channels as much as possible, and enhancing the non-linear expression ability of the GSConv, as Figure 7 shown.

[0106] Figure 7 This is a schematic diagram of a GSConv structure in this specification. Among them, C1 represents the number of input channels, and C2 represents the number of output channels. From Figure 7It can be seen that "123456" and "abcdef" in the figure represent different feature layers.

[0107] Assume that W and H are the width and height of the output feature map, K1·K2 is the convolution kernel size, C1 is the number of channels of the input feature map, and C2 is the number of channels of the output feature map. The time complexities of ordinary convolution, DSConv, and GSConv are respectively:

[0108] Time SC ~O(W·H·K1·K2·C1·C2)

[0109] Time DSC ~O(W·H·K1·K2·1·C2)

[0110]

[0111] It can be seen that the time complexity of GSConv is between that of ordinary convolution and DSConv convolution. While maintaining a high non - linear expression ability, GSConv maintains a low time complexity, making the advantages of GSConv in lightweight networks more obvious. Replacing the ordinary convolution in the CBS structure with GSConv convolution can effectively reduce the time complexity of this structure.

[0112] The upsampling and Concat operations fuse the low - level features with the high - level features. High - level features contain more abstract semantic information, while low - level features contain more detailed information. By fusing these features together, the model becomes more adaptable and robust, capable of processing engine damage image content with different scales and complexities.

[0113] The GS C3 structure is improved from the C3 structure, replacing the BottleNeck module in the C3 structure with the GSBottleNeck module, as Figure 8 shown.

[0114] Figure 8 This is a schematic diagram of a GS C3 structure in this specification. It can be Figure 8 seen that the GS C3 structure divides the input into two parts. One part passes through a path containing N GSBottleNeck modules for convolution, and the other part passes through a residual edge. Then, the outputs of the two are combined through Concat, enabling information exchange and fusion between feature maps at different stages, increasing the model's expression ability and receptive field. The GSBottleNeck module constructed based on the CBS structure and GSConv convolution divides the input into two parts. One part undergoes convolution operations through two GSConv convolutions, and the other part passes through the CBS structure. Then, the outputs of the two are combined, enabling information exchange and fusion between different engine damage feature maps.

[0115] The detection head of the damage detection model consists of output layers of three different scales. Each output layer is responsible for detecting target objects of different sizes, including target position, category, and confidence information.

[0116] Figure 9 This is a schematic diagram of an aero-engine damage detection model in this specification. Figure 9 In it, the backbone network of the improved damage detection model is within the left dashed box. The middle part within the dashed box is the improved neck network. Within the dashed box of the neck network, the dashed box marked with "B" is the improved GBS structure, and the dashed box marked with "C" is the improved GSC3 structure. The right dashed box is the detection head.

[0117] In the construction process of the above damage detection model, the backbone network of the YOLOv5m model is replaced with the FasterNet network, and the SPPF structure of YOLOv5m is retained. The backbone network is used to extract damage features from the input engine damage images, helping the model learn low-level and high-level features of the images. The embedding layer and the merging layer are used for spatial downsampling and channel number expansion. The FasterNet block is used to learn and extract damage features at different levels, enabling the model to better understand and represent engine damage of different scales and complexities. The SPPF can capture damage features at different scales, increase the receptive field, reduce information loss, and improve the model's adaptability to multi-scale damage. Compared with the original backbone network, the improved backbone network has a simpler structure, fewer parameters, and faster running speed.

[0118] And the convolution within the CBS structure in the neck network of the YOLOv5m model is replaced with GSConv convolution, and the Bottleneck module in the C3 structure of the neck network is replaced with the GS Bottleneck module. The neck network is used to fuse features of different levels and scales, providing the model with multi-scale features and more extensive context information, enabling it to more comprehensively understand the content of the damage images and improving the model's performance in complex scenarios and when facing engine damage of different sizes. Through the application of DSConv convolution in the two structures, replacing the convolution within the CBS structure with GSConv convolution reduces the number of parameters in the neck network, and replacing the Bottleneck module in the C3 structure with the GS Bottleneck module enhances the non-linear expression ability of the C3 structure.

[0119] Finally, the detection head consists of output layers of three different scales, which is used to accurately locate and classify the damage in the engine damage images. Each output layer is responsible for detecting damage objects of different sizes.

[0120] S103: Input each sample data into the constructed damage detection model for damage detection to obtain the predicted damage type and predicted damage location of each sample data.

[0121] S104: Take minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location as the optimization objectives, train the damage detection model, and perform aero-engine damage detection through the trained damage detection model.

[0122] After constructing the damage detection model, the server can input each sample data into the constructed damage detection model to extract aero-engine damage features for damage detection, so as to obtain the predicted damage type and predicted damage location of each sample data predicted by the damage detection model.

[0123] During the training process of the damage detection model, in order to enable the damage detection model to fully learn damage features, in one or more embodiments of this specification, the server can use methods such as rotation, scaling, and adding noise to perform data augmentation on the sample data set.

[0124] After that, the server can take minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location as the optimization objectives, and optimize the parameters of the damage detection model through multiple rounds of iterative training.

[0125] After completing the training of the damage detection model, the server can perform aero-engine damage detection through the trained damage detection model.

[0126] Based on Figure 1 The aero-engine damage detection method shown above, first obtain aero-engine damage images as a sample data set, and label each sample data. Then, replace the backbone network of the YOLOv5m model with the FasterNet network, use depthwise separable convolution in the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure in the neck network with the GS Bottleneck module to construct a damage detection model. Then, perform damage detection on the sample data through the constructed damage detection model to obtain the predicted damage type and predicted damage location of each sample data. Finally, train the damage detection model according to the gap between the labeled true damage type and true damage location and the prediction results, and perform aero-engine damage detection through the trained damage detection model.

[0127] In the present invention, the backbone network of the YOLOv5m model is replaced with the FasterNet network, and the FasterNet network is used to extract engine damage features at different levels of the aero-engine damage image to represent engine damage of different scales and different complexities. Since the FasterNet network consists of partial convolutional layers and pointwise convolutional layers, where the partial convolutional layers only perform convolution on a part of the input channels, effectively reducing the computational complexity of the model while retaining spatial information, and the pointwise convolutional layers perform convolution on each pixel point of the input, capable of making full use of the information of all channels, the FasterNet network composed of partial convolutional layers and pointwise convolutional layers can effectively reduce the computational complexity and improve the output quality at the same time.

[0128] Meanwhile, depthwise separable convolutions are used in the CBS within the neck network of the YOLOv5m model, and the Bottleneck module in the C3 structure of the neck network is replaced with the GS Bottleneck module to fuse engine damage features at different levels and scales through the neck network. By applying depthwise separable convolutions in the two structures within the neck network, compared with ordinary convolution operations that need to be performed on each input channel and each channel requires a set of convolutional kernels, resulting in a large number of redundant parameters during the convolution process, depthwise separable convolutions apply a convolutional kernel to each channel of the input through one-stage channel-wise convolution without changing the number of channels, and then adjust the number of channels of the result of channel-wise convolution using 1×1 convolution through two-stage pointwise convolution, effectively reducing the computational amount of the model.

[0129] The damage detection model is obtained by improving the YOLOv5m model from both the backbone network and the neck network, enabling the damage detection model to have high detection accuracy while ensuring lightweight.

[0130] When applying the aero-engine damage detection method provided in this specification, it is not necessary to execute according to Figure 1 the order of the steps shown. The specific execution order of each step can be determined as needed, and this specification does not limit this.

[0131] In addition, the above are all described with the server as the execution entity. In one or more embodiments of this specification, a mobile Internet terminal can also be used as the execution entity, that is, the mobile Internet terminal executes the aero-engine damage detection method provided in this specification. Of course, it can also be that after training the damage detection model on other devices, the trained damage detection model is deployed to the mobile Internet terminal, and the mobile Internet terminal performs aero-engine damage detection based on the trained damage detection model.

[0132] Generally, the volume of mobile Internet terminals is small and their usage range is wide, so there is a need to detect aero-engine damage through mobile Internet terminals. However, the memory of mobile Internet terminals is small and their computing power is low.

[0133] Therefore, in one or more embodiments of this specification, when the execution entity is a mobile Internet terminal, when using depthwise separable convolution in the CBS structure in the neck network of the YOLOv5m model in step S102, the convolution in the neck network of the YOLOv5m model can be replaced with depthwise separable convolution (DSConv).

[0134] Specifically, as described above, the neck network of the YOLOv5m model includes an FPN network and a PAN network. The FPN network mainly includes a CBS structure, upsampling, Concat, and C3 structure. The PAN network mainly includes a CBS structure and a C3 structure. That is, the ordinary convolution in the CBS structure in the FPN network and the PAN network can be replaced with DSConv convolution to form a DBS structure. Of course, the Bottleneck module in the C3 structure in the FPN network and the PAN network can also be replaced with a GS Bottleneck module to form a GSC3 structure. Similarly to the corresponding description above, it will not be elaborated here.

[0135] Then, the FPN network in the neck network of the damage detection model in the present invention mainly includes a DBS structure, upsampling, Concat, and GS C3 structure, and the PAN network mainly includes a DBS structure and a GS C3 structure.

[0136] Among them, the DBS structure is improved from the CBS structure, and the ordinary convolution in the CBS structure is replaced with depthwise separable convolution. Ordinary convolution operations need to be performed on each input channel, and each channel requires a set of convolution kernels, which will generate a large number of redundant parameters during convolution, which is not conducive to mobile applications. DSConv includes two stages. The first stage is depthwise convolution, in which a convolution kernel is applied to each input channel separately for convolution without changing the number of channels. The second stage is pointwise convolution, in which the result of depthwise convolution is adjusted for the number of channels using 1×1 convolution, which can effectively reduce the model calculation amount, as Figure 10 shown.

[0137] Figure 10 This is a schematic diagram comparing the processes of ordinary convolution and DSConv in this specification. D F is the input feature size, M is the number of input channels, D H is the output feature size, N is the number of output channels, D Kis the convolution kernel size, and the "1" in the pointwise convolution stage indicates that the convolution kernel size is 1.

[0138] The computational amount C1 of ordinary convolution is:

[0139] C1 = M × D K × D K × N × D F × D F

[0140] The computational amount C2 of DSConv is:

[0141] C2 = M × D K × D K × D F × D F + M × N × D F × D F

[0142] The ratio of the computational amount of DSConv to that of ordinary convolution is:

[0143]

[0144] It can be seen from the above formula that the computational amount of DSConv is much smaller than that of ordinary convolution. Figure 11 is a schematic diagram of a DBS structure in this specification. As Figure 11 can be seen, the DBS structure consists of depthwise separable convolution, batch normalization layer and SiLU activation function. By replacing the CBS structure in the neck network of the YOLOv5m model with the DBS structure, compared with the original CBS structure and the aforementioned GBS structure, it can further reduce the computational amount of the damage detection model, so as to facilitate the deployment of the damage detection model on mobile Internet terminals for application. When the mobile Internet terminal performs engine damage detection through this solution, it can ensure the detection speed with lightweight and have high detection accuracy at the same time.

[0145] Figure 12 is a schematic diagram of a mobile terminal aero-engine damage detection model in this specification, Figure 12 in which the left dashed box is the backbone network of the improved damage detection model. The middle dashed box is the improved neck network. Among them, the dashed box marked with "B" is the DBS structure of the improved damage detection model, and the dashed box marked with "C" is the improved GS C3 structure. The right dashed box is the detection head.

[0146] In addition, in one or more embodiments of this specification, the server or the mobile Internet terminal can also use the trained weight file to perform damage detection tests on the test set, conduct ablation experiments on the test results of the lightweight deep learning aero-engine damage detection model, compare the performance of each model, and select the damage detection model with the best performance from them. Subsequently, the trained or the best damage detection model obtained through comparison can be deployed to the server or the Internet mobile terminal to perform damage detection on the aero-engine damage images to be detected.

[0147] In addition, this specification also provides an application embodiment of the aero-engine damage detection model. In this embodiment, the mean average precision (mAP), the number of parameters, GFLOPs, and FPS are selected as the evaluation indicators of the model. Experiments are conducted on a device with an Intel Core i7-13700F 2.10GHz processor, an NVIDIA GeForce RTX 4060Ti graphics card, and 8GB of memory. mAP represents the average of the average precision (AP) of all damage categories when the Intersection over Union (IoU, representing the overlap degree between the predicted box and the ground truth box) threshold is 0.5. AP represents the area under the PR curve formed with the recall rate R (Recall) and the precision rate P (Precision) as the coordinate axes. The calculation formulas for the recall rate, precision rate, average precision, and mean average precision are as follows:

[0148] The following is the calculation formula for the recall rate:

[0149]

[0150] The following is the calculation formula for the precision rate:

[0151]

[0152] The following is the calculation formula for the average precision:

[0153]

[0154] The following is the calculation formula for the mean average precision:

[0155]

[0156] In the formula, TP represents the correctly predicted positive examples, FN represents the incorrectly predicted positive examples, FP is the incorrectly predicted negative examples, and c is the total number of damage categories.

[0157] First, in the same experimental environment and dataset, ablation experiments are designed to evaluate the aero-engine damage detection model.

[0158] Experiment a is to train and test YOLOv5m on the aero-engine damage dataset; experiment b is to use FasterNet as the backbone feature extraction network; experiment c is to use FasterNet as the backbone network and introduce GSConv into the neck network at the same time; experiment d is a lightweight deep learning aero-engine damage detection model improved by adding the GS BottleNeck module on the basis of experiment c. The detection performance comparison of the improved models is shown in Table 2. Table 2 is a schematic table for comparing the detection performance of different improvement methods in this specification.

[0159] Table 2 Schematic table for comparing the detection performance of different improvement methods

[0160] Model Number of parameters mAP / % GFLOPs FPS Experiment a 20,865,057 92 47.9 92 Experiment b 11,654,613 88.3 22.5 139 Experiment c 10,655,253 88.2 21.3 145 Experiment d 10,931,445 91 17.6 138

[0161] As can be seen from Table 2, the FasterNet network and GSConv can effectively reduce the model complexity. The number of parameters and GFLOPs of the damage detection models improved by these two lightweight methods are reduced to 1 / 2 of the original model, and the detection speed is increased by 53. Introducing the GS BottleNeck module improves the detection accuracy of the lightweight model. When the number of parameters increases slightly and the detection speed decreases slightly, the detection accuracy is increased by 2.8%, and at the same time, the GFLOPs is reduced by 3.7. Compared with the original model, the lightweight deep learning aero-engine damage detection model reduces the number of parameters to about 1 / 2 of the original model and the GFLOPs to about 1 / 3 of the original model while only losing 1% of the detection accuracy, and the detection speed is increased by 46.

[0162] To further verify the effectiveness of the model improvement, under the same experimental conditions, a comparative experiment was carried out on the aero-engine damage detection method based on the lightweight deep learning model with YOLOv3, YOLOv3-Tiny and YOLOv7-Tiny. The experimental results are shown in Table 3. Table 3 is a schematic table for comparing the detection performance of different models in this specification.

[0163] Table 3 Schematic table for comparing the detection performance of different models

[0164] Model Number of parameters mAP / % GFLOPs FPS YOLOv3 61,513,585 91.3 154.6 38 YOLOv3 - tiny 8,673,622 86.7 12.9 159 YOLOv7 - tiny 6,014,737 89.5 13 63 Experiment d 10,931,445 91 17.6 138

[0165] As can be seen from Table 3, in terms of detection accuracy, the mAP value of the improved damage detection model is close to that of YOLOv3, 4.3% higher than that of YOLOv3-tiny, and 1.5% higher than that of YOLOv7-tiny; further comparing the detection speed, the FPS value of the improved model is 3.63 times that of YOLOv3. Therefore, the damage detection model in the aero-engine damage detection method provided by the present invention takes into account both the detection speed and the detection accuracy, and the comprehensive performance is better than other models.

[0166] Figure 13a ~d show the detection results of different models. Among them, Figure 13a is a schematic diagram of the detection results of a lightweight damage detection model for an ablation damage type in this specification. Figure 13b is a schematic diagram of the detection results of a lightweight damage detection model for a dent damage type in this specification. Figure 13c is a schematic diagram of the detection results of a lightweight damage detection model for a crack damage type in this specification. Figure 13d is a schematic diagram of the detection results of a lightweight damage detection model for a material loss damage type in this specification. Figure 13a Columns 1 to 4 of ~d are the detection results of YOLOv3, YOLOv3-tiny, YOLOv7-tiny, and Experiment d respectively. The figures more intuitively compare the detection effects of different models.

[0167] From Figure 13a ~d, it can be seen that when detecting the ablation damage type, the prediction boxes of YOLOv3 and YOLOv7-tiny failed to completely cover the damaged area. YOLOv3-tiny generated two prediction boxes for one damage, splitting the damage in a continuous area into two parts. The prediction box of Experiment d accurately covered the damaged area. When detecting the dent damage type, YOLOv3 and YOLOv3-tiny missed detections, and YOLOv7-tiny had false detections. When detecting the crack damage type, the prediction box of YOLOv3 failed to completely cover the damaged area. Both YOLOv3-tiny and YOLOv7-tiny generated two prediction boxes for one damage, identifying one crack as two damages. When detecting the material loss damage type, YOLOv3, YOLOv3-tiny, and YOLOv7-tiny had different degrees of missed detections. Experiment d reduced the phenomena of missed detections and false detections, and had higher detection accuracy and better detection performance than other detection models.

[0168] Furthermore, this specification also provides an application embodiment of an aero-engine damage detection model based on a mobile Internet terminal. In this embodiment, the mean average precision (mAP), number of parameters, GFLOPs, and FPS are selected as the evaluation indicators of the model. Experiments are carried out on a device with an Intel Core i7-13700F 2.10GHz processor, an NVIDIA GeForce RTX4060Ti graphics card, and 8GB of memory. The corresponding descriptions of each evaluation indicator can refer to the previous embodiments and will not be elaborated here one by one.

[0169] First, in the same experimental environment and dataset, ablation experiments were designed to evaluate the aero-engine damage detection method based on the lightweight deep learning model. Experiment a was to train and test YOLOv5m on the aero-engine damage dataset; Experiment b was to use FasterNet as the backbone feature extraction network; Experiment c was to use FasterNet as the backbone network and introduce DSConv into the neck network at the same time; Experiment d was to add the GS BottleNeck module on the basis of Experiment c, and finally the improved model was obtained. The comparison of the detection performance of the improved model is shown in Table 4. Table 4 is a schematic table for comparing the detection performance of different improvement methods in this specification.

[0170] Table 4 Schematic table for comparing the detection performance of different improvement methods

[0171] Model Number of parameters mAP / % GFLOPs FPS Experiment a 20,865,057 92 47.9 92 Experiment b 11,654,613 88.3 22.5 139 Experiment c 9,633,429 88.1 20 150 Experiment d 9,909,621 90.5 16.3 142

[0172] As can be seen from Table 4, the FasterNet network and DSConv can effectively reduce the model complexity. The number of parameters and GFLOPs of the models improved by these two lightweight methods are reduced to 1 / 2 of the original model, and the detection speed is increased by 58. Introducing the GSBottleNeck module improves the detection accuracy of the lightweight model. Under the condition that the number of parameters increases slightly and the detection speed decreases slightly, the detection accuracy is increased by 2.4%, and at the same time, the GFLOPs is reduced by 3.7. Compared with the original model, the mobile lightweight deep learning aero-engine damage detection model reduces the number of parameters to about 1 / 2 of the original model and the GFLOPs to about 1 / 3 of the original model while only losing 1.5% of the detection accuracy, and the detection speed is increased by 50.

[0173] To further verify the effectiveness of the model improvement, under the same experimental conditions, a comparative experiment was carried out between the aero-engine damage detection method based on the mobile lightweight deep learning model and YOLOv3, YOLOv3-Tiny, and YOLOv7-Tiny. The experimental results are shown in Table 5. Table 5 is a schematic table for comparing the detection performance of different models in this specification.

[0174] Table 5 Schematic table for comparing the detection performance of different models

[0175] Model Number of parameters mAP / % GFLOPs FPS YOLOv3 61,513,585 91.3 154.6 38 YOLOv3 - tiny 8,673,622 86.7 12.9 159 YOLOv7 - tiny 6,014,737 89.5 13 63 Experiment d 9,909,621 90.5 16.3 142

[0176] As can be seen from Table 5, in terms of detection accuracy, the mAP value of the improved model is close to that of YOLOv3, 3.8% higher than YOLOv3-tiny, and 1% higher than YOLOv7-tiny; further comparing the detection speed, the FPS value of the improved model is 3.73 times that of YOLOv3. The aero-engine damage detection model based on the mobile Internet terminal takes into account both the detection speed and the detection accuracy, and its comprehensive performance is better than other models.

[0177] Figure 14a ~d gives the detection results of different models. Among them, Figure 14a is a schematic diagram of the detection results of a damage detection model for ablation damage type based on a mobile Internet terminal in this specification. Figure 14b is a schematic diagram of the detection results of a damage detection model for dent damage type based on a mobile Internet terminal in this specification. Figure 14c is a schematic diagram of the detection results of a damage detection model for crack damage type based on a mobile Internet terminal in this specification. Figure 14d is a schematic diagram of the detection results of a damage detection model for material loss damage type based on a mobile Internet terminal in this specification. Figure 14a ~d The first to fourth columns are the detection results of YOLOv3, YOLOv3-tiny, YOLOv7-tiny, and Experiment d respectively. The figures more intuitively compare the detection effects of different models.

[0178] From Figure 14a ~d, it can be seen that when detecting the ablation damage type, the prediction boxes of YOLOv3 and YOLOv7-tiny failed to completely cover the damaged area. YOLOv3-tiny generated two prediction boxes for one damage, splitting the damage in a continuous area into two parts. The prediction box of Experiment d accurately covered the damaged area; when detecting the dent damage type, all four models accurately detected the position and category of the dent. The confidence levels detected by YOLOv3, YOLOv3-tiny, YOLOv7-tiny, and Experiment d are 0.91, 0.82, 0.81, and 0.94 respectively. Obviously, the confidence level of the detection result of Experiment d is the highest; when detecting the crack damage type, the prediction box of YOLOv3 failed to completely cover the damaged area and there were false detections. The prediction box of YOLOv3-tiny failed to completely cover the damaged area. YOLOv7-tiny had missed detections; when detecting the material loss damage type, YOLOv3, YOLOv3-tiny, and YOLOv7-tiny had different degrees of missed detections. Experiment d reduced the phenomena of missed detections and false detections, and had higher detection accuracy. Its detection performance was better than other detection models.

[0179] This embodiment also designed a comparative experiment on a Jetson Orin Nano Internet mobile terminal to verify the detection performance of the model for aero-engine damage on mobile devices. The experimental results are shown in Table 6. Table 6 is a schematic table of the experimental results of a mobile terminal in this specification.

[0180] Table 6 Schematic Table of Experimental Results of Mobile Terminal

[0181] Model mAP / % FPS YOLOv3 91 45 YOLOv3 - tiny 85.4 76 Our model 89.6 61

[0182] As can be seen from Table 6, the mAP of the YOLOv3 model is 91%, slightly higher than the damage detection model provided by the present invention. However, the FPS can only reach 45, which has a certain gap from the data acquisition speed of 60 FPS of the mainstream borescope video, and does not meet the application requirements for deploying the damage detection model on mobile Internet terminals to detect aero-engine damage. The FPS of both YOLOv3-tiny and the damage detection model provided by the present invention can reach above 60, but the detection accuracy of the damage detection model provided by the present invention is 4.2% higher than that of YOLOv3-tiny. Generally speaking, the damage detection model provided by the present invention maintains a high detection accuracy while ensuring the detection speed on mobile devices.

[0183] In summary, compared with the YOLOv5m model, the aero-engine damage detection model constructed by the present invention has fewer parameters, less memory occupancy, lower requirements for device computing power, and faster detection speed, and can realize efficient and automatic detection of aero-engine damage on mobile Internet terminals. Compared with other lightweight deep learning models, the aero-engine damage detection method proposed by the present invention has higher detection accuracy. The aero-engine damage detection method proposed by the present invention realizes the automatic detection of aero-engine damage on mobile Internet terminals, has a high detection accuracy for aero-engine damage detection on mobile Internet terminals, and the detection speed can reach above 60 FPS. It reduces the dependence on the work experience of technicians, improves the efficiency of aero-engine damage detection, and reduces the maintenance cost.

[0184] The above is the aero-engine damage detection method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding aero-engine damage detection device, as Figure 15 shown.

[0185] Figure 15 Schematic diagram of an aero-engine damage detection device provided by this specification, including:

[0186] An acquisition module 201, configured to acquire aero-engine damage images as a sample data set, and label the true damage type and true damage location of each sample data;

[0187] A construction module 202, configured to use depthwise separable convolutions within the CBS structure in the neck network of the YOLOv5m model, and replace the Bottleneck module in the C3 structure in the neck network with a GS Bottleneck module to construct a damage detection model;

[0188] A damage prediction module 203, configured to input each sample data into the constructed damage detection model for damage detection, and obtain the predicted damage type and predicted damage location of each sample data;

[0189] The training and detection module 204 is used to train the damage detection model with the optimization objectives of minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location, and perform aero-engine damage detection through the trained damage detection model.

[0190] Optionally, the device is a mobile Internet terminal, and the mobile Internet terminal includes an acquisition module 201, a construction module 202, a damage prediction module 203, and a training and detection module 204. The construction module 202 replaces the convolution in the neck network of the YOLOv5m model with a depthwise separable convolution.

[0191] Optionally, the FasterNet network includes: an embedding layer, a merging layer, and FasterNet blocks. The embedding layer or the merging layer is located before the FasterNet blocks. Among them, the embedding layer includes a convolutional kernel with a size of 4×4 and a stride of 4 and a batch normalization layer, the merging layer includes a convolutional kernel with a size of 2×2 and a stride of 2 and a batch normalization layer, and the FasterNet block includes an inverted residual structure composed of a partial convolutional layer and two pointwise convolutional layers, and also includes a batch normalization layer and an activation function between the two pointwise convolutional layers.

[0192] Optionally, the construction module 202 replaces the CBS structure and the C3 structure in the backbone network of the YOLOv5m model with the FasterNet network, and concatenates the SPPF structure in the backbone network of the YOLOv5m model.

[0193] Optionally, the construction module 202 replaces the convolution in the CBS structure in the FPN network with a GSConv convolution to form a GBS structure, and replaces the Bottleneck module in the G3 structure in the FPN network with a GS Bottleneck module to form a GS G3 structure. It also replaces the convolution in the CBS structure in the PAN network with a GSConv convolution to form a GBS structure, and replaces the Bottleneck module in the C3 structure in the PAN network with a GS Bottleneck module to form a GS C3 structure. Among them, the GSConv convolution is formed by combining the convolution in the CBS structure and the depthwise separable convolution through a shuffle mixing strategy. The GS Bottleneck module performs convolution on the input through two GSConv convolutions, and at the same time processes the input through a CBS structure, and combines the convolution result and the processing result and then outputs.

[0194] Optionally, the true damage location of the sample data includes: the horizontal axis coordinate of the true damage center after normalization of the sample data, the vertical axis coordinate of the true damage center after normalization, the true damage width after normalization, and the true damage height after normalization. The true damage type and the predicted damage type include chip damage, curling damage, material loss damage, crack damage, deformation damage, ablation damage, corrosion damage, and dent damage.

[0195] For the specific limitations of the aero-engine damage detection device, reference can be made to the limitations of the aero-engine damage detection method described above, which will not be elaborated here. Each module in the above aero-engine damage detection device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0196] This specification also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 provided aero-engine damage detection method.

[0197] This specification also provides Figure 16 the structural schematic diagram of the computer device shown, as Figure 16 described, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, there may also be other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above Figure 1 provided aero-engine damage detection method.

[0198] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0199] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

Claims

1. A method for detecting damage to an aeroengine, characterized in that Including: Obtain the damage images of the aero-engine as a sample data set, and label the true damage types and true damage locations of each sample data; Replace the backbone network of the YOLOv5m model with the FasterNet network, and replace the convolutions in the FPN network and PAN network in the neck network of the YOLOv5m model with GSConv convolutions to form a GBS structure, and replace the Bottleneck module in the G3 structure of the FPN network and PAN network with a GS Bottleneck module to form a GS G3 structure, and construct a damage detection model; among them, the GSConv convolution is formed by combining the convolution in the CBS structure and the depthwise separable convolution through a shuffle mixing strategy, and the GS Bottleneck module convolves the input through two GSConv convolutions, and at the same time processes the input through a CBS structure, and combines the convolution result and the processing result and then outputs; Input each sample data into the constructed damage detection model for damage detection to obtain the predicted damage type and predicted damage location of each sample data; Take minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location as the optimization objective, and train the damage detection model; and perform aero-engine damage detection through the trained damage detection model.

2. The method for detecting damage to an aeroengine according to claim 1, characterized in that, Use depthwise separable convolution inside the CBS structure in the neck network of the YOLOv5m model, specifically including: Replace the convolution in the neck network of the YOLOv5m model with depthwise separable convolution.

3. The aviation engine damage detection method according to claim 1, wherein, The FasterNet network includes: an embedding layer, a merging layer and a FasterNet block, and the embedding layer or the merging layer is located before the FasterNet block; among them, the embedding layer contains a convolution kernel with a size of 4×4 and a stride of 4 and a batch normalization layer, the merging layer contains a convolution kernel with a size of 2×2 and a stride of 2 and a batch normalization layer, and the FasterNet block contains an inverted residual structure composed of a partial convolution layer and two pointwise convolution layers, and also contains a batch normalization layer and an activation function between the two pointwise convolution layers.

4. The aircraft engine damage detection method according to claim 1, wherein, The replacement of the backbone network of the YOLOv5m model with the FasterNet network specifically includes: Replace the CBS structure and C3 structure in the backbone network of the YOLOv5m model with the FasterNet network, and connect the SPPF structure in the backbone network of the YOLOv5m model in series.

5. The method for detecting damage to an aeroengine according to claim 1, wherein The true damage location of the sample data includes: the horizontal axis coordinate of the true damage center after normalization of the sample data, the vertical axis coordinate of the true damage center after normalization, the true damage width after normalization, and the true damage height after normalization; The true damage type and the predicted damage type include spalling damage, curling damage, material loss damage, crack damage, deformation damage, ablation damage, corrosion damage and dent damage.

6. An aircraft engine damage detection device, characterized in that, Including: An acquisition module, configured to obtain the damage images of the aero-engine as a sample data set, and label the true damage types and true damage locations of each sample data; A building module is used to replace the backbone network of the YOLOv5m model with the FasterNet network, replace the convolutions in the FPN network and the PAN network in the neck network of the YOLOv5m model with GSConv convolutions to form a GBS structure, and replace the Bottleneck module in the G3 structure of the FPN network and the PAN network with a GS BottleNeck module to form a GS G3 structure, thereby building a damage detection model; wherein, the GSConv convolution is formed by combining the convolution in the CBS structure and the depthwise separable convolution through a shuffle mixing strategy, and the GS BottleNeck module performs convolution on the input through two GSConv convolutions, simultaneously processes the input through a CBS structure, and combines and outputs the convolution result and the processing result; A damage prediction module is used to input each sample data into the built damage detection model for damage detection, and obtain the predicted damage type and predicted damage location of each sample data; A training and detection module is used to train the damage detection model with the optimization goal of minimizing the deviation between the predicted damage type and the true damage type, and minimizing the deviation between the predicted damage location and the true damage location; and perform aero-engine damage detection through the trained damage detection model.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 5 above is implemented.

8. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of claims 1 to 5 above is implemented.

Images