Lightweight deployment method based on borehole probing crack detection in aircraft engines

By introducing the GhostBottleneck module, CBAM attention mechanism and BiFPN in the YOLOv5 model, combined with the dark light enhancement algorithm, the problem of reduced accuracy in the detection of cracks of aircraft engine holes is solved, and efficient and accurate on the embedded platform is achieved object detection.

CN114998255BActive Publication Date: 2024-11-01NANJING TECH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210606883.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-31
Publication Date
2024-11-01
Estimated Expiration
2042-05-31

AI Technical Summary

Technical Problem

The existing lightweight target detection network is difficult to achieve real-time application on embedded platforms, and due to the sacrificing detection accuracy to reduce the calculation amount, it cannot meet the accuracy requirements of aircraft engine hole crack detection.

Method used

Using the YOLOv5 lightweight model based on GhostNet, the Bottleneck module in the original network is replaced by the GhostBottleneck module, combined with the CBAM attention mechanism and BiFPN, reduce convolution operations and improve multi-scale feature fusion, and propose non-uniform light image self-image for the uneven light phenomenon. Adapt to dark light enhancement algorithm.

Benefits of technology

While reducing network parameters and computing volume, the accuracy and detection speed of target detection are improved, and the detection accuracy and speed on embedded platforms are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114998255B_ABST
    Figure CN114998255B_ABST
Patent Text Reader

Abstract

A lightweight deployment method for crack detection based on boresight of an aircraft engine comprises the following steps: 1) acquiring images or videos of boresight cracks of an aircraft engine and forming a boresight crack image dataset; the video is a plurality of continuous frame images; 2) establishing a YOLOv5 convolutional neural network model based on GhostNet; 3) training an improved YOLOv5 convolutional neural network model; 4) testing the improved model with the trained weight file and the test dataset, performing an ablation experiment on the test results of the improved model, comparing the performance of each model, and selecting the YOLOv5 improved network model with the best performance; 5) deploying the improved network model to a Jetson Nano embedded platform, and detecting the boresight crack images and videos with the trained weight file; and finally evaluating the lightweight deployment of the improved model by comparing the crack detection accuracy and the single-frame image inference time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a lightweight deployment target detection method for an embedded platform, which reduces the parameter scale and computational complexity of the network while further ensuring the accuracy of target detection, and improves the detection speed and detection accuracy of an aircraft engine borehole crack detection network in actual embedded application scenarios. Background Art

[0002] At present, the work of crack detection in aircraft engine borehole probing is characterized by tight time, heavy tasks, and high detection requirements. Therefore, inefficient manual detection methods and high-cost server-side image and video recognition can no longer meet the actual engineering needs at this stage. At present, there is an urgent need for a lightweight network that can automatically detect targets on an embedded platform to relieve the pressure on detection personnel, reduce detection costs, and further improve detection speed and accuracy.

[0003] As one of the research directions in the field of computer vision, target detection can provide valuable information for the semantic understanding of images and videos. Its essence is to locate and classify targets. Today, target detection based on convolutional neural networks has surpassed traditional target detection algorithms with an absolute advantage. However, due to the complex structure and large amount of computation of convolutional neural networks, it is difficult to achieve real-time application on embedded platforms with limited computing power and sensitive to the storage size of AI models. In order to facilitate the deployment of intelligent detection algorithms on embedded devices, the industry has proposed many lightweight networks. However, since these lightweight networks in the industry reduce a large number of convolution operations and floating-point operations at the expense of their own detection accuracy, they still cannot meet the accuracy requirements in actual engineering.

[0004] In summary, the research on lightweight deployment algorithms that can reduce the parameter scale and computational complexity of the network while ensuring the accuracy of target detection has important practical significance for improving the reliability and convenience of aircraft engine borehole detection crack detection, as well as promoting the innovative application and technological progress of artificial intelligence in fault diagnosis. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a lightweight deployment method based on aircraft engine borehole detection crack detection, which further ensures the accuracy of target detection while reducing the parameter scale and computational complexity of the network, and is more suitable for deployment on an embedded platform.

[0006] The idea of ​​this method is:

[0007] First of all, compared with other lightweight deployment networks, the YOLOv5 algorithm has the problems of large network model, large amount of calculation, and low operating efficiency. A YOLOv5 lightweight model based on GhostNet is proposed. The GhostBottleneck module is used to replace the Bottleneck module in the original network C3 structure. Some simple linear operations are used to replace some convolutions to reduce convolution operations, thereby reducing the number of parameters and floating-point operations.

[0008] Then, in order to solve the problem of accuracy degradation caused by replacing the lightweight model, the CBAM attention mechanism is introduced in the newly added GhostBottleneck module, combining the channel attention mechanism and the spatial attention mechanism to enhance the model's attention to effective features. In addition, in the original network Neck structure, BiFPN is used instead of PANet to achieve more efficient multi-scale feature fusion, increase reasoning speed and detection accuracy.

[0009] Finally, to address the uneven illumination phenomenon in actual borehole crack detection, an adaptive dark light enhancement algorithm for non-uniform illumination images is proposed to further solve the problem of accuracy loss caused by replacing lightweight structures.

[0010] The final improved model reduces the parameter scale and computational complexity of the network while further ensuring the accuracy of target detection and improving the detection speed and accuracy of the network in an embedded environment.

[0011] The specific technical solutions of the present invention are as follows:

[0012] Step 1: Collect images and video data of aircraft engine borescope cracks at China Southern Airlines, then annotate the collected data images, use a rectangular frame to mark the location of the cracks, and mark the image category as cracks. Finally, perform data enhancement on the collected images to expand the data set; the above operations form a borescope crack data set;

[0013] Step 2, construct an improved YOLOv5 convolutional neural network;

[0014] Step 2.1, in the Backbone network part of YOLOv5, use the GhostBottleneck module with stride=1 to replace the Bottleneck module in the original network C3 structure;

[0015] Step 2.2, add the CBAM attention mechanism to GhostBottleneck to form an improved C2Ghost module;

[0016] Step 2.3, in the Neck network part of YOLOv5, use BiFPN to replace PANet in the original network;

[0017] Step 3, train the improved YOLOv5 convolutional neural network;

[0018] Set training parameters: initial learning rate, impulse size, input image size, batch size, and number of iterations;

[0019] The borehole crack dataset is used. When loading the image data, the dark light enhancement algorithm is used to enhance the dark light of the dataset, and then the improved YOLOv5 convolutional neural network is trained.

[0020] Step 4: Use the trained weight file and the test data set to test, conduct an ablation experiment on the test results of the improved model, compare the performance of each model, and thus obtain the final YOLOv5 improved network;

[0021] Step 5: Deploy the improved network model to the Jetson Nano embedded platform, use the trained weight file to detect borehole crack images and videos, and use TensorRT for acceleration. By comparing the accuracy of crack detection and the inference time of a single frame image, the lightweight deployment of the improved model is finally evaluated.

[0022] Furthermore, the data enhancement method in step 1 includes:

[0023] Mosaci data enhancement was used to scale and randomly distribute the four borehole crack images;

[0024] Add noise to the borescope crack image;

[0025] The CutMix method is used to combine two borehole crack images.

[0026] Furthermore, the GhostBottleneck module in step 2.1 consists of two stacked Ghost modules. The first Ghost module is used as an expansion layer to increase the number of channels. The second Ghost module reduces the number of channels to match the shortcut channel number. The stride=1 structure uses a shortcut to connect the input and output of the two Ghost modules. The second Ghost module does not use the ReLU activation function, while other layers apply batch normalization (BN) and ReLU activation functions after each layer. The stride=2 structure is that the two Ghost modules are connected through a deep convolution (DWConv), and then the input and output of the two Ghost modules are connected using a shortcut. This method uses GhostBottleneck with stride=1 to replace the more complex Bottleneck module in the C3 structure in the original network.

[0027] Furthermore, the above Ghost module algorithm flow is as follows:

[0028] First, a small number of convolution operations are performed to obtain the essential feature maps, and then linear transformations are applied to these essential feature maps to obtain Ghost feature maps (redundant feature maps). Finally, the essential feature maps and redundant feature maps are cascaded to obtain the same number of feature maps as the output of the ordinary convolution layer.

[0029] Furthermore, the implementation process of the CBAM attention mechanism in step 2.2 is as follows:

[0030] In the first step, the input data is subjected to global maximum pooling and global average pooling to obtain two feature maps, which are based on the channel attention mechanism. In the second step, the two feature maps are concatenated and reduced to one channel. In the third step, a spatial attention feature map is generated through the Sigmoid activation function. In the fourth step, the feature map in the third step is multiplied by the input feature map of the module to generate the final feature map.

[0031] Furthermore, the implementation of the C2Ghost module based on the CBAM improvement described in step 2.2 includes the following sub-steps:

[0032] Step 2.2.1, add the CBAM attention mechanism after the two Ghost modules in GhostBottleneck;

[0033] In step 2.2.2, a Ghost module and a CBAM attention mechanism are added to the shortcut path.

[0034] Furthermore, the implementation of BiFPN in step 2.3 includes the following sub-steps:

[0035] Step 2.3.1, remove the nodes with only one input in PANet to reduce the computational overhead;

[0036] Step 2.3.2, take the top-down and bottom-up bidirectional paths of PANet as the base layer and repeat it multiple times;

[0037] Step 2.3.3, an additional connection edge is added between the input and output nodes of the same level in a similar way to the residual connection;

[0038] In step 2.3.4, a weighted fusion method is used to fuse feature layers of different resolutions, and the contribution of features in feature fusion is adjusted through trainable weights.

[0039] Furthermore, the training parameters are set in step 3:

[0040] The optimal initial learning rate is 0.01, the impulse size is 0.937, the input image size is 1024×1024, the batch size is 32, and the number of iterations is 200.

[0041] Furthermore, the dark light enhancement algorithm process in step 3 is as follows:

[0042] Retinex theory is used to separate the brightness component and reflection component of the image, adaptive gamma correction is performed on the brightness component, and finally the reflection component is used to restore the details and color of the image.

[0043] Furthermore, the process of using TensorRT to accelerate reasoning in step 5 is as follows:

[0044] The improved network model file is converted into a model. The conversion process is pt format file, wts format file, engine format file, and finally the engine file is inferred.

[0045] The beneficial effects of the present invention are:

[0046] This technical solution focuses on the lightweight deployment of aircraft engine borehole crack detection. While reducing the parameter scale and computational complexity of the network, it further ensures the accuracy of target detection and makes the target detection model easier to deploy on embedded platforms.

[0047] The technical reasons for the above achievements of this technical solution are:

[0048] 1) Use the GhostBottleneck module to replace the Bottleneck module in the original network C3 structure, and use some simple linear operations to replace some convolutions, thereby reducing the number of convolution operations and the number of parameters and floating-point operations.

[0049] 2) The CBAM attention mechanism is introduced into the newly added GhostBottleneck module, combining the channel attention mechanism and the spatial attention mechanism to enhance the model's attention to effective features and improve the accuracy of target detection.

[0050] 3) Use BiFPN to replace PANet in the original network to more efficiently achieve multi-scale feature fusion and increase model detection accuracy and inference speed.

[0051] 4) Aiming at the uneven illumination phenomenon existing in actual borehole crack detection, an adaptive dark light enhancement algorithm for non-uniform illumination images is proposed to further solve the problem of accuracy loss caused by replacing lightweight structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0053] Figure 1 This is a flow chart of the lightweight deployment method of this method based on aircraft engine borehole detection crack detection.

[0054] Figure 2a to Figure 2b It is an algorithm flow chart of the Ghost module of the embodiment of the method.

[0055] Figure 3 It is a structural diagram of the CBAM attention mechanism of an embodiment of the present method.

[0056] Figure 4a to Figure 4c This is the implementation process of the C2Ghost module in the embodiment of the method.

[0057] Figure 5a to Figure 5c It is a schematic diagram of the structure of the BiPFN of the embodiment of the method optimizing the feature network based on PANet.

[0058] Figure 6 This is a diagram of the improved YOLOv5s network architecture based on a lightweight deployment method for aircraft engine borehole crack detection in an embodiment of the present method. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0060] The invention provides a lightweight deployment method for crack detection based on aircraft engine borehole probing. The method reduces the parameter scale and computational complexity of the network model while further ensuring the accuracy of target detection.

[0061] The specific implementation of the present invention is as follows:

[0062] like Figure 1 As shown, a lightweight deployment method based on aircraft engine borehole detection crack detection includes the following steps:

[0063] Step 1: collect images and video data of aircraft engine borehole cracks at China Southern Airlines (collected on-site at China Southern Airlines in this embodiment), and then annotate the collected data images, use a rectangular frame to mark the location of the cracks, and mark the image category as cracks. After all images are annotated, export the corresponding category label file in the pascal-voc format.

[0064] In view of the problem of insufficient borescope crack image data set, the characteristics of the actual detection scene are fully considered, and the borescope crack image data set is amplified by using Mosaci, adding noise and CutMix. Considering the characteristics of many small crack targets in borescope crack images, Mosaci data enhancement is selected, and the background and small targets of the detection objects are enriched by randomly using pictures for scaling and randomly distributed splicing; considering that noise will be generated during the shooting and transmission of borescope crack images and videos, the data set is preprocessed by adding noise; at the same time, in order to improve the model's recognition ability of occluded objects, the CutMix method is used to randomly combine two borescope crack images.

[0065] Finally, the borescope crack image dataset and the category labels generated by the labeling work are divided into a training set and a test set, of which 906 images are used as a training set and 72 images are used as a test set, and the image size is 1024×1024.

[0066] The above operations are completed to form the borehole detection crack image dataset.

[0067] Step 2, construct an improved YOLOv5 convolutional neural network;

[0068] After preparing the data set, in view of the shortcomings of the original YOLOv5s algorithm, the network structure of the YOLOv5s algorithm will be improved from two aspects: improving the accuracy of borehole detection crack target detection and reducing the parameter scale and computational complexity of the detection network. For this purpose, an improved YOLOv5s algorithm based on GhostNet, attention mechanism, BiFPN and dark light enhancement algorithm is proposed.

[0069] Specifically, step 2 includes the following sub-steps:

[0070] Step 2.1, in the Backbone network part of YOLOv5, use the GhostBottleneck module with stride=1 to replace the Bottleneck module in the original network C3 structure;

[0071] The feature GhostBottleneck module consists of two stacked Ghost modules. The first Ghost module is used as an expansion layer to increase the number of channels. The second Ghost module reduces the number of channels to match the shortcut channel number. The stride=1 structure uses a shortcut to connect the input and output of the two Ghost modules. The second Ghost module does not use the ReLU activation function, while other layers apply batch normalization (BN) and ReLU activation functions after each layer. The stride=2 structure is that the two Ghost modules are connected through a deep convolution (DWConv), and then the input and output of the two Ghost modules are connected using a shortcut. For the implementation process of the GhostBottleneck module, please refer to Figure 4a to Figure 4b .

[0072] This method uses GhostBottleneck with stride=1 to replace the more complex Bottleneck module in the C3 structure in the original network.

[0073] The algorithm flow of the above Ghost module is as follows Figure 2a to Figure 2b As shown in the figure, the essential feature maps are first obtained through a small number of convolution operations, and then linear transformation is applied to these essential feature maps to obtain Ghost feature maps (redundant feature maps). Finally, the essential feature maps and redundant feature maps are cascaded to obtain the same number of feature maps as the output of the ordinary convolution layer.

[0074] Step 2.2, add the CBAM attention mechanism to GhostBottleneck to form an improved C2Ghost module;

[0075] like Figure 3 As shown in the figure, the structure of the CBAM attention mechanism includes two parts: channel attention and spatial attention. By using maximum pooling and average pooling on the feature map F of size H×W×C, two 1×1×C feature vectors are obtained. The feature vectors obtained by pooling are transmitted to the shared network, and the dependency between channels is obtained by the ReLU activation function, and the sum operation is performed. Then, the channel attention features are generated through the Sigmoid activation function. Finally, the generated channel attention features are multiplied to obtain F′ and transmitted to the spatial attention module.

[0076] First, F′ is subjected to global maximum pooling and average pooling to obtain two H×W×1 feature vectors, which are then channel-connected. Then, it is processed by 7×7 convolution to reduce the dimension into a spatial vector H×W×1. The spatial features are then obtained through the Sigmoid activation function, so as to better learn important features, suppress unimportant feature information, obtain more and more complete target feature information, and better grasp the global information of the image receptive field.

[0077] like Figure 4c As shown in Figure 1, the implementation process of the C2Ghost module. In the Ghost module, the redundant feature map obtained after linear transformation retains the redundant feature information in a more cost-effective way. However, compared with the feature map obtained after normal convolution, the similar feature map obtained after linear transformation still has information missing to a certain extent, thus affecting the accuracy of the final detection result.

[0078] Since the output of the Ghost module after normal convolution and linear transformation operations is a symmetrical and non-sparse distribution, processing them through the BN layer will make them closer to the origin distribution, and the data will also become more compact from a relatively sparse distribution. The CBAM attention mechanism is added after the BN layer, so that the network can better automatically learn the spatial position of the feature map of the data and the importance of the channel, enhance the useful features in the redundant feature map generated by the Ghost module according to the importance, and suppress the useless features, so as to achieve adaptive calibration of space and channels, and finally improve the detection accuracy of the model. Secondly, a Ghost module is added to the shortcut path first, on the one hand, to reduce the number of parameters and floating-point operations of the module, and on the other hand, to match the input channel of the CBAM attention mechanism added later, so that the number of output channels on the shortcut path matches the number of output channels on the main path. A CBAM module is added after the Ghost module on the shortcut path, in order to further ensure the detection accuracy of the model while reducing the amount of operations.

[0079] Step 2.3, in the Neck network part of YOLOv5, use BiFPN to replace PANet in the original network;

[0080] Aiming at the problems of diverse shooting angles in the images taken by aircraft engine borescope cameras, crack deformation and crack scale changes caused by borescope camera parameter settings, the bidirectional pyramid network multi-scale fusion method is used to replace the feature pyramid network stacking fusion method in the original YOLOv5 network to improve the detection accuracy of multi-scale cracks.

[0081] like Figure 5a to Figure 5cAs shown in the figure, BiPFN optimizes the feature network from three aspects based on PANet.

[0082] First, some nodes in PANet have only one input, which means that these nodes do not perform feature fusion. The purpose of the feature network is to fuse different features, so these nodes contribute little to feature fusion. BiFPN removes these nodes to simplify the network structure and reduce computational overhead.

[0083] Second, adding a connection between input and output of the same level incorporates more features without increasing much computational overhead.

[0084] Finally, BiFPN is used as a basic feature network layer and repeated multiple times. Each layer will further extract higher-level and more abstract fusion features based on the original fusion features. At the same time, the weighted fusion method is used to fuse feature layers of different resolutions, and the contribution of features in feature fusion is adjusted through trainable weights.

[0085] When fusing features of different resolutions, all input features are considered equally important. Simply inputting the same feature weight is too simple and may lose some useful information during the fusion process. Weighted feature fusion assigns different weights to each input feature and allows the network to automatically learn the weight of each input feature. The function expression of weighted feature fusion is:

[0086]

[0087] Among them, ω i is a learnable weight, ε is a number much smaller than 1, I i It is a feature. From the functional expression of weighted feature fusion, it can be seen that the weight of feature fusion is limited between 0 and 1, and the use of softmax function which will significantly increase the calculation time is successfully avoided. The following two formulas describe the cross-scale connection and weighted feature fusion process of BiFPN network at level 6:

[0088]

[0089]

[0090] Among them, P in is the input feature, P out is the output feature, P td It is the intermediate layer in the top-down feature fusion process.

[0091] like Figure 6The figure shows the network architecture of the improved version of YOLOv5s. The improved YOLOv5 network model takes the output layer of the 4th network module as the P3 layer (low feature layer), the output layer of the 6th network module as the P4 layer (middle feature layer), and the output layer of the 9th network module as the P5 layer (high feature layer). At the same time, a connection is added between the output layers of the 6th and 13th network modules. Each fusion further extracts higher-level and more abstract features based on the original fusion features, which improves the robustness of the detection.

[0092] Step 3, train the improved YOLOv5 convolutional neural network;

[0093] The improved network model training parameter settings are as follows: the initial learning rate is 0.01, the impulse size is 0.937, the input image size is 1024×1024, the batch size is 32, and the number of iterations is 200.

[0094] In order to further improve the accuracy of borehole crack detection, the dark light enhancement algorithm is first used to enhance the dark light of the data set when loading the image data.

[0095] The dark light enhancement algorithm uses the Retinex theory to separate the brightness component and the reflection component of the image, performs adaptive gamma correction on the brightness component, and finally uses the reflection component to restore the details and color of the image. The Retinex theory believes that the image is composed of brightness component and reflection component. Assume that each channel of the RGB image I (x, y) has the same brightness, that is:

[0096] I c (x, y) = R c (x, y)·L(x, y), c∈{r, g, b}

[0097] Where R c (x, y) represents the reflection component of each channel, and L(x, y) represents the brightness component of the image. Usually, the maximum value of the RGB channels is used as the illumination V(x, y) observed by the human eye.

[0098]

[0099] According to Retinex theory, the luminance component L(x, y) of the image can be obtained by convolving the luminance image V(x, y) with a 2D Gaussian filter G(x, y).

[0100] L(x, y) = G(x, y) * V(x, y)

[0101] Therefore, the reflection component R c (x, y) can be separated out.

[0102] R c(x, y) = I c (x, y) / L(x, y), c∈{r, g, b}

[0103] The reflection component mainly contains the high-frequency components of the image, including edges and details.

[0104] After obtaining the image brightness component, an adaptive Gamma correction is performed on it.

[0105] L en (x, y) = L(x, y)γ (x,y)

[0106]

[0107]

[0108] L en (x, y) represents the enhanced brightness component, γ(x, y) represents the Gamma correction coefficient matrix, P ω (l) is the weight distribution function corresponding to each brightness value:

[0109]

[0110] Where P(l) is the probability density function of the brightness component, p max is the maximum value of P(l), p min is the minimum value of P(l), P(l) can be obtained by the following formula:

[0111] P(l)=n l / n p

[0112] Where n l is the number of pixels corresponding to the brightness, n p The total number of pixels contained in the brightness component.

[0113] Fusion L en (x, y) and R c (x, y) to get the final enhanced image I en (x, y).

[0114]

[0115] After dark-light enhancement of the dataset using the dark-light enhancement algorithm, the improved YOLOv5 network is trained.

[0116] Step 4: Use the trained weight file and the test data set to test, conduct an ablation experiment on the test results of the improved model, compare the performance of each model, and thus obtain the final YOLOv5 improved network;

[0117] The best.pt weight file is obtained through training and loaded during testing. The performance of the improved model is evaluated by testing the harmonic mean of F1 precision and recall, P precision, R recall, mAP (average AP of different categories) and other indicators in the result file. Ablation experiments are performed on each improved model, and the best performing model and its corresponding weight file are selected and deployed to the embedded platform.

[0118] Step 5: Deploy the improved model to the Jetson Nano embedded platform and load the trained weight file during detection. During deployment, convert the Pytorch pt format weight file to a wts format file, and finally to an engine format file, accelerate it through TensorRT, and perform the final inference.

[0119] The inference time required for each frame in the detection process is obtained, and the lightweight deployment of the improved model is finally evaluated by comparing the detection accuracy of the model and the inference time of a single frame image. Specific embodiment:

[0121] The NVIDIA Jetson Nano 2GB development board is used as the embedded platform. The device on the development board contains a quad-core A57@1.43HZ CPU and 128-core Maxwell GPU, which are characterized by low power consumption and low energy consumption. Therefore, the improved algorithm for lightweight deployment of aircraft engine borehole crack detection proposed in the present invention is tested based on the NVIDIA Jetson Nano 2GB development board. The experimental method is as follows:

[0122] 1) The improved YOLOv5s algorithm model was trained using the borehole crack dataset on the GPU platform, and the various improvements were compared through ablation experiments. As can be seen from Table 1, by adding the dark light enhancement algorithm and replacing PANet in the Neck structure of the original model with BiFPN, the mAP value of the model detection has been significantly improved. After using the GhosBottleneck module to replace the Bottleneck module in the original network C3 structure, it is not difficult to find that the number of parameters and GFLOPS of the improved model have been significantly reduced, making the model easier to deploy on embedded platforms. But at the same time, with the lightweight model, the detection accuracy value is lower than the original YOLOv5s network, and the mAP improvement rate is also lower than other improvements.

[0123] Table 1 Ablation experiments based on the first three improvements

[0124]

[0125] 2) In order to further improve the detection accuracy while significantly reducing the parameter scale and computational complexity of the network, the next step is to improve the GhostBottleneck module to the C2Ghost module.

[0126] 3) Deploy the improved final model to the Jetson Nano embedded platform. Optimize and compile the corresponding model files on the Jetson Nano development board, use TensorRT for acceleration, convert the model format to TensorRT format (TRT), and use C++ program interface to implement API calls.

[0127] 4) Compare the improved final network model with the original network model and the GhostBottleneck module model before improvement. As shown in Table 2, by comparing evaluation indicators such as single-frame image inference time (ms), the lightweight deployment of the improved model is finally evaluated.

[0128] Table 2 Ablation experiment on embedded platform based on the final improved model

[0129]

Claims

1. A lightweight deployment method based on aircraft engine borehole detection crack detection, characterized in that: The method comprises the following steps: Step 1) obtaining an image or video of aeroengine borescope cracks and forming a borescope crack image dataset; the video is a plurality of continuous frame images; Step 2) Establish a YOLOv5 convolutional neural network model based on GhostNet; Step 2.1) In the Backbone network part of YOLOv5, use the GhostBottleneck module with stride=1 to replace the Bottleneck module in the original YOLOv5 network C3 structure; Step 2.2) Add the CBAM attention mechanism to GhostBottleneck to form an improved C2Ghost module; Step 2.3) In the neck network part of YOLOv5, use BiFPN to replace PANet in the original YOLOv5 network; Step 3) training the improved YOLOv5 convolutional neural network model; Using the borehole crack dataset, when loading image or video data, the dataset is first enhanced by dark light using the dark light enhancement algorithm, and then the improved YOLOv5 convolutional neural network is trained; Step 4) Use the trained weight file and test data set to test, conduct ablation experiments on the test results of the improved model, compare the performance of each model, and select the YOLOv5 improved network model with the best performance; Step 5) deploying the improved network model to the Jetson Nano embedded platform, and using the trained weight file to detect borehole crack images and videos; The lightweight deployment of the improved model is finally evaluated by comparing the crack detection accuracy and single-frame image inference time. In the step 1), firstly, the image or video data of the boresight crack of the aircraft engine is collected; then, the collected data image is annotated, the position of the crack is annotated with a rectangular frame, and the image category is annotated as crack; finally, the collected image is data enhanced to expand the data set; and the boresight crack image data set is obtained; The data enhancement method of step 1) is: using Mosaci data enhancement, scaling and randomly distributing images; preprocessing the data set by adding noise; using the CutMix method to randomly combine two borehole crack images; dividing the borehole crack image data set and the category labels generated by labeling into a training set and a test set; The GhostBottleneck module in step 2.1) is composed of two stacked Ghost modules; The first Ghost module is used as an expansion layer to increase the number of channels; the second Ghost module reduces the number of channels to match the number of shortcut channels; The stride=1 structure uses shortcut to connect the input and output of these two Ghost modules; Among them, the second Ghost module does not use the ReLU activation function, while other layers apply batch normalization BN and ReLU activation functions after each layer; The stride=2 structure is that two Ghost modules are connected through a deep convolution DWConv, and then a shortcut is used to connect the input and output of the two Ghost modules; The steps of the C2Ghost module based on the CBAM improvement in step 2.2) include: Step 2.2.1) Add the CBAM attention mechanism after the two Ghost modules in GhostBottleneck; Step 2.2.2) Add a Ghost module and a CBAM attention mechanism to the shortcut path; The implementation steps of BiFPN in step 2.3) include: Step 2.3.1) Remove nodes with only one input in PANet to reduce computational overhead; Step 2.3.2) Take the top-down and bottom-up bidirectional paths of PANet as the base layer and repeat it multiple times; Step 2.3.3) Add an additional connection edge between input and output nodes of the same level in a manner similar to residual connection; Step 2.3.4) Use weighted fusion to fuse feature layers of different resolutions, and adjust the contribution of features in feature fusion through trainable weights; In step 2.3.4), the function expression of weighted feature fusion is: Among them, ω i is a learnable weight, ε is a number much smaller than 1, I i is a feature; the weight of feature fusion is limited to between 0 and 1; The following two formulas describe the process of cross-scale connection and weighted feature fusion of the BiFPN network at level 6: Among them, P in is the input feature, P out is the output feature, P td It is the middle layer in the top-down feature fusion process; in the improved YOLOv5 network model, the output layer of the 4th network module is taken as the low feature layer P3 layer, the output layer of the 6th network module is taken as the middle feature layer P4 layer, and the output layer of the 9th network module is taken as the high feature layer P5 layer; At the same time, add a connection between the output layers of the 6th and 13th network modules; In step 3), set the training parameters: initial learning rate, impulse size, input image size, batch size, and number of iterations; The dark light enhancement algorithm in step 3) includes: 3.1) using Retinex theory to separate the brightness component and the reflection component of the image; 3.2) performing adaptive gamma correction on the brightness component; 3.3) using the reflection component to restore the details and color of the image; 3.1) Based on Retinex theory, an image is composed of brightness component and reflection component; assuming that each channel of the RGB image I(x, y) has the same brightness, that is: I c (x,y)=R c (x,y)·L(x,y),c∈{r,g,b}, Where R c (x, y) represents the reflection component of each channel, L(x, y) represents the brightness component of the image; the maximum value of the RGB channel is taken as the illumination V(x, y) observed by the human eye; According to the Retinex theory, the luminance image V(x, y) is convolved with a 2D Gaussian filter G(x, y) to obtain the luminance component L(x, y) of the image: L(x,y)=G(x,y)*V(x,y); Therefore, the reflection component R c (x, y) is separated; The reflection component includes high-frequency components such as edges and details of the image; 3.2) After obtaining the brightness component of the image, perform adaptive Gamma correction on the brightness component: L en (x,y)=L(x,y) γ(x,y) L en (x, y) represents the enhanced brightness component, γ(x, y) represents the Gamma correction coefficient matrix, P ω (l) is the weight distribution function corresponding to each brightness value: Where P(l) is the probability density function of the brightness component, p max is the maximum value of P(l), p min is the minimum value of P(l), and P(l) is obtained by the following formula: Where n l is the number of pixels corresponding to the brightness, n p The total number of pixels contained in the brightness component; 3.3) Fusion L en (x, y) and R c (x, y) is the final enhanced image I en (x, y); I en (x,y)=R c (x,y)·L en (x,y),c∈{r,g,b} After dark-light enhancement of the dataset using the dark-light enhancement algorithm, the improved YOLOv5 network is trained.

2. The lightweight deployment method based on aircraft engine borehole detection crack detection according to claim 1 is characterized in that: The Ghost module algorithm is as follows: first, an essential feature map is obtained through a small amount of convolution operation; then, a linear transformation is applied to these essential feature maps to obtain a Ghost feature map, i.e., a redundant feature map; finally, the essential feature map and the redundant feature map are cascaded to obtain the same number of feature maps as the output of the ordinary convolution layer.

3. The lightweight deployment method based on aircraft engine borehole detection crack detection according to claim 1 is characterized in that: The implementation steps of the CBAM attention mechanism in step 2.2) are: In the first step, the input data is subjected to global maximum pooling and global average pooling to obtain two feature maps, which are based on the channel attention mechanism. In the second step, the two feature maps are concatenated and reduced to one channel. The third step is to generate a spatial attention feature map through the Sigmoid activation function; The fourth step is to multiply the feature map of the third step by the input feature map of the module to generate the final feature map.

4. The lightweight deployment method based on aircraft engine borehole detection crack detection according to claim 1 is characterized in that: In step 5), TensorRT is also used for acceleration. The process of using TensorRT for reasoning acceleration is as follows: the improved network model file is converted into a model in the order of pt format file, wts format file, and engine format file, and finally the engine format file is subjected to final reasoning.