Multi-modal information fusion aeronautical hole exploration blade defect detection method and multi-modal information fusion aeronautical hole exploration blade defect detection system

Abstract

The invention belongs to the technical field of aero-engine operation reliability and intelligent maintenance, and discloses a multi-modal information fusion aero-borescope blade defect detection method and a multi-modal information fusion aero-borescope blade defect detection system. According to the method, data are collected and preprocessed, a YOLOv11 target detection model is used for target detection of a two-dimensional image, a PointNet + + deep learning network is used for feature extraction of three-dimensional point cloud data, and comprehensive defect detection of the blade surface is completed. The fusion module is used for comprehensively processing the detection results from the YOLOv11 and the PointNet + + to obtain comprehensive defect evaluation; the detected defects are dynamically tracked, and a user interacts through a control interface. The method is suitable for regular inspection, on-line monitoring and dynamic tracking of the aircraft, and accurate identification, positioning and quantitative evaluation of surface damage of the aircraft can be realized under the condition that engine blades are not disassembled.

Description

Technical Field

[0001] The present invention belongs to the technical field of the operation reliability and intelligent maintenance of aero-engines, and particularly relates to a method and system for detecting aeroengine blade defects by multi-modal information fusion. Background Art

[0002] Aeroengine blades are key components in the power system of aircraft. They are long-term subjected to high temperature, high pressure and severe airflow impact, and are vulnerable to damage by various factors. As the operation time of the aircraft increases, defects such as cracks, corrosion and wear may occur on the blade surface. These defects not only affect the performance of the engine, but also may endanger flight safety. Therefore, how to efficiently and accurately detect defects and evaluate damage of aeroengine blades has become an important issue in the field of aircraft maintenance.

[0003] Traditional blade defect detection methods mainly include manual inspection, ultrasonic inspection, X-ray inspection and eddy current inspection, etc. Although these technologies have achieved certain results in some applications, they still have many limitations. For example, manual inspection is easily affected by human factors, with low efficiency and difficult to ensure consistency; although ultrasonic inspection can effectively detect internal defects, its detection accuracy for surface defects is limited, and contact detection is required, and it cannot monitor in real time; X-ray inspection is limited by equipment cost, operation complexity and safety issues, and it is difficult to meet the requirements of complex structures and large-scale inspections.

[0004] In order to overcome the limitations of traditional detection methods, in recent years, automated defect detection technologies based on computer vision and deep learning have received extensive attention. Using two-dimensional images and three-dimensional point cloud data for defect detection can not only improve the detection accuracy, but also reduce human operation errors and improve the detection efficiency. For example, in recent years, object detection methods based on convolutional neural networks (CNNs) have been widely applied in the field of image processing, and have achieved remarkable results especially in industrial surface defect detection, medical image analysis and other fields. At the same time, with the development of laser scanning technology and three-dimensional point cloud data processing technology, point cloud data has gradually become an important means for detecting and evaluating surface defects of complex three-dimensional objects.

[0005] However, most of the existing defect detection methods based on images or point cloud data focus on the application of a single data source and lack effective fusion of multi-modal information. Two-dimensional images can provide rich surface texture and color information, but when dealing with complex surface defects or micro-cracks, there are often problems such as detail loss or perspective dependence. Three-dimensional point cloud data can provide accurate geometric information of the object surface and can effectively identify the morphology of surface defects, but its ability to extract texture features is weak, and it is difficult to handle some defects with rich textures. A single data mode cannot comprehensively reflect the defect characteristics of the blade surface, thereby affecting the detection accuracy.

[0006] Therefore, how to effectively fuse multi-modal data (such as 2D images and 3D point cloud data) to make up for their respective deficiencies and improve the accuracy and robustness of aero-engine blade defect detection has become a hot issue in current research. In recent years, the application of deep learning in multi-modal information fusion has gradually received attention. Some studies have begun to attempt to combine image and point cloud data, use convolutional neural networks (CNNs) to process image data, combine point cloud networks (such as PointNet, PointNet++) to process 3D data, and achieve joint learning and feature fusion through multi-modal deep learning models to improve detection performance. However, the current multi-modal fusion methods still face some challenges, especially in the design of data fusion, feature matching, and multi-task learning, and further research and optimization are still needed.

[0007] In addition, although the existing technologies can better achieve the defect detection of static images, in practical applications, the detection of aero-engine blades usually needs to be carried out in a dynamic environment. Traditional methods have great difficulties in real-time identifying and tracking targets in dynamic video streams, especially when facing complex and rapidly changing defects, the real-time performance and accuracy are insufficient. Therefore, how to combine multi-modal data for target detection and tracking based on real-time monitoring has become the key to improving detection efficiency and accuracy.

[0008] In summary, the defects and problems existing in the existing technologies are as follows: With the increasing complexity of the operating environment of aircraft and the continuous improvement of safety requirements, higher requirements are put forward for the defect detection of aero-engine blades and other key components. Traditional manual detection methods, ultrasonic, X-ray detection, etc. have obvious limitations under the requirements of high frequency and high precision: Manual inspection is not only inefficient but also easily affected by human factors. Traditional non-destructive testing methods (such as ultrasonic, X-ray imaging, etc.) often require long-term operations, and there are certain blind spots in the detection of complex surfaces or micro-cracks.

[0009] Therefore, the defect detection methods based on computer vision and deep learning have become a research hotspot. In particular, with the rapid development of deep learning technologies, especially the emergence of convolutional neural networks (CNNs), object detection algorithms (such as the YOLO series), and point cloud processing networks (such as PointNet++), new solutions have been provided for the defect detection on complex surfaces such as blades. However, most of the existing technologies only rely on single 2D image data or 3D point cloud data, ignoring the complementarity of image and point cloud data, resulting in unsatisfactory performance in some complex defect scenarios. In particular, the surface damage of blades usually has complex geometric shapes and subtle changes, and a single data mode cannot fully capture these changes, affecting the accuracy and robustness of detection. Summary of the Invention

[0010] To overcome the problems existing in the related technologies, the disclosed embodiments of the present invention provide a method and system for detecting blade defects by multi-modal information fusion in aviation borescope inspection.

[0011] The technical solution is as follows: A method for detecting blade defects by multi-modal information fusion, the method includes:

[0012] S1, synchronously collect the two-dimensional image and three-dimensional point cloud data of the blade surface through a high-definition image sensor and a three-dimensional laser scanner; perform denoising, image enhancement and smoothing processing on the collected data;

[0013] S2, use the YOLOv11 object detection model to perform object detection on the two-dimensional image, and mark various types of defects on the blade surface. Use the PointNet++ deep learning network to extract the features of the three-dimensional point cloud data, map the features through a fully connected layer, and use the softmax function to output the classification result; classify the point cloud data by gradually extracting local features, aggregating global information and final classification processing to complete the comprehensive defect detection of the blade surface;

[0014] S3, the fusion module comprehensively processes the detection results from the YOLOv11 object detection model and the PointNet++ deep learning network, uses multi-modal learning technology, combines the two-dimensional image and the three-dimensional point cloud to obtain a comprehensive defect assessment; the decision support module based on deep learning performs regression analysis according to the comprehensive threshold of the position, size and depth information of the defect, predicts the impact degree of the defect on the overall structure, automatically judges the severity of the defect, and provides intelligent maintenance suggestions;

[0015] S4, use the DeepSORT object tracking algorithm to dynamically track the detected defects, combine the Kalman filter and the Hungarian algorithm to complete the real-time tracking and status update of the defects; through dynamic monitoring, analyze the evolution process of the blade surface defects in real time and predict potential risks;

[0016] S5, perform interaction through the control interface, use a linear rocker to adjust the probe angle, adjust the detection area, view the image in real time, perform data zooming in or out, view the details of the blade surface, and locate the defect position.

[0017] In step S2, using the YOLOv11 object detection model to perform object detection on the two-dimensional image includes:

[0018] (1) The Backbone part extracts multi-scale features from the input image;

[0019] (2) The Head part performs object detection according to the feature map extracted by the Backbone.

[0020] In step (1), the Backbone part extracts multi-scale features from the input image, including:

[0021] (a) After the image passes through the first layer of convolution, downsampling is performed, and the size of the feature map becomes half of the original image; the second layer of convolution extracts features and downsamples again, and the size of the output feature map becomes P2 / 4; the third layer uses C3k2_OREPA_neck, and the size of the output feature map is P3 / 8;

[0022] (b) Through another layer of convolution downsampling, a feature map of P4 / 16 is output; the C3k2_SAConv module captures multi-scale features. After convolution, the size of the feature map becomes P5 / 32; the C3k2_OREPA_neck module enhances the global information of the feature map;

[0023] (c) Use convolution to perform the last downsampling on the image, and the size of the feature map becomes P5 / 32; the C3k2_SAConv module extracts high-level features; the SPPF module performs spatial pyramid pooling to increase multi-scale information; the representation ability of high-level features is enhanced through the C2PSA module.

[0024] In step (2), the Head part performs object detection based on the feature map extracted by the Backbone, including:

[0025] First, the P5 feature map is upsampled and concatenated with the P4 feature map to generate a multi-scale feature map;

[0026] Second, use the C3k2_OREPA_neck module to extract the concatenated features; the P4 feature map is upsampled and enlarged and concatenated with the P3 feature map, and the C3k2_SAConv module is used to extract the features of small-scale objects;

[0027] Then, the P3 feature map is convolved and downsampled again, from P3 / 8 to P4 / 16, and concatenated with the processed P4 feature map to enhance the mid-scale features; processed by the C3k2_OREPA_neck module; the P4 feature map is convolved and downsampled again from P4 / 16 to P5 / 32 and concatenated with the P5 feature map;

[0028] Subsequently, the C3k2_SAConv module extracts large-scale features;

[0029] Finally, object detection is performed through the ADDWConvHead module, and the category and bounding box of the object are output.

[0030] In the ADDWConvHead module, the FADC module is used to dynamically adjust the dilation rate according to the local frequency components of the image, enabling the network to adjust the receptive field according to the local changes in the image content and complete the processing of multi-scale and multi-directional targets.

[0031] In step S2, the PointNet++ deep learning network is used to extract the features of the 3D point cloud data, including: defining the structure through the get_model class, where the input of the model is a point cloud data xyz with a shape of (B, 3, N) or (B, 6, N), where B is the batch size, 3 or 6 represents the point coordinates and normal information, and N is the number of points in each point cloud.

[0032] First, in the sa1 layer, the PointNet++ model processes the input point cloud using the PointNetSetAbstraction module, extracts local features by specifying the sampling number, radius, and number of neighborhood points; sa1 selects 512 points from the original point cloud data and selects 32 points from the neighborhood of each point according to a radius of 0.2, and maps the neighborhood features to a 128-dimensional feature space through a multi-layer perceptron.

[0033] Second, sa2 processes the output of sa1, selects 128 points from it for processing through a radius of 0.4 and neighborhood points, and maps them to a 256-dimensional space.

[0034] Third, in the sa3 layer, the PointNet++ model aggregates all points to extract global features without point sampling, expands the 256-dimensional features to 1024 dimensions, and captures global information through the MLP.

[0035] Then, the network enters the fully connected layer. In fc1 and fc2, the 1024-dimensional features are transformed through 512 dimensions and 256 dimensions respectively, and then processed through the batch normalization and Dropout layers.

[0036] Finally, fc3 maps the features to the number of target categories and uses log_softmax for classification output.

[0037] The loss calculation of the PointNet++ model is carried out through the get_loss class, and the negative log-likelihood loss is used to calculate the error between the prediction result of the PointNet++ model and the true label, and the final loss value is returned.

[0038] After step S5, data storage and report generation are also performed, automatically generating a defect detection report containing complete detection data, defect locations, severity assessments, and repair advice information; all detection data and reports are stored in the system database.

[0039] Another object of the present invention is to provide an aviation borescope blade defect detection system for multi-modal information fusion, which implements the above-mentioned multi-modal information fusion aviation borescope blade defect detection method. The system includes:

[0040] A data acquisition and preprocessing module, which is used to synchronously acquire two-dimensional images and three-dimensional point cloud data of the blade surface through a high-definition image sensor and a three-dimensional laser scanner; denoise, enhance the image, and smooth the three-dimensional point cloud data after acquisition.

[0041] A defect detection and classification module, which is used to perform object detection on two-dimensional images using the YOLOv11 object detection model, mark various types of defects on the blade surface, use the PointNet++ deep learning network to extract features of three-dimensional point cloud data, map the features through a fully connected layer, and use the softmax function to output classification results; classify the point cloud data by gradually extracting local features, aggregating global information, and final classification processing to complete comprehensive defect detection on the blade surface.

[0042] A data fusion and decision support module. The fusion module comprehensively processes the detection results from the YOLOv11 object detection model and the PointNet++ deep learning network, uses multi-modal learning technology, combines two-dimensional images and three-dimensional point clouds to obtain a comprehensive defect assessment; based on the comprehensive threshold of the position, size, and depth information of the defect, the decision support module based on deep learning performs regression analysis, predicts the impact of the defect on the overall structure, automatically judges the severity of the defect, and provides intelligent maintenance suggestions.

[0043] An object tracking and dynamic monitoring module, which uses the DeepSORT object tracking algorithm to dynamically track the detected defects, combines the Kalman filter and the Hungarian algorithm to complete real-time tracking and status update of the defects; through dynamic monitoring, it analyzes the evolution process of the blade surface defects in real time and predicts potential risks.

[0044] A user interaction and operation interface module, which is used to interact through a control interface, adjust the probe angle and detection area using a linear rocker, view images in real time, zoom in or out on data, view the details of the blade surface, and locate the defect position.

[0045] Furthermore, the aviation borescope blade defect detection system for multi-modal information fusion is carried on a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the functions in the above-mentioned aviation borescope blade defect detection system for multi-modal information fusion can be realized.

[0046] Combining all the above technical solutions, the beneficial effects of the present invention are as follows: The present invention applies multi-modal deep learning technology. By fusing two-dimensional image data and three-dimensional point cloud data, it realizes efficient and automated detection of surface defects of aero-engine blades and damage assessment. This technology is especially suitable for regular inspections, online monitoring, and dynamic tracking of aircraft, and can accurately identify, locate, and quantitatively evaluate surface damage of engine blades without disassembling them.

[0047] Based on the principle of multi-modal data fusion, the present invention combines two-dimensional images and three-dimensional point cloud data, and designs a deep learning framework that integrates object detection, three-dimensional data processing, and dynamic tracking. Through an efficient object detection algorithm (such as YOLOv11), defect location and classification are carried out. At the same time, the PointNet++ model is used to extract deep features from three-dimensional point cloud data, and further combined with the DeepSORT object tracking algorithm to achieve dynamic tracking, comprehensively improving the detection accuracy and real-time performance of blade defects. The technical solution of the present invention can not only automatically detect and track various defects on engine blades, but also perform three-dimensional damage assessment based on the detection results, providing a scientific basis for subsequent maintenance decisions and meeting the high-efficiency and safe operation requirements of aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure;

[0049] Figure 1 is a flowchart of the method for detecting aeroengine blade defects by multi-modal information fusion provided by an embodiment of the present invention;

[0050] Figure 2 is a flowchart of multi-source data acquisition and processing provided by an embodiment of the present invention;

[0051] Figure 3 is a flowchart of the network structure operation of the detection module provided by an embodiment of the present invention;

[0052] Figure 4 is a network structure diagram of C3k2—OREPA provided by an embodiment of the present invention;

[0053] Figure 5 is a flowchart of the operation of the recognition system provided by an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of the system for detecting aeroengine blade defects by multi-modal information fusion provided by an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of the principle of the system for detecting aeroengine blade defects by multi-modal information fusion provided by an embodiment of the present invention;

[0056] In the figure: 1. Data acquisition and preprocessing module; 2. Defect detection and classification module; 3. Data fusion and decision support module; 4. Target tracking and dynamic monitoring module; 5. User interaction and operation interface module. Specific implementation manner

[0057] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific implementation manner of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific implementations disclosed below.

[0058] The innovation of the present invention lies in: by extracting aircraft defect features in different dimensions and fusing them, a neural network model suitable for detecting defects and damages of engine blades is established to achieve precise identification of surface defects and damages of engine blades.

[0059] Example 1, as Figure 1 shown, the multi-modal information fusion method for detecting aviation borescope blade defects provided by the embodiment of the present invention includes the following steps:

[0060] S1. Synchronously collect two-dimensional images and three-dimensional point cloud data on the blade surface through a high-definition image sensor and a three-dimensional laser scanner; perform denoising, image enhancement on the collected data, and smoothing processing on the three-dimensional point cloud data;

[0061] S2. Use the YOLOv11 object detection model to perform object detection on the two-dimensional image and mark various types of defects on the blade surface. Use the PointNet++ deep learning network to extract features of the three-dimensional point cloud data, map the features through a fully connected layer, and use the softmax function to output the classification result; classify the point cloud data by gradually extracting local features, aggregating global information, and final classification processing to complete a comprehensive defect detection on the blade surface;

[0062] S3. The fusion module comprehensively processes the detection results from the YOLOv11 object detection model and the PointNet++ deep learning network, uses multi-modal learning technology, combines the two-dimensional image and the three-dimensional point cloud to obtain a comprehensive defect assessment; based on the comprehensive threshold size of the position, size, and depth information of the defect, the decision support module based on deep learning performs regression analysis, predicts the impact degree of the defect on the overall structure, automatically judges the severity of the defect, and provides intelligent maintenance suggestions;

[0063] S4. Use the DeepSORT object tracking algorithm to dynamically track the detected defects. Combine the Kalman filter and the Hungarian algorithm to complete the real-time tracking and status update of the defects. Through dynamic monitoring, analyze the evolution process of the defects on the blade surface in real time and predict potential risks.

[0064] S5. Interact through the control interface. Use the linear rocker to adjust the probe angle, adjust the detection area, view the image in real time, zoom in or out the data, view the details of the blade surface, and locate the defect position.

[0065] Embodiment 2. Exemplarily, as another embodiment of the present invention, the provided multi-modal information fusion method for detecting blade defects by borescope includes:

[0066] Step 1. Data acquisition and transmission; as Figure 2 shown, it shows the entire process of data from acquisition, processing to training: The system starts to execute the task. At the beginning of the detection, the system simultaneously acquires the image and three-dimensional point cloud data of the blade surface through the high-definition image sensor and the three-dimensional laser scanner. The image data captured by the high-definition image sensor can provide high-resolution surface texture information, while the point cloud data obtained by the three-dimensional laser scanner provides the precise geometric shape of the blade surface. These two data sources provide strong support for subsequent defect detection and analysis. After data acquisition, all image and point cloud data will be transmitted to the data processing module through the electrical control cabinet. In the data processing module, the data will undergo denoising, enhancement, and normalization processing to ensure data quality. In particular, problems such as noise, reflection, or uneven illumination in the image will be eliminated through image enhancement techniques, and outliers and errors in the point cloud data will also be removed to ensure the accuracy of subsequent detection.

[0067] Step 2. Defect detection and classification: The preprocessed data will be fed into the YOLOv11 object detection model for defect detection of two-dimensional images. At the same time, use the PointNet++ model to process the three-dimensional point cloud data and extract the geometric features of the blade surface; as Figure 3 the network structure working flowchart of the detection module; the network structure diagram of C3k2—OREPA is as Figure 4 shown.

[0068] First, the Backbone part is responsible for extracting multi-scale features from the input image. After the input image passes through the first convolutional layer (Conv(64,3,2)), downsampling is performed, and the size of the feature map becomes half of the original image (P1 / 2). Then, the second convolutional layer (Conv(128,3,2)) further extracts features and downsamples again, and the size of the output feature map becomes P2 / 4. The next third layer uses C3k2_OREPA_neck(256,False,0.25), which is a multi-scale feature extraction module that enhances the receptive field ability of the network. The size of the output feature map of this layer is P3 / 8.

[0069] C3k2_OREPA reduces the computational and memory costs of model training through Online Convolutional Re-parameterization while improving the efficiency in the inference stage. The whole method consists of two-stage processes, namely Block Linearization and Block Squeezing.

[0070] Block Linearization: In the training stage, OREPA first performs a linearization operation on the standard convolutional block. The main steps are as follows:

[0071] 1. Remove non-linear components (such as ReLU, BatchNorm): This enables the convolutional operation to be equivalently transformed mathematically.

[0072] 2. Merge the Batch Normalization (BN) layer: Directly merge the BN layer into the convolutional weights to avoid independent calculation of BN and improve computational efficiency.

[0073] 3. Introduce a Scaling Layer: The role of this layer is to adjust the weight scale of the convolutional layer to make it more stable during training and retain the gradient flow.

[0074] In the standard CNN training structure, a convolutional layer + BN operation can be expressed as:

[0075]

[0076] In the formula, γ and β are both trainable parameters of BN, x is the input feature map, μ and σ are the mean and standard deviation of BN respectively;

[0077] BN can be merged into the convolutional kernel W and bias b, and the expression is:

[0078]

[0079] In this way, the original "convolution + BN" structure can be transformed into a single convolutional layer, avoiding the additional computational overhead during training.

[0080] Block Squeezing: After linearization, OREPA further reduces computations through structure merging:

[0081] 1. Merging convolutional layers of different scales: For example, convolutions of various sizes such as 1×1, 3×3, 5×5, etc. can be replaced with an equivalent 3×3 convolution.

[0082] 2. Reducing redundant parameters: Through adjustments during training, the parameters of multiple convolutional layers can be merged into a single equivalent convolutional layer during the inference stage.

[0083] During training, multiple convolutional kernels (e.g., W 1 , W 2 , W 3 ) can be used to enhance the expressive power of the model:

[0084] y = (W 1 + W 2 + W 3 ) × x + (b 1 + b 2 + b 3 )

[0085] In the formula, W 1 , W 2 , W 3 are different convolutional weights, and b 1 , b 2 , b 3 are the corresponding bias terms;

[0086] During the inference stage, all convolutional kernels are merged:

[0087] W′ = W 1 + W 2 + W 3 , b′ = b 1 + b 2 + b 3

[0088] Finally, the entire training structure is converted into a single convolutional layer, improving the inference efficiency.

[0089] Then, further downsampling is performed through another layer of convolution (Conv(256, 3, 2)), and a feature map of P4 / 16 is output. The subsequent C3k2_SAConv(512, False, 0.25) module is specifically used to capture multi-scale features, helping the YOLOv11 object detection model extract information better at different scales.

[0090] Switchable Atrous Convolution (SAC) is an advanced convolution mechanism used to enhance feature extraction in object detection and segmentation tasks. The following are the main principles and mechanisms of SAC:

[0091] 1. Application of different dilation rates: The core idea of SAC is to apply convolutions with different dilation rates to the same input features. Atrous convolution expands the receptive field by introducing additional space (i.e., holes) in the convolution kernel without increasing the number of parameters or computational complexity. SAC utilizes this to capture features at different scales.

[0092] 2. Use of switching functions: Another key feature of SAC is the use of switching functions to combine the results of convolutions with different dilation rates. These switching functions are spatially dependent, meaning that each location in the feature map may have a different switch to control the output of SAC, making the network more flexible with respect to the size and scale of features.

[0093] 3. Conversion mechanism: SAC can convert a traditional convolutional layer into an SAC layer. This is achieved by using the same weights (except for a trainable difference) in the convolutional operations with different dilation rates. This conversion mechanism includes an average pooling layer and a 1x1 convolutional layer to implement the switching function.

[0094] 4. Structure design: The architecture of SAC consists of three main parts: two global context modules located before and after the SAC component respectively. These modules help to understand the image content more comprehensively, enabling the SAC component to work effectively in a broader context.

[0095] In terms of capturing multi-scale features, Switchable Atrous Convolution (SAC) has made technical improvements based on traditional Atrous Convolution. The following describes its key improvement points in combination with mathematical expressions:

[0096] 1. Application of different dilation rates. Traditional atrous convolution can be expressed as:

[0097]

[0098] where \(y(p)\) is the value at pixel \(p\) in the output feature map, \(w\) k is the weight of the convolution kernel, \(R\) is the standard convolution receptive field (e.g., 3×3), \(d\) is the dilation rate, and \(r\) k is the offset corresponding to the convolution kernel.

[0099] SAC performs feature extraction by introducing multiple dilation rates \(d\) i i.e.:

[0100]

[0101] Among them, i represents different porosity rates.

[0102] 2. Dual observation mechanism; A key improvement of SAC is that it performs convolutions with two different porosity rates on the same input feature:

[0103]

[0104] Here, d 1 and d 2 are two different porosity rates used to capture features at different scales.

[0105] 3. Fusion of switching functions; To switch between two porosity rates, SAC uses a spatially position-dependent switching function S(p):

[0106] y SAC (p) = S(p)·y 1 (p) + (1 - S(p))·y 2 (p)

[0107] In the formula, S(p) is a trainable weight that determines that by introducing S(p), SAC can adaptively select an appropriate receptive field according to the spatial information of the input feature map.

[0108] 4. Weight sharing and global context

[0109] SAC shares the same convolution weights in the convolution calculations with different porosity rates, but adds a trainable bias term Δw:

[0110] w′ k = w k + Δw

[0111] In this way, even when using different porosity rates, the parameters of the convolution kernel itself remain consistent, and only the influence is adjusted through Δw. This design reduces the computational overhead and at the same time ensures the ability to fuse multi-scale features.

[0112] In addition, SAC also combines a global context module (Global Context Module, GCM) to model the global information of the features, ensuring that the convolution operation not only focuses on local features but also can combine the global information of the entire image.

[0113] After convolution (Conv(512, 3, 2)), the size of the feature map becomes P5 / 32. The subsequent C3k2_OREPA_neck(512, True) module further expands the receptive field of the network. First, a 1×1 convolution is performed on the input feature map for feature channel mixing to extract local features. Then, a 3×3 frequency prior filter (similar to Gaussian smoothing or Fourier transform filtering) is used to enhance the global information of the features. This method can capture dependencies in a larger range, making the feature representation more global. The Scaling Layer scales the output and adjusts the range of feature values to better suit subsequent calculations. Finally, depthwise separable convolution and reparameterized 1×1 convolution are used to enhance the global information while reducing the computational complexity.

[0114] Then, the image is downsampled one last time using convolution (Conv(1024, 3, 2)), and the size of the feature map becomes P5 / 32.

[0115] The subsequent C3k2_SAConv(1024, True) module first performs a dual observation on the input features through switchable atrous convolution (SAC), extracts multi-scale information using different atrous rates, and adaptively fuses features of different scales using a spatially dependent switching function, expanding the receptive field while avoiding information loss. Subsequently, local information is further aggregated through a 3×3 standard convolution, combined with batch normalization (BN) to normalize the feature distribution, improve training stability, and enhance the non-linear representation ability through SiLU / Swish or ReLU activation. High-level features are further extracted.

[0116] Next, the SPPF(1024, 5) module performs spatial pyramid pooling. By replacing the multi-scale pooling of the traditional SPP with three consecutive 5×5 max pooling operations, the computational amount is reduced while multi-scale feature information is retained. Subsequently, feature fusion is performed through concatenation + 1×1 convolution for dimensionality reduction, increasing multi-scale information and enabling the network to better capture the features of targets of different sizes.

[0117] Finally, through the C2PSA module, the representation ability of high-level features is further enhanced, providing a richer feature map for the Head part.

[0118] (2) Next, enter the Head part, which is responsible for object detection based on the feature maps extracted by the Backbone. First, the P5 feature map is upsampled (Upsample, doubled), and then concatenated (Concat) with the P4 feature map (the fourth layer from the Backbone). The P5 feature map is upsampled to the same spatial size as the P4 feature map through Upsample to ensure pixel-by-pixel alignment. Subsequently, P5 (after upsampling) and P4 are concatenated in the channel dimension, so that the fused features contain both high-level semantic information and low-level detail information. The number of channels of the concatenated feature map is equal to the sum of the channels of P5 and P4, and the spatial size is the same as that of P4, generating a richer multi-scale feature map.

[0119] Subsequently, the C3k2_OREPA_neck(512,False) module is used to further extract the concatenated features and enhance their expressive power.

[0120] Next, the P4 feature map is also upsampled through Upsample and concatenated with the P3 feature map, and the C3k2_SAConv(256,False) module is used to extract the features of small-scale objects.

[0121] Then, the P3 feature map is downsampled again through convolution (Conv(256,3,2)), from P3 / 8 to P4 / 16, and concatenated with the previously processed P4 feature map to further enhance the mid-scale features.

[0122] Then, the input features are processed by the C3k2_OREPA_neck(512,False) module.

[0123] Next, the P4 feature map is downsampled from P4 / 16 to P5 / 32 through convolution (Conv(512,3,2)) again and concatenated with the P5 feature map.

[0124] Subsequently, the C3k2_SAConv(1024,True) module extracts the large-scale features.

[0125] Finally, object detection is performed through the ADDWConvHead module. The fused multi-scale features enter the Head for final detection. The ADDWConvHead, as the detection head, adopts the FADC (Frequency Adaptive Dilated Convolution) technology. By dynamically adjusting the convolution dilation rate, the receptive field is optimized according to the frequency components of different regions, so that regions with rich high-frequency details can obtain more refined feature extraction, while low-frequency regions can still maintain global perception ability. Finally, the ADDWConvHead combines object category classification, bounding box regression, and angle prediction (if OBB object detection is involved) to output the complete detection results, including category information, object location, and rotated box parameters.

[0126] In the final ADDWConvHead module, Frequency-Adaptive Dilated Convolution (FADC) is used. The core idea of FADC is to dynamically adjust the dilation rate according to the local frequency components of the image, enabling the network to adjust the receptive field according to the local changes in the image content. This can improve the detection performance in regions with rich details or dense high-frequency information, while reducing the computational amount in low-frequency regions.

[0127] Local frequency calculation (calculating high-frequency information using local gradient changes):

[0128]

[0129] In the formula, I is the input feature map, and f(x, y) is the frequency information at the pixel point (x, y), and the gradient can be calculated by the Sobel operator or Laplacian.

[0130] Adaptive dilation rate calculation (adapting the dilation rate according to frequency):

[0131] d(x, y) = d min +(d max -d min )·σ(f(x, y))

[0132] In the formula, d(x, y) is the dynamic dilation rate at the pixel point (x, y), d min , d max are the minimum and maximum values of the dilation rate respectively, and σ(f(x, y)) is the normalized frequency response function, and usually Sigmoid normalization can be used:

[0133]

[0134] In the formula, α is the control slope, and β is the control frequency offset.

[0135] FADC convolution calculation (introducing the dilation rate on the basis of the standard convolution):

[0136]

[0137] In the formula, y(p) is the value of the input feature map at the pixel p, w k is the weight of the convolution kernel, k is the index of the standard convolution kernel, and d(x, y) is calculated by the above formula.

[0138] Finally, through the FADC module, the network can effectively process multi-scale and multi-directional targets, improving the accuracy of target detection.

[0139] Exemplarily, the entire network of the YOLOv11 object detection model extracts features at different levels through the Backbone, and dynamically adjusts the receptive field through multi-scale feature fusion and FADC in the Head part, so as to efficiently detect objects of different sizes and directions in complex scenes, including different types of defects such as cracks, corrosion, and wear.

[0140] Exemplarily, at the same time, the PointNet++ model processes 3D point cloud data, extracts the geometric features of the blade surface, and defines its structure through the get_model class. The input of this model is a point cloud data xyz, whose shape is (B, 3, N) or (B, 6, N), where B is the batch size, 3 (or 6) represents the coordinates and normal information of the points, and N is the number of points in each point cloud. First, at the sa1 layer, the PointNet++ model uses the PointNetSetAbstraction module to process the input point cloud.

[0141] 1. Sampling Stage

[0142] First, a set of representative points is selected from the original point cloud using Farthest Point Sampling (FPS) or other sampling strategies as the centers of local regions. These representative points will serve as the basis for the next feature extraction.

[0143] 2. Neighborhood Selection

[0144] Then, for each sampled point, a local neighborhood of each sampled point is constructed by specifying a radius r or a fixed number N of neighboring points. The selection of neighboring points is usually based on a spatial distance metric, such as the Euclidean distance. For each sampled point p i , its set of neighboring points P i is selected to satisfy the following conditions:

[0145] P i = {p i || p i - p j || ≤ r}

[0146] where p i is the current sampled point, p j is the point in the neighborhood, and r is the given radius.

[0147] 3. Local Feature Learning

[0148] When extracting local features, PointNet++ uses the PointNet core module to process the set of points P in each local regioni Specifically, for each local region, it is processed through the following operation steps:

[0149] Feature Transformation: For each local point set P i , a Multi-Layer Perceptron (MLP) network is used for feature extraction to obtain the feature representation of the local region:

[0150]

[0151] where f i is the feature representation of the local point.

[0152] Global Feature Aggregation: Pooling operations (such as max pooling or average pooling) are performed on the local features to obtain the global feature representation of the local region:

[0153] f local = Pooling({f i | p i ∈ P i})

[0154] The feature vectors within the local region can be aggregated into a global feature vector through max pooling operation.

[0155] 1.4 Feature Fusion and Output

[0156] Finally, the local features are fused with the position information of the original points to output the global representation after local feature learning. By iterating this hierarchical sampling and feature learning multiple times, PointNet++ can capture the local structural features of the point cloud at different scales.

[0157] Specifically, sa1 selects 512 points from the original point cloud data and selects 32 points from the neighborhood of each point according to a radius of 0.2. These neighborhood features are mapped to a 128-dimensional feature space through a multi-layer perceptron (MLP). Next, sa2 further processes the output of sa1. With a larger radius of 0.4 and more neighborhood points (64), 128 points are selected for processing from it, so as to extract features in a larger range and map them to a 256-dimensional space. In the sa3 layer, the PointNet++ model aggregates all points for global feature extraction without point sampling, directly expanding the 256-dimensional features to 1024 dimensions, and further capturing global information through a deeper MLP. Feature extraction is performed using neighborhoods with multiple different radii. In PointNet++, multi-scale features can be extracted separately by using different radii in different sampling layers (Set Abstraction layers). When processing in the MLP layer, assuming there are features f i (each scale represents a different local area), then multi-scale feature fusion can be performed through the following formula:

[0158]

[0159] where w i is the weight of each scale, f i is the feature of the i-th scale, and MLP i is the MLP for each scale. Next, the network enters the fully connected layer. In fc1 and fc2, the 1024-dimensional features are respectively transformed through 512-dimensional and 256-dimensional transformations,

[0160] For each feature, calculate the mean μ batch of the feature in the entire batch and the variance

[0161]

[0162] Normalize each input sample x i by subtracting the mean μ batch and dividing by the standard deviation

[0163]

[0164] In the formula, ò is a small constant used to prevent division by zero errors.

[0165] Scale (through the learnable parameter γ) and shift (through the learnable parameter β) the normalized output:

[0166]

[0167] Where γ and β are parameters obtained through training and learning.

[0168] Finally, the output y i is the sample after batch normalization. Then, it is processed through batch normalization (bn1, bn2) and Dropout layers (drop1, drop2) to avoid overfitting. Finally, fc3 maps the features to the number of target classes (num_class) and uses log_softmax for classification output. In classification tasks, the value output by log-softmax is usually used together with the Negative Log-Likelihood Loss (NLLLoss) to calculate the logarithmic probability corresponding to the true label:

[0169]

[0170] Where z i is the original model output (usually referred to as "logits"). is the denominator of the softmax operation, representing the sum of the exponential values of all classes.

[0171] For each class i, y i is the logarithmic probability of class i, which is z i minus the logarithm of the sum of the exponentials of all classes Z j The loss calculation of the PointNet++ model is carried out through the get_loss class, using the negative log-likelihood loss (F.nll_loss) to calculate the error between the prediction result of the PointNet++ model and the true label, and returning the final loss value. The entire PointNet++ model extracts local and global features layer by layer through the Set Abstraction layer, further learns through the fully connected layer and performs classification, and finally outputs class predictions to complete the point cloud classification task. To accurately judge the tiny structural changes and potential defects on the blade surface, the detection of defects can be described by the following mathematical expression:

[0172]

[0173] Where is the predicted class probability. If the probability corresponding to the defect class is relatively high, it indicates that there may be a defect in a certain area of the point cloud data.

[0174] Step 3, data fusion and decision support;

[0175] By fusing the detection results of YOLOv11 with the 3D data analysis results of PointNet++, the 2D image features of YOLOv11 (such as the position and class information of the detection boxes) are concatenated with the 3D point cloud features of PointNet++ (such as the position and local features of the point cloud in space). The outputs of YOLOv11 and PointNet++ are processed separately (e.g., through classification or regression tasks), and then the final outputs are combined. Fusion can be performed by methods such as weighted average and logistic regression:

[0176] f fused = Concat(f 2D , f 3D )

[0177] where f 2D is the 2D feature from YOLOv11 (such as bounding boxes, class information, etc.), f 3D is the 3D feature from PointNet++ (such as local and global features of the point cloud), and f fused is the fused feature vector.

[0178] The system can obtain more comprehensive defect detection information. YOLOv11 is mainly responsible for identifying surface defects in image data, while PointNet++ focuses on morphological changes and depth analysis in 3D space. The data fusion of the two ensures a comprehensive defect analysis of the blade from different angles and dimensions.

[0179] The defect information after data fusion is sent to the decision support module. The system performs regression analysis based on the comprehensive threshold values of data such as the type, size, and depth of the defects, predicts the impact of the defects on the overall structure, automatically judges the severity of the defects, and provides corresponding repair suggestions. This module can provide actionable repair guidance for operators to help determine whether immediate repair or further inspection is required.

[0180] Step 4, real-time dynamic monitoring and target tracking;

[0181] The system uses the DeepSORT target tracking algorithm to track the detected defects. By combining the Kalman filter and the Hungarian algorithm, in target tracking, DeepSORT uses the Kalman filter to predict the position of the target, then uses the Hungarian algorithm for data association, and finally enhances the matching accuracy through deep features (such as appearance features).

[0182] State prediction of the Kalman filter:

[0183] x k = Ax k-1 + Bu k

[0184] Observation Update of the Kalman Filter:

[0185] x k = x k + K(y k - Hx k )

[0186] where y k is the observed value and K is the Kalman gain.

[0187] DeepSORT can track the dynamic changes of surface defects of blades in real time, ensuring accurate monitoring of defect evolution. Especially for the defect changes in real-time video streams, the system can continuously track and update the status of defects, providing a dynamic basis for subsequent maintenance decisions. The workflow diagram of the recognition system is as Figure 5 , which details the process of defect recognition and tracking evaluation.

[0188] This dynamic monitoring function is particularly important for complex defect evolution processes. For example, some micro-cracks or wear points may expand over time, and these potential risks can be identified in advance through dynamic tracking.

[0189] Step 5, Real-time Feedback and Operation Optimization;

[0190] The system displays all detection results and real-time monitoring data to the operator through an industrial display screen. The industrial display screen can not only show the current detection status of the blade, but also enable the operator to view specific areas in detail through image zoom-in and zoom-out buttons.

[0191] The operator can adjust the position and angle of the probe through a linear rocker and a control keyboard to ensure a comprehensive inspection of the surface of blades with complex shapes. Through these interactive operations, the system can optimize the detection process to ensure that every area can be accurately detected.

[0192] When the operator confirms that the defect has been effectively detected, further data analysis and archiving can be carried out through the report generated by the system. The report not only includes the specific location, type, and size of the defect, but also provides repair suggestions and possible treatment methods.

[0193] Step 6, Data Storage and Report Generation;

[0194] The system automatically generates a defect detection report, including information such as complete detection data, defect location, severity assessment, and repair suggestions. All detection data and reports can be stored in the system database for subsequent query and traceability.

[0195] The operator can also generate visual reports according to requirements, and use charts to display the distribution of defects and the trend of detection results. These reports play an important role in subsequent maintenance plans and maintenance decisions.

[0196] As can be seen from the above embodiments, the present invention proposes a method for detecting aeroengine blade defects based on multi-modal information fusion, which combines the real-time processing capabilities of two-dimensional images, three-dimensional point cloud data, and dynamic video streams, and uses deep learning technology to achieve efficient and accurate defect recognition, positioning, and damage assessment. By combining the YOLOv11 object detection algorithm, the PointNet++ point cloud processing network, and the DeepSORT object tracking algorithm, this system can achieve multi-modal fusion detection of the surface defects of aeroengine blades, effectively identify various defects in static images, and also track the development of defects in real time in dynamic video streams, improving the accuracy, efficiency, and reliability of detection. The technical solution of the present invention not only solves the limitations of traditional detection methods, but also fills the technical gap in current multi-modal fusion defect detection, providing more intelligent and efficient technical support for the regular inspection, online monitoring, and maintenance decision-making of aircraft.

[0197] Furthermore, the purpose of the present invention is to provide a method for detecting aeroengine blade defects based on multi-modal information fusion, aiming to comprehensively solve the problems of insufficient accuracy, poor robustness, and poor real-time performance caused by defect detection based on a single data source in the prior art, and especially aiming at the deficiencies of traditional detection methods in terms of low manual efficiency, high missed detection and false detection rates. By integrating the advantages of two-dimensional image data and three-dimensional point cloud data, the present invention can effectively improve the accuracy, robustness, and real-time performance of aeroengine blade defect detection, and at the same time realize the full automation of the detection process, significantly improve the detection efficiency, and reduce the influence of human factors on the detection results. The purposes of the present invention include:

[0198] Improve defect detection accuracy: Through multi-modal information fusion, the present invention combines the surface texture information of two-dimensional images and the geometric shape information of three-dimensional point clouds to comprehensively and accurately capture various defect features on the blade surface. Two-dimensional images can reveal surface texture details, while three-dimensional point clouds provide accurate geometric information of the blade. By integrating these two types of data, the system can effectively identify and locate various types of defects such as micro-cracks, corrosion, and wear, especially in the detection of complex shapes and micro-defects, making up for the deficiencies of traditional single data sources and avoiding the occurrence of missed detection and false detection phenomena, thus greatly improving the detection accuracy.

[0199] Enhance system robustness and overcome environmental interference: Since the blade surface often faces complex environmental conditions (such as light changes, reflections, dirt, etc.), traditional two-dimensional image-based defect detection methods are easily affected by environmental changes, resulting in a decrease in detection accuracy. The multi-modal data fusion technology of the present invention makes full use of the complementarity of image and point cloud data and can maintain high detection stability under adverse environmental conditions. Three-dimensional point cloud data is insensitive to light changes and reflection interference, while two-dimensional images can provide detailed texture information. The combination of the two enables the system to adapt to various working environments, improve robustness, and reduce the interference of external factors on the detection results.

[0200] Achieve fully automated real-time detection and dynamic monitoring: Traditional methods often rely on manual detection, which is inefficient, labor-intensive, and easily affected by human factors, resulting in missed or misdetected phenomena. Through the automated detection system of the present invention, fully automated processing from data acquisition to defect recognition can be achieved, greatly improving the detection efficiency and reducing the impact of human factors on the detection results. At the same time, combined with the YOLOv11 object detection algorithm, PointNet++ point cloud processing network, and DeepSORT object tracking algorithm, the system can not only detect defects in real time but also track the evolution process of defects in a dynamic video stream, providing real-time monitoring and feedback. This function can significantly improve the response speed and accuracy of the system in complex dynamic environments and meet the high-efficiency requirements for defect monitoring of modern aircraft engines.

[0201] Significantly improve detection efficiency and reduce human errors: The multi-modal defect detection system of the present invention can automatically process large-scale data, significantly improving the detection efficiency. Through the deep learning model, the system can automatically identify defects on the blade surface and perform precise classification and positioning, avoiding missed and misdetected phenomena caused by problems such as visual fatigue and improper operation in traditional manual detection. Compared with traditional manual inspection, the present invention greatly shortens the detection time, and due to fully automated processing, the detection results are more consistent, accurate, and less affected by human bias. This is particularly important for regular inspections and large-scale inspection tasks of high-precision components such as aircraft engine blades.

[0202] Reduce the missed detection and false detection rates and optimize the detection quality: In traditional detection methods, manual inspection is not only inefficient but also has relatively high missed detection and false detection rates. Especially when faced with complex blade surfaces or tiny cracks, it is difficult to guarantee the accuracy of manual detection. The multi-modal information fusion method of the present invention combines the advantages of two-dimensional images and three-dimensional point cloud data, can accurately identify tiny defects on complex surfaces, and avoid missed detection. Through the training and optimization of deep learning models, the system can significantly reduce the false detection rate, ensure that each defect can be accurately detected, reduce the misguidance to maintenance personnel, and avoid unnecessary maintenance and cost waste caused by false detection. Provide accurate three-dimensional damage assessment and decision support: The present invention can not only detect defects but also conduct precise damage assessment through three-dimensional point cloud data. The system can evaluate the depth, shape, and possible influence range of defects, providing a scientific basis for subsequent maintenance decisions. Through three-dimensional reconstruction and damage assessment, the system can judge the severity of defects, help maintenance personnel make accurate maintenance decisions, avoid over-repair or missed repair, optimize the allocation of maintenance resources, and thus improve the maintenance efficiency and service life of aircraft. Improve the scalability and adaptability of the system: The detection system design of the present invention has good scalability and can be flexibly adjusted according to different detection requirements. The system is not only applicable to the detection of aero-engine blades but can also be extended to the detection of other aircraft components and different types of industrial components. Through modular design, the system can add more data sources (such as infrared imaging, ultrasonic detection, etc.) according to requirements to further enhance the detection ability and adapt to detection requirements under different materials, structures, and environmental conditions.

[0203] Through the aero-inspection blade defect detection system based on multi-modal information fusion of the present invention, while solving the problems of insufficient accuracy, low efficiency, and large errors in traditional detection methods, it can provide more accurate and real-time defect detection and damage assessment. This will provide important support for the safe operation and maintenance decision-making of aircraft, effectively improve the reliability of aircraft, reduce the failure rate, and provide a strong technical guarantee for intelligent aircraft maintenance.

[0204] Example 2, the aero-inspection blade defect detection system with multi-modal information fusion provided by the embodiment of the present invention combines high-precision hardware devices and advanced algorithm software. The system aims to achieve automatic, precise, and efficient detection of surface defects of aero-engine blades and provide real-time dynamic monitoring and damage assessment. Through this system, the accuracy, robustness, and real-time performance of defect detection can be comprehensively improved, while significantly enhancing the detection efficiency, reducing the occurrence of missed detection and false detection, and supporting the safe operation and intelligent maintenance of aircraft. The software architecture of the system is based on advanced deep learning and multi-modal data fusion technologies and can efficiently and accurately complete defect detection and damage assessment tasks.

[0205] As Figure 6 shown, it specifically includes:

[0206] Data Acquisition and Preprocessing Module 1: The system synchronously acquires two-dimensional images and three-dimensional point cloud data on the blade surface through a high-definition image sensor and a three-dimensional laser scanner. After acquisition, the system preprocesses the data, including denoising, image enhancement, and smoothing of the three-dimensional point cloud data. These preprocessing steps ensure the data quality, making subsequent analysis and detection more accurate. Through advanced image processing techniques, problems such as light reflections and noise in the images are effectively removed, ensuring that the system can extract clear and accurate features from the original data.

[0207] Defect Detection and Classification Module 2: The system uses the YOLOv11 object detection model for object detection in 2D images, which can quickly detect and mark various defects on the blade surface, such as cracks, corrosion, wear, etc. The structure of the YOLO11n model consists of three main parts: feature extraction (Backbone), feature fusion (Neck), and object detection (Head). In the Backbone part, it is mainly composed of convolutional layers (Conv), C3k2_OREPA_neck, and C3k2_SAConv. By gradually extracting multi-scale features and enhancing the receptive field layer by layer, it captures low-level to high-level features in the image. The Neck part performs multi-scale feature fusion through modules such as SPPF, C2PSA, Concat, and Upsample, ensuring that object information at different scales can be effectively interconnected, and improving the detection ability for small and large objects. Finally, the Head part consists of ADDWConvHead, which is responsible for processing the fused features and finally outputting the category and bounding box of the object to complete the object detection task. Through the collaborative work of these three parts, YOLO11n can efficiently perform multi-scale and multi-directional object detection. At the same time, the PointNet++ deep learning network is used for feature extraction of 3D point cloud data. The structure of the PointNet2 model can also be divided into three main parts: feature extraction (Backbone), feature aggregation (Neck), and object classification (Head). In the feature extraction (Backbone) part, the model gradually extracts multi-scale local features through the SetAbstraction (SA) layer. Each SA layer captures different-scale geometric information by selecting a subset of points in the point cloud and aggregating features within its local neighborhood. This process gradually improves the receptive field layer by layer by setting different numbers of sampling points, radii, and neighborhood points, and gradually extracts features from local to global. In the feature aggregation (Neck) part, the network processes the features output from the SA layer through a series of fully connected layers and batch normalization (BatchNorm) modules, further integrating local features and converting them into high-dimensional feature representations suitable for classification. In the object classification (Head) part, PointNet2 performs the final mapping of the features through fully connected layers and uses the softmax function to output the classification result. By gradually extracting local features, aggregating global information, and final classification processing, PointNet2 can efficiently classify point cloud data or perform other tasks, achieving more comprehensive defect detection and avoiding false detection or missed detection caused by a single data source.

[0208] Data Fusion and Decision Support Module 3: It includes a data fusion module and a decision support module; the data fusion module comprehensively processes the detection results from YOLOv11 and PointNet++, utilizes multi-modal learning technology, combines the advantages of 2D images and 3D point clouds, and obtains a more accurate and comprehensive defect assessment. The decision support module based on deep learning automatically judges the severity of the defect according to information such as the location, size, and depth of the defect, and provides intelligent maintenance suggestions for the operator. This module can generate a defect severity report to help decision-makers make scientific and reasonable maintenance decisions.

[0209] Target Tracking and Dynamic Monitoring Module 4: The system uses the DeepSORT target tracking algorithm to dynamically track the detected defects, combines the Kalman filter and the Hungarian algorithm to achieve real-time tracking and status update of the defects. Through dynamic monitoring, the system can analyze the evolution process of the defects on the blade surface in real time, predict potential risks, and provide a basis for subsequent maintenance plans. This tracking function ensures the performance of the system in a dynamic environment, ensuring real-time feedback and accurate processing of defect changes.

[0210] User Interaction and Operation Interface Module 5: The user interacts through the system's control interface, uses a linear joystick to adjust the probe angle, adjust the detection area, view images in real time, and zoom in or out on data. The image zoom-in button and zoom-out button can help the operator view the details of the blade surface more clearly and accurately locate the defect position. The industrial display screen shows the detection results in real time and provides feedback to the operator, showing the working status and real-time images of the system, and supporting the operator's immediate decision-making.

[0211] Another exemplary Figure 7 is the schematic diagram of the multi-modal information fusion aeroengine borescope blade defect detection system provided by the embodiment of the present invention;

[0212] As can be seen from the above embodiments, the multi-modal information fusion aeroengine borescope blade defect detection system of the present invention, relying on its advanced hardware configuration and deep learning algorithm, can achieve high-precision and high-efficiency automated processing in the defect detection of aeroengine blades, significantly improving the performance and quality of defect detection. Compared with traditional manual inspection and conventional non-destructive testing techniques, the present invention demonstrates the following remarkable effects and advantages.

[0213] Significantly improve the defect detection accuracy: Traditional defect detection methods often rely on single two-dimensional images or traditional three-dimensional scanning techniques. When faced with complex blade surfaces and tiny cracks, false detections and missed detections are prone to occur. By combining high-resolution two-dimensional image data and high-precision three-dimensional point cloud data, this invention effectively makes up for their respective deficiencies, enhancing the comprehensiveness and accuracy of defect recognition. The system can not only accurately detect common defects such as cracks, corrosion, and wear on the blade surface, but also identify hidden defects such as tiny internal cracks, pores, and inclusions, ensuring comprehensive detection of all potential defects and reducing the risk of omission. Through the YOLOv11 object detection model and the PointNet++ three-dimensional data processing model, the system can simultaneously analyze image and three-dimensional point cloud data, comprehensively judge the type, location, and size of blade defects, achieving a substantial improvement in detection accuracy. This fusion technology is particularly suitable for the detection of complex structures and tiny defects, ensuring high accuracy of the detection results.

[0214] Greatly improve the detection efficiency: Compared with traditional manual detection methods, the automated detection system of this invention can greatly increase the detection speed. Especially when faced with a large number of blades or large-scale maintenance tasks, the high efficiency of the system enables the detection process to be completed quickly, significantly reducing the time required for manual detection. The automated process not only avoids errors caused by operator fatigue and experience differences, but also improves work efficiency, enabling the detection process to proceed continuously and stably, meeting the high-efficiency requirements of modern aviation maintenance. The system can automatically complete data acquisition, defect detection, evaluation, and report generation in a real-time dynamic environment, greatly shortening the detection cycle and providing immediate feedback to maintenance personnel. Operators only need to review and make decisions on the detection results without participating in cumbersome operation details, further improving the overall efficiency of the detection work. Enhance the system's robustness and adaptability. This system can adapt to various complex and changing working environments and has strong robustness. Whether it is light changes, reflection interference, or factors such as dirt and coatings on the blade surface, the system can effectively compensate through image enhancement technology and point cloud data processing technology, thus ensuring the accuracy of the detection results. Traditional manual inspections and detection methods with a single data source are easily affected by the external environment, resulting in unstable detection results. However, through the fusion of multi-modal information, this invention effectively reduces the interference of these environmental factors on the detection. The system can handle different types and shapes of blades and has high adaptability. Whether it is a blade with a standard shape or a blade with complex holes or surface textures, the system can accurately identify the defects therein, meeting the diverse detection requirements of aeroengine blades.

[0215] Implement real-time dynamic monitoring and target tracking: The built-in DeepSORT target tracking algorithm in the system, combined with the Kalman filter and the Hungarian algorithm, can track targets in real time during the dynamic detection process and continuously update the status of defects. This function is especially suitable for monitoring the development of defects on the blade surface. Especially during long-term detection or in complex environments, the system can dynamically track the defect location and timely detect potential deformation or crack expansion. Through real-time dynamic monitoring, the system can not only detect existing defects but also give early warnings for defects that may develop into serious problems, helping maintenance personnel to respond in a timely manner. The dynamic monitoring function enables the system to adapt to complex working environments, provides real-time feedback, and improves the timeliness and accuracy of defect detection.

[0216] Reduce the missed detection and false detection rates and enhance the consistency of detection results: In the traditional manual detection process, the subjective judgment and experience differences of operators are likely to lead to missed detections and false detections. Especially when the defects on the blade surface are not obvious or there are obstructions on the surface, the automated detection system of the present invention can reduce the influence of these human factors. Through automated defect identification and classification, the system can conduct a comprehensive inspection in all blade areas to ensure that no potential defects are missed. The deep learning model of the system has been trained and optimized with a large amount of data and can accurately distinguish different types of defects to avoid false detections. The types, sizes, and severities of defects will be accurately classified, providing a reliable basis for subsequent maintenance decisions. This mechanism greatly improves the consistency and reliability of detection results and avoids errors caused by subjective judgment in traditional methods.

[0217] Improve the scientificity and accuracy of maintenance decisions: The defect reports generated by the system not only include the detected defect locations and types but also can provide maintenance suggestions according to the severity of the defects. The system automatically evaluates the risk and impact degree of defects to help maintenance personnel determine which defects need to be processed immediately and which can be repaired later. This intelligent decision support enables maintenance personnel to make quick and reasonable decisions based on the accurate data of the system, reducing maintenance delays or errors caused by insufficient information or misjudgment. In some cases, the system can provide alternative solutions or recommended priorities for maintenance personnel to help them perform the most effective maintenance operations with limited time and resources. Through precise maintenance decision support, the system not only improves the maintenance efficiency but also extends the service life and maintenance cycle of the blades and reduces the maintenance cost.

[0218] High degree of automation and intelligence: The system of the present invention adopts deep learning and artificial intelligence technologies to achieve fully automated and intelligent operations throughout the process. Through automated data collection, defect detection, analysis, evaluation, and report generation, the system can significantly reduce manual intervention, reduce the occurrence of human errors, and provide real-time defect detection and repair suggestions. The intelligent design of the system greatly reduces the workload of operators, while improving work efficiency and accuracy. The adaptive learning ability of the system enables it to continuously optimize the model according to new data, ensuring that the detection accuracy can continue to improve as the data accumulates and the detection tasks increase. This self-learning mechanism enables the system to adapt to different types of blades and various complex detection scenarios, ensuring efficient operation during long-term use.

[0219] Promote the digital and intelligent transformation of aircraft maintenance: The present invention not only improves the defect detection accuracy and efficiency of aircraft engine blades, but also promotes the digital transformation of aircraft maintenance. By integrating big data, Internet of Things, and artificial intelligence technologies, the system can monitor and analyze the operating status of aircraft in real time, helping airlines and maintenance manufacturers achieve precise predictive maintenance. With the continuous development of intelligent maintenance technologies, the system can be further integrated with other devices and platforms to achieve full-process digital tracking and management, improving the efficiency of the entire aircraft maintenance industry. The present invention provides important technical support for the intelligent and digital transformation of the future aircraft maintenance industry, promoting the industry to develop in a more efficient, precise, and intelligent direction.

[0220] Reduce costs and improve aviation safety: Precise defect detection can reduce maintenance accidents and safety hazards caused by missed inspections, ensuring the safe operation of aircraft. By improving detection efficiency and accuracy, the system effectively reduces aviation material waste and unnecessary maintenance, optimizing the procurement and inventory management of aviation materials by airlines. Through precise defect reports and repair suggestions, the system helps airlines reduce maintenance costs, reduce inventory backlogs, and improve the utilization efficiency of resources. Providing precise defect detection and repair decision support greatly reduces the risk of aircraft failures caused by undetected defects in a timely manner, ensuring the long-term stable operation and flight safety of aircraft.

[0221] Experimental Example: A large airline company owns multiple commercial aircraft. Due to long-term operation under high temperature and high pressure, the engine blades of some aircraft have varying degrees of damage on the surface and inside. Traditional blade defect detection methods mainly rely on manual visual inspection and some non-destructive testing (NDT) techniques (such as ultrasonic and X-ray). However, due to the complex surface structure, irregular shape of the blades, and different material properties, these traditional methods often face problems of missed detection and false detection when detecting defects such as micro-cracks and pores. To solve this problem, the airline company decided to introduce a more efficient, accurate, and automated defect detection system to improve the accuracy of blade detection, reduce missed detection and false detection, and improve maintenance efficiency.

[0222] The goal of this implementation case is to solve the problems of inaccurate detection, low efficiency, and high missed detection rate in traditional methods by introducing an aviation borescope blade defect detection system based on multi-modal information fusion. The specific goals include:

[0223] Improve defect detection accuracy: Utilize the fusion of image data and 3D point cloud data to ensure the precise positioning and classification of defects on the complex blade surface.

[0224] Shorten the detection cycle: Through the automated detection process of the system, significantly improve the detection speed, reduce manual intervention, and shorten the time required for blade detection.

[0225] Reduce false detection and missed detection rates: Through deep learning and multi-modal data fusion, ensure the comprehensive detection of defects on each blade, and reduce errors caused by manual operation and experience differences.

[0226] Support intelligent decision-making and maintenance: Automatically generate maintenance reports based on the detection results, provide intelligent maintenance decision-making suggestions, and help maintenance personnel make decisions efficiently.

[0227] Implementation process:

[0228] 1. System initialization and self-check.

[0229] After the system is installed, the operator starts the electrical control cabinet for self-check. The self-check ensures that hardware such as the chassis, universal wheels, telescopic probe, laser scanner, and image sensor can all work normally, avoiding malfunctions during subsequent detections.

[0230] The operator adjusts the working parameters through the operation interface to ensure that the system can precisely control the movement of the equipment according to the morphology and detection requirements of different blades.

[0231] 2. Data acquisition.

[0232] After ensuring that all hardware is operating properly, the system uses a linear joystick to control the chassis to move to the position of the target blade. The high-definition image sensor and the 3D laser scanner are started simultaneously to begin data collection on the blade surface.

[0233] The high-definition image sensor captures high-resolution images of the blade surface, and the 3D laser scanner generates 3D point cloud data of the blade surface. The system simultaneously stores and transmits the real-time data to ensure data integrity.

[0234] 3. Defect detection and classification.

[0235] The collected data is preprocessed by the data processing module in the system to remove noise, enhance image quality, and standardize the point cloud data.

[0236] The YOLOv11 object detection model performs rapid defect detection on the two-dimensional image data, and the system can quickly identify whether there are defects such as cracks, corrosion, and wear on the blade surface.

[0237] At the same time, the PointNet++ model processes the 3D point cloud data, accurately extracts the geometric features of the blade surface, and identifies complex defects such as micro-cracks and pores.

[0238] 4. Data fusion and intelligent decision support.

[0239] The detection results of the image data and the point cloud data are comprehensively analyzed through multi-modal information fusion technology, and the system automatically generates a detailed report according to the type, location, and severity of the defects.

[0240] The report content includes information such as the classification of defects (such as cracks, corrosion, pores, etc.), location, size, and depth. The system also provides intelligent decision support, automatically generates maintenance suggestions, and prioritizes the maintenance according to the defect severity to help maintenance personnel formulate a scientific maintenance plan.

[0241] 5. Dynamic monitoring and real-time feedback.

[0242] During the entire detection process, the DeepSORT object tracking algorithm real-time tracks the detected defects to ensure that the changes in the defect location and status are updated in a timely manner. Especially for micro-cracks or developing defects, the system can capture their expansion trend and give an early warning.

[0243] The industrial display screen shows the blade detection process in real time, and the operator can view the specific defect information and tracking results to ensure that each defect point has been fully inspected.

[0244] 6. Data storage and subsequent analysis.

[0245] All the detection data, including images, point cloud data, and defect reports, are stored in the database by the system. This facilitates subsequent data analysis, trend monitoring, and historical data querying.

[0246] The system can also generate reports of the detection results and export them as electronic documents for convenient archiving and sharing. The reports contain detailed information for each stage of the detection process, ensuring the integrity and accuracy of each detection can be traced.

[0247] Implementation results.

[0248] Improve detection accuracy: The system can accurately identify various defects on the surface and inside of the blade, including tiny cracks, pores, etc. that are difficult to detect by traditional methods. Compared with the results of traditional X-ray and ultrasonic inspections, the missed detection rate and false detection rate are greatly reduced.

[0249] Enhance detection efficiency: The automated defect recognition and evaluation process reduces manual intervention and greatly improves the detection speed. Multiple blades can be inspected in a short time, avoiding the time consumption and efficiency bottleneck in manual inspection.

[0250] Reduce false and missed detections: Through the fusion of deep learning models and multi-modal data, the system can accurately distinguish different types of defects, avoiding false and missed detections in traditional methods. Especially when dealing with tiny cracks and surface defects, the system demonstrates excellent performance.

[0251] Improve the scientific nature of maintenance decisions: The detailed defect reports and intelligent maintenance suggestions generated by the system help maintenance personnel make more scientific and accurate maintenance decisions, ensuring the efficiency and precision of maintenance work.

[0252] Reduce maintenance costs: Through accurate defect detection and intelligent decision support, the system helps airlines reduce maintenance costs caused by false and missed detections. At the same time, it optimizes resource allocation and maintenance cycles, improving the utilization rate of aircraft materials.

[0253] In summary, the present invention provides an aviation borescope blade defect detection system based on multi-modal information fusion. By combining image data and point cloud data obtained from high-definition image sensors and three-dimensional laser scanners, and using deep learning algorithms (such as YOLOv11 and PointNet++) to efficiently and accurately detect and evaluate surface defects of the blade. This system can not only significantly improve the accuracy and efficiency of blade defect detection, but also through real-time dynamic monitoring and target tracking functions, achieve comprehensive monitoring and tracking of defects, reduce the missed detection and false detection rates, and ensure the accuracy of detection results.

[0254] The system of the present invention has the following remarkable advantages.

[0255] High-precision detection: Through multi-modal data fusion, the system can comprehensively identify and locate various defects on the blade surface, including cracks, corrosion, wear, etc., and can accurately detect tiny defects on complex surfaces.

[0256] High-efficiency automation: The automated detection process greatly improves the detection efficiency, reduces manual intervention, and significantly shortens the detection time, especially outstanding in large-scale detection tasks.

[0257] Dynamic monitoring and real-time feedback: Through target tracking and dynamic monitoring, the system can update the defect status in real time, timely warn of potential safety risks, and ensure the safe operation of the aircraft.

[0258] Intelligent decision support: Based on the detection results, the system can automatically generate maintenance suggestions and provide intelligent decision support to help maintenance personnel optimize the maintenance plan and improve the maintenance efficiency.

[0259] Cost reduction: By reducing misdetection and missed detection, improving detection accuracy and efficiency, the system effectively reduces the costs caused by maintenance delays, misjudgments, etc., and improves the utilization rate of maintenance resources.

[0260] The present invention not only solves the problems of insufficient accuracy, low efficiency, high missed detection and misdetection rate in traditional manual inspection methods, but also promotes the intelligent and automated process of aircraft maintenance, and has broad application prospects. The system can be widely applied to the defect detection of aero-engine blades, other components of aircraft, and other fields, providing strong technical support for the safe operation and maintenance of aircraft.

[0261] Therefore, the aero-probe blade defect detection system based on multi-modal information fusion of the present invention not only has high precision and high efficiency, but also shows great technical advantages in aspects such as intelligence and automation, conforms to the development trend of the aviation industry, and has significant economic benefits and social value.

[0262] The above is only a relatively preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A multi-modal information fusion method for detecting blade defects by aerial borescope, characterized in that: The method includes: S1, synchronously collects 2D images and 3D point cloud data of the blade surface through high-definition image sensors and 3D laser scanners; performs denoising, image enhancement and 3D point cloud data smoothing on the collected data; S2 uses the YOLOv11 target detection model to detect targets in two-dimensional images and mark various defects on the blade surface. It uses the PointNet++ deep learning network to extract features from three-dimensional point cloud data, maps features through a fully connected layer, and uses the softmax function to output classification results. It classifies point cloud data by gradually extracting local features, aggregating global information, and finally classifying them, completing comprehensive defect detection on the blade surface. S3, the fusion module comprehensively processes the detection results from the YOLOv11 target detection model and the PointNet++ deep learning network, and uses multimodal learning technology to combine two-dimensional images and three-dimensional point clouds to obtain a comprehensive defect assessment; the decision support module based on deep learning performs regression analysis based on the comprehensive threshold of the defect's location, size, and depth information, predicts the degree of impact of the defect on the overall structure, automatically determines the severity of the defect, and provides intelligent maintenance suggestions; S4 uses the DeepSORT target tracking algorithm to dynamically track the detected defects, and combines the Kalman filter and Hungarian algorithm to complete real-time tracking and status update of defects; through dynamic monitoring, the evolution process of blade surface defects is analyzed in real time to predict potential risks; S5 interacts through the control interface, uses the linear joystick to adjust the probe angle, adjust the detection area, view the image in real time, zoom in or out the data, view the blade surface details, and locate the defect position.

2. The multi-modal information fusion aviation borescope blade defect detection method according to claim 1 is characterized in that: In step S2, the YOLOv11 target detection model is used to perform target detection on the two-dimensional image, including: (1) The Backbone part extracts multi-scale features from the input image; (2) The Head part performs target detection based on the feature map extracted by the Backbone.

3. The multi-modal information fusion aviation borescope blade defect detection method according to claim 2 is characterized in that: In step (1), the Backbone part extracts multi-scale features from the input image, including: (a) After the image passes through the first layer of convolution, it is downsampled and the size of the feature map becomes half of the original image; the second layer of convolution extracts features and downsamples again, and the output feature map size becomes P2 / 4; the third layer uses C3k2_OREPA_neck, and the output feature map size is P3 / 8; (b) Downsampling is performed through another layer of convolution, and the feature map of P4 / 16 is output; the C3k2_SAConv module captures multi-scale features, and after convolution, the size of the feature map becomes P5 / 32; the C3k2_OREPA_neck module enhances the global information of the feature map; (c) Use convolution to downsample the image for the last time, and the feature map size becomes P5 / 32; the C3k2_SAConv module extracts high-level features; the SPPF module performs spatial pyramid pooling to increase multi-scale information; and the C2PSA module enhances the representation capability of high-level features.

4. The multi-modal information fusion aviation borescope blade defect detection method according to claim 2 is characterized in that: In step (2), the Head part performs target detection based on the feature map extracted by the Backbone, including: First, the P5 feature map is upsampled and concatenated with the P4 feature map to generate a multi-scale feature map; Secondly, the C3k2_OREPA_neck module is used to extract the spliced ​​features; the P4 feature map is upsampled and amplified and spliced ​​with the P3 feature map, and the features of small-scale targets are extracted through the C3k2_SAConv module; Then, the P3 feature map is downsampled again through convolution from P3 / 8 to P4 / 16, and is concatenated with the processed P4 feature map to enhance the mid-scale features; processed by the C3k2_OREPA_neck module; the P4 feature map is downsampled again through convolution from P4 / 16 to P5 / 32, and is concatenated with the P5 feature map; Subsequently, the C3k2_SAConv module extracts large-scale features; Finally, the ADDWConvHead module is used to perform object detection and output the category and bounding box of the object.

5. The multi-modal information fusion aviation borescope blade defect detection method according to claim 4 is characterized in that: In the ADDWConvHead module, the FADC module is used to dynamically adjust the expansion rate according to the local frequency components of the image, so that the network can adjust the receptive field according to the local changes in the image content and achieve the goal of processing multi-scale and multi-directional images.

6. The multi-modal information fusion aviation borescope blade defect detection method according to claim 1 is characterized in that: In step S2, the PointNet++ deep learning network is used to realize feature extraction of three-dimensional point cloud data, including: defining the structure through the get_model class, the input of the model is a point cloud data xyz, the shape is (B, 3, N) or (B, 6, N), where B is the batch size, 3 or 6 represents the coordinates and normal information of the point, and N is the number of points in each point cloud; First, in the sa1 layer, the PointNet++ model uses the PointNetSetAbstraction module to process the input point cloud and extract local features by specifying the number of samples, radius, and number of neighborhood points. sa1 selects 512 points from the original point cloud data and selects 32 points from the neighborhood of each point according to the radius of 0.

2. The neighborhood features are mapped to a 128-dimensional feature space through a multi-layer perceptron. Secondly, sa2 processes the output of sa1, selects 128 points from it for processing through a radius of 0.4 and neighborhood points, and maps them to a 256-dimensional space; Again, in the sa3 layer, the PointNet++ model aggregates all points to extract global features without sampling the points, expands the 256-dimensional features to 1024 dimensions, and captures global information through MLP; Then, the network enters the fully connected layer. In fc1 and fc2, the 1024-dimensional features are transformed into 512-dimensional and 256-dimensional features respectively, and then processed by batch normalization and Dropout layers; Finally, fc3 maps the features to the number of target categories and uses log_softmax for classification output.

7. The multi-modal information fusion aviation borescope blade defect detection method according to claim 6 is characterized in that: The loss calculation of the PointNet++ model is performed through the get_loss class, which uses negative log-likelihood loss to calculate the error between the PointNet++ model prediction result and the true label, and returns the final loss value.

8. The multi-modal information fusion aviation borescope blade defect detection method according to claim 1, characterized in that: After step S5, data storage and report generation are also performed, and a defect detection report is automatically generated, including complete detection data, defect location, severity assessment, and repair suggestion information; all detection data and reports are stored in the system database.

9. A multi-modal information fusion aviation borescope blade defect detection system, characterized in that: The system implements the multi-modal information fusion aviation borescope blade defect detection method according to any one of claims 1 to 8, and the system comprises: The data acquisition and preprocessing module (1) is used to synchronously acquire the two-dimensional image and three-dimensional point cloud data of the blade surface through a high-definition image sensor and a three-dimensional laser scanner; and to perform denoising, image enhancement and three-dimensional point cloud data smoothing on the acquired data; The defect detection and classification module (2) is used to use the YOLOv11 target detection model to perform target detection on the two-dimensional image and mark various defects on the blade surface. The PointNet++ deep learning network is used to extract the features of the three-dimensional point cloud data. The features are mapped through the fully connected layer and the softmax function is used to output the classification results. The point cloud data is classified by gradually extracting local features, aggregating global information and finally classifying the data to complete the comprehensive defect detection on the blade surface. Data fusion and decision support module (3): The fusion module comprehensively processes the detection results from the YOLOv11 target detection model and the PointNet++ deep learning network, and uses multimodal learning technology to combine two-dimensional images and three-dimensional point clouds to obtain a comprehensive defect assessment. The deep learning-based decision support module performs regression analysis based on the comprehensive threshold of the defect's location, size, and depth information, predicts the impact of the defect on the overall structure, automatically determines the severity of the defect, and provides intelligent maintenance suggestions. The target tracking and dynamic monitoring module (4) uses the DeepSORT target tracking algorithm to dynamically track the detected defects, and combines the Kalman filter and the Hungarian algorithm to complete the real-time tracking and status update of the defects. Through dynamic monitoring, the evolution process of the blade surface defects is analyzed in real time to predict potential risks. The user interaction and operation interface module (5) is used to interact through the control interface, use the linear rocker to adjust the probe angle, adjust the detection area, view the image in real time, zoom in or out the data, view the blade surface details, and locate the defect position.

10. The multi-modal information fusion aviation borescope blade defect detection system according to claim 9, characterized in that: The multimodal information fusion aviation borescope blade defect detection system is carried on a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it can realize the functions of the above-mentioned multimodal information fusion aviation borescope blade defect detection system.

Images