Borescope detection image damage identification method based on directional alternating pooling and decoupling feature matching

Abstract

The invention relates to the technical field of civil aviation engine borescope damage identification, in particular to a borescope image damage identification method based on directional alternating pooling and decoupling feature matching, which is characterized by comprising the following steps: establishing a borescope image damage identification model, and performing borescope image damage identification by using the borescope image damage identification model. Wherein the step of establishing the borescope image damage identification model comprises the steps of providing a bidirectional step-by-step pooling pyramid module BSPP, providing a two-dimensional decoupling feature matching aggregation network DFM-PAN, and integrating a hierarchical semantic-spatial decoupling attention mechanism HSDA in an original PAN path. The problem of feature mismatching in the engine multi-scale damage feature fusion process is effectively optimized, and the generalization ability and robustness of the model in coping with large-scale and small-scale damage scenes are remarkably improved. A normalized distance loss function is introduced to enhance the discrimination capability of the model in a complex scene.

Description

Technical field:

[0001] The present invention relates to the technical field of borescope damage identification for civil aviation engines. Specifically, the present invention relates to a borescope image damage identification method based on directional alternating pooling and decoupled feature matching, which can effectively alleviate matching deviations in the feature fusion process, thereby improving the detection accuracy of damage of different scales and complex shapes. Background technology:

[0002] With breakthroughs in artificial intelligence (AI) and hardware computing power, supervised deep learning technology, with convolutional neural networks as its mainstream architecture, has demonstrated tremendous potential in industrial surface defect detection thanks to its high-precision labeled samples. Since the advent of landmark algorithms such as FasterR-CNN, YOLO, and U-Net, deep learning-based visual inspection systems have gradually established a multi-scale, multi-modal damage recognition paradigm. This has provided breakthroughs in multi-target detection research, forcing the aviation industry to raise new requirements for civil aviation engine damage detection, and researchers have shifted their focus to automated and intelligent damage recognition.

[0003] Supervised object detection based on deep neural networks has undergone a transition from a two-stage approach to a single-stage approach. Two-stage detection networks, exemplified by Faster R-CNN, pioneered a "region proposal + refined classification" architecture. Specifically, this architecture, based on a Region Proposal Network (RPN), generates multiple anchor boxes of varying scales at each point in the feature map. These boxes are then evaluated for object or background, forming multiple candidate regions. Subsequently, feature resampling is performed through Region of Interest Pooling (ROI Pooling), achieving spatial alignment of regional features. The candidate regions are then mapped to a fixed-size output to ensure efficient processing of features at different scales by the fully connected layer. This cascaded mechanism, combining multi-scale anchor box refinement with subsequent refined segmentation, demonstrates strong adaptability and stability in multi-scale detection of industrial surface damage and forms a core approach to early deep learning-based industrial defect detection technology. Notably, to provide pixel-level segmentation information alongside object localization obtained by Faster R-CNN, researchers also proposed the Mask R-CNN model. Based on Faster R-CNN, this model constructs a branch for predicting the pixel-level mask of each object, which can simultaneously complete object detection and instance segmentation. Some scholars have improved this model and applied it to the field of borescope damage identification in civil aviation engines. It can accurately provide the shape, location and even size of the damage, so that damage detection can obtain more precise results. However, since the core requirements of borescope damage detection tasks are usually damage area positioning and type discrimination, although Mask R-CNN has the advantage of pixel-level segmentation, the introduction of the mask branch causes the model to bear a large computational load, resulting in a decrease in inference speed. At the same time, pixel-level segmentation does not show a significant gain in the quantitative analysis of typical damage such as cracks and ablation. On the contrary, the need to annotate high-precision mask data significantly increases the annotation cost. These have significant limitations in the application of civil aviation engine borescope damage.

[0004] As the industry's demand for efficient surface defect detection continues to rise, defect detectors based on two-stage architectures have faced significant limitations in real-time, high-precision inspection applications due to the computational overhead of their candidate region generation mechanisms and the spatial quantization bias caused by feature alignment. This has prompted the evolution of industrial visual inspection algorithms towards end-to-end architectures. With the emergence of single-stage detectors such as SSD, RetinaNet, and YOLO, object detection algorithms are evolving towards a dual-optimization paradigm of "high precision and high efficiency." Single-stage detectors utilize full-image scanning and multi-level feature fusion strategies to perform object classification and position regression directly on the input image, eliminating the need to generate candidate regions. This significantly reduces computational time while maintaining recognition accuracy, leading to their widespread application in the industrial sector in recent years. In the field of industrial defect detection, research focuses on achieving rapid and accurate identification and localization of multi-scale defects in complex backgrounds. Model improvements primarily focus on two aspects: first, improving feature extraction capabilities by modifying the backbone network structure or introducing attention mechanisms; and second, enhancing model performance by designing new feature propagation and interaction pathways. However, although these methods can improve the overall effect of model recognition to a certain extent, for the damage identification of core components inside civil aviation engines, there is still a lack of attention to the effective coordination of local details and overall morphological hierarchical relationships, which leads to the destruction of semantic consistency between cross-resolution feature maps, especially affecting the recognizability of small damage at low-resolution levels. Overall, first, the existing methods do not pay enough attention to the anisotropic structure of internal damage in civil aviation engines, and fail to fully consider the random directional characteristics of damage in borescope images, making it difficult to accurately capture the tiny details of damage in different directions. Secondly, due to the differences in the service environment of different components of civil aviation engines, their damage patterns show significant regional specificity, but existing methods generally ignore the coupling relationship between the type of internal engine damage and the engine part in which it is located, and lack the capture of background information, which affects the model's diagnostic accuracy for damage. Finally, when processing multi-scale damage features of borescope images, it is difficult to achieve semantic alignment between micron-scale cracks and macroscopic corrosion areas in the feature space. The feature fusion strategy cannot effectively coordinate the hierarchical relationship between local details and overall morphology. The existing element-by-element feature fusion strategy fails to establish an adaptive cross-scale semantic compensation mechanism, resulting in the destruction of semantic consistency between cross-resolution feature maps, especially affecting the recognizability of small damage at the low-resolution level. Summary of the invention:

[0005] Aiming at the characteristics of civil aviation engine damage in terms of background structure correlation and random anisotropy, the present invention proposes a borescope image damage recognition method based on directional alternating pooling and decoupled feature matching, which can effectively alleviate the matching deviation in the feature fusion process, thereby improving the detection accuracy of damage of different scales and complex morphologies.

[0006] The present invention is achieved by the following measures:

[0007] A borescope image damage recognition method based on directional alternating pooling and decoupled feature matching is characterized by establishing a borescope image damage recognition model and performing borescope image damage recognition using the borescope image damage recognition model, wherein establishing the borescope image damage recognition model comprises the following steps:

[0008] Step 1: To address the background structure correlation and random anisotropy of damage, a bidirectional step-by-step pooling pyramid module (BSPP) is proposed. This module uses a bidirectional step-by-step pooling mechanism to effectively extract the semantic information of the target while capturing anisotropic features that match the target semantics from different directions. It also mines the global structural information associated with these anisotropic features.

[0009] Step 2: To further capture the multi-scale characteristics of damage and optimize the model's feature matching and coordination processes, a two-dimensional decoupled feature matching aggregation network (DFM-PAN) is proposed. This integrates a hierarchical semantic-spatial decoupled attention mechanism (HSDA) into the existing PAN path to effectively address feature mismatches in the fusion of multi-scale engine damage features. This significantly improves the model's generalization and robustness in handling both large-scale and small-scale damage scenarios.

[0010] Step 3: The normalized Wasserstein distance loss function (NWD Loss) is introduced to enhance the model's discrimination ability in complex scenes.

[0011] The borescope image damage recognition model established in the present invention includes three parts: a backbone network, a neck fusion network, and a detection head. The borescope image damage recognition model is used to perform borescope image damage recognition, specifically including:

[0012] First, the borescope image damage recognition model preprocesses the input images, resizing them to a uniform 640×640 resolution to make them suitable for model training and inference. Data augmentation is then performed, combining the enhanced images into a single enhanced image to enrich the dataset.

[0013] Subsequently, the image is convolved layer by layer through Backbone, outputting feature maps containing rich semantic information for use by subsequent modules. After Backbone processing is completed, the feature maps enter the BSPP module, which captures multi-scale context and high-level directional semantic information in the feature maps through bidirectional alternating pooling operations, and inputs the extracted information into the C2PSA module for key feature screening. Next, the processed feature maps are input into the Neck module, which includes the FPN and PAN mechanisms. The FPN combines high-resolution low-level features with low-resolution high-level features through the FPN, and then outputs the two fused features to the DFM-PAN path. The DFM-PAN path embeds the hierarchical semantic-spatial decoupling attention mechanism HSDA to enhance and match features of different scales and ensure that information can flow efficiently between different layers. Finally, the NWD loss is introduced to improve the model's sensitivity to imbalanced samples and small target damage. Finally, the model integrates, transforms, and enhances the three feature maps of different sizes through the bottom-up DFM-PAN path, outputting them to the Head module to complete the target detection task.

[0014] The bidirectional step-by-step pooling pyramid module BSPP described in step 1 of the present invention designs a bidirectional step-by-step pooling operation mechanism named Max-Average Block, abbreviated as MA-Block. It consists of two parts, which respectively perform guided modeling for information in different directions:

[0015] Part 1: First, the input feature map X is max-pooled along the X-axis, that is, the maximum value is taken in each 1×k window. This operation compresses the information in the X-axis direction while keeping the Y-axis resolution unchanged. The size of the generated feature map is C×H×W / k. Then, the result of the X-axis max-pooling is averaged along the Y-axis, that is, the average value is taken in each k×1 window. The height k of the window is the same as the previous X-axis max-pooling. The size of the feature map generated in this step becomes C×H / k×W / k;

[0016] Part 2: First, re-perform maximum pooling on the original input feature map X along the Y axis with a window size of k×1. This will compress the information in the Y axis direction while maintaining the X axis resolution, and the output feature map size is C×H / k×W. Then, the result of the previous step of maximum pooling on the Y axis is averaged along the X axis, that is, the average value is taken in each 1×k window, and the window size is also k. The output feature map size of this step becomes C×H / k×W / k;

[0017] Finally, the results of the two parts are added element by element and the average is taken to ensure the stability of model training to obtain the final feature map. The size of the feature map obtained is C×H / k×W / k, which is consistent with the pooling output size in the original SPPF.

[0018] The specific calculation formula of the multi-directional alternating pooling operation in the present invention is as follows:

[0019] A: Input feature map: denoted as F, size is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively;

[0020] B: X-axis maximum pooling + Y-axis average pooling: Take the maximum value in each 1×k window of the input feature map in step A, set the stride to 1, and keep the output width unchanged by padding.

[0021]

[0022] Where F max_x Represents the feature map after the maximum pooling of the X axis, F max_x ∈R C×H×W ;

[0023] Perform average pooling of Fmax_x along the Y axis, taking the average value in each k×1 window with a stride of 1, and padding to ensure that the output height remains unchanged;

[0024]

[0025] Where F avg_yx Represents the feature map after average pooling on the Y axis, F avg_yx ∈R C×H×W ;

[0026] C: Y-axis maximum pooling + X-axis average pooling: Take the maximum value in each 1×k window of the input feature map in the original A step, set the stride to 1, and keep the output width unchanged by padding;

[0027]

[0028] Where, F max_y Represents the feature map after the maximum pooling of the X axis, F max_y ∈R C×H×W , perform average pooling of Fmax_y along the Y axis, take the average value in each k×1 window, with a stride of 1, and ensure the output height remains unchanged through appropriate padding;

[0029]

[0030] Where, F avg_yx Represents the feature map after the maximum pooling of the X axis, F avg_yx ∈RC×H×W ;

[0031] D: Bidirectional pooling fusion: Add the output results of steps B and C element by element and take the average to obtain the final output feature map F out (c,h,w):

[0032] F out (c,h,w)=[F avg_yx (c,h,w)+F avg_xy (c,h,w)] / 2 (29),

[0033] The size of the above pooling kernel k will remain consistent with the original SPPF pooling kernel size, with values ​​of 5, 9, and 13. The adaptive pooling pyramid module still uses multi-scale pooling, performing multi-directional alternating pooling operations at each scale. At the same time, the stride value is 1, and the padding operation is used to ensure that the size of the output feature map after pooling is the same as the size of the input feature map.

[0034] The dual-dimensional decoupled feature matching aggregation network (DFM-PAN) in step 2 of the present invention follows the bottom-up information transfer method, but constructs an effective collaborative processing strategy between global and local information through the hierarchical semantic-spatial decoupled attention mechanism (HSDA). That is, during the transfer process, each layer not only dynamically weights and filters the features from the previous layer in different dimensions, but also deeply explores directional information at different levels. The specific working method includes:

[0035] First, the bottom-level feature B1 of the FPN path is fused with the mid- and high-level semantic features F3 to generate the N3 feature, which combines shallow spatial position information with deep semantic understanding capabilities. Subsequently, N3 is passed to the bottom layer of the DFM-PAN path and processed by the HSDA module. HSDA adaptively weights the input features in the channel dimension and spatial dimension, and extracts key directional features from different layers, thereby enhancing the expression capabilities of small targets and heterogeneous damage areas. After HSDA dynamically enhances the features, the resulting features are concatenated with N4 to further integrate the contextual information from the intermediate layer. Then, the fused features are input to the C3k2 module for feature extraction to obtain the D2 feature, which is used for the small and medium target detection layer. Finally, the D2 feature is passed to the HSDA module for further optimization to enhance its ability to adapt to high-level semantic expression. The optimized features are fused with B3 to generate a new feature map, which is input to the C3k2 module for feature extraction to obtain the D3 feature, which is used as the prediction layer for medium and large targets.

[0036] The hierarchical semantic-spatial decoupled attention mechanism (HSDA) described in the present invention consists of two stages. The purpose of the first stage is to perform comprehensive feature extraction on the input feature map, optimize the mining of directional information and multi-scale information of damage by DFM-PAN, and provide sufficiently comprehensive and rich feature information for adaptive feature screening and enhancement. First, a convolution operation is performed on the input feature map. Through the action of the convolution kernel, the feature map can more clearly show the damage characteristics, and the subsequent sensitivity of the enhancement module to the morphological characteristics of damage at different scales and directions is enhanced; then, the input feature map is flattened along the X-axis and Y-axis respectively. Average pooling and maximum pooling are performed, and they are spliced ​​along the X-axis and Y-axis directions respectively to form feature maps of size C×H×2 and C×2×H; then, different feature maps are spliced ​​together through the splicing operation to form a feature map of size C×(H+W)×2; then, the convolution operation is applied to the spliced ​​feature map to process features in two directions at the same time, and the balance between global information and local information is achieved through the local receptive field and hierarchical stacking characteristics of the convolution, and a compact and concentrated feature map is constructed; finally, a batch normalization (BN) layer is added after the convolution operation to keep the distribution of different images consistent during the feature extraction process.

[0037] The first stage of the hierarchical semantic-spatial decoupled attention mechanism (HSDA) described in the present invention has the following specific steps:

[0038] Step (1) Input feature map, and represent the input feature map as X∈R C×H×W , where C represents the number of channels, H represents the height, and W represents the width;

[0039] Step (2): Global pooling in horizontal and vertical directions. Perform global average pooling in horizontal and vertical directions on the input feature map. The formula is as follows:

[0040]

[0041] In the formula, c represents the number of channels of the current feature map, h represents the height of the current feature map, and w represents the width of the current feature map. The global maximum pooling in the horizontal and vertical directions is calculated as follows:

[0042]

[0043] The average pooling and maximum pooling feature maps of the two are connected along the X direction and the Y direction respectively to form two new feature maps, as shown in the following formula:

[0044] F concat,h =[X avg,h (c,w);X max,h (c,w)]∈R C×2×W (5),

[0045] F concat,w =[X avg,w (c,h);X max,w (c,h)]∈R C×2×W (6);

[0046] Step (3): Global pooling splicing operation. After completing the above splicing, another splicing operation is performed along the Y direction to prepare for global convolution. The calculation formula is as follows:

[0047] F concat1 =[F concat,h ; F concat,w ]∈R C×(H+W)×2 (7),

[0048] Where, F concat,h Represents the global pooling information in the X direction, F concat,w Represents the global pooling information in the Y direction;

[0049] Step (4): Convolution operation, apply a 1×1 convolution operation to the newly formed feature map to generate an attention map:

[0050] M′ F =Conv 1×1 (F concat1 )(8),

[0051] Where M′ F represents the generated attention map; F concat1 Represents the global pooling feature of the splicing;

[0052] Step (5): Batch normalization and activation function processing to process the generated attention map:

[0053] M F =φ(BN(M′ F ))(10),

[0054] Where γ represents a learnable scale parameter; ε represents a constant, BN(·) represents the batch normalization layer operation, μ represents the mean of each channel in the attention map, and σ 2 represents the variance of each channel of the attention map, β represents the offset parameter, and φ(·) represents the activation function calculation.

[0055] The second stage of the hierarchical semantic-spatial decoupling attention mechanism (HSDA) of the present invention includes: decoupling the feature map output by the first stage in the spatial direction, decomposing the feature map of size C×(H+W)×1 into C×H×1 and C×1×W, performing convolution operations respectively to capture local detail features in a specific direction, and generating feature weights in different directions through activation functions; then, further solving the problem of spatial information loss caused by resolution difference and scale mismatch at the spatial level, performing average pooling and maximum pooling on the feature map output by the first stage along the channel dimension, and performing convolution operations on the feature map along the channel dimension. Splicing, using convolution operation to further refine features to form a feature map of size 1×(H+W)×1, and obtain weights through activation function, and then split the result to form feature maps of size 1×H×1 and 1×1×W, and then directly use activation function to obtain weights for the feature map of shape 1×(H+W)×1, which further enhances the model's sensitivity to directional features; finally, multiply the feature maps with weights generated at the semantic level and the spatial level to obtain the final calculation, and obtain the feature map with each feature weight as the final output of the hierarchical semantic-spatial decoupled attention mechanism (HSDA).

[0056] The specific steps of the second stage of the hierarchical semantic-spatial decoupled attention mechanism (HSDA) described in the present invention are as follows:

[0057] Step a: Channel dimension pooling. In order to further extract global information, based on the attention map generated above, the pooling information extraction on the channel dimension is performed again. The calculation formula is as follows:

[0058]

[0059] Where, Represents the feature map information in the channel dimension; F avg Represents the average pooling information in the channel dimension;

[0060] F max (i, j) represents the maximum pooling information in the channel dimension; then splicing is performed:

[0061] F concat2 =[F avg ; F max ]∈R 1×(H+W)×2 (13);

[0062] Step b: Weight acquisition in the channel dimension: Apply a 1×1 convolution operation to the newly formed feature map to generate attention features in the channel dimension:

[0063] M C =Conv 1×1 (F concat2) (14), A C =σ(M C ) (15),

[0064] Where A C Represents the weight information on the channel dimension; F concat2 Represents the pooling information in the channel dimension of the splicing;

[0065] Step c: Segmentation operation: Two segmentation operations are required here. The first segmentation operation is to segment the pooled attention features in the channel dimension in the horizontal and vertical directions respectively. The second segmentation operation is to segment the attention map obtained in the first stage D step. The first segmentation operation is as follows:

[0066] A h,C =A C [:,:H]∈R 1×H×1 (16), A w,C =A C [:,W:]∈R 1×1×W (17),

[0067] Where A C Represents the weight information on the channel dimension, H represents height, and W represents width; the second split operation is as follows:

[0068] M h,F =M F [:,:H]∈R C×H×1 (18), M w,F =M F [:,W:]∈R C×1×W (19),

[0069] Where M F Represents the attention map generated by the first stage D step, M h,F Represents the feature map in the vertical direction of the cut; M w,F Represents the feature map in the horizontal direction of the cut;

[0070] Step d: Obtaining weights in the horizontal and vertical directions: After obtaining the segmentation, horizontal and vertical feature maps, the weights in the horizontal and vertical directions are obtained respectively through the activation function. The specific calculation process is as follows: First, a convolution operation is required:

[0071] M h =Conv 1×1 (M h,F )(20), M w =Conv 1×1 (M w,F )(twenty one),

[0072] Where M w Represents the feature map in the horizontal direction generated by convolution; M h Representatives passed

[0073] The feature map in the vertical direction generated by convolution;

[0074] Through the activation function, the weights in the horizontal and vertical directions are obtained:

[0075] A h =σ(M h ) (22), A w =σ(M w ) (twenty three),

[0076] Where A w Represents the weight information obtained in the horizontal direction; A h Represents the obtained weight information in the vertical direction; Step e: Feature fusion: Multiply the obtained weight information in the horizontal, vertical and channel directions by the input feature map, and finally obtain a feature map with importance distinction:

[0077]

[0078] In the formula, Y represents the final generated feature map with importance distinction; X represents the input feature map; A C Represents the weight information on the channel dimension.

[0079] In step 3 of the present invention, the specific design of introducing NWD loss is as follows:

[0080] First, consider each target box as a two-dimensional Gaussian distribution:

[0081] N = (μ, Σ) (30), where μ represents the two-dimensional mean vector, which is used to describe the center position of the target box; Σ represents the covariance matrix corresponding to the target box; then, the 2-Wasserstein distance between the two-dimensional Gaussian distribution is used to measure the difference between the predicted box and the true box. Let the predicted box and the true box be N(μ1, Σ1) and N(μ2, Σ2) respectively. The square formula of the 2-Wasserstein distance is shown as follows:

[0082]

[0083] μ1 represents the center coordinate vector of the predicted box, μ2 represents the center coordinate vector of the real box; Σ1 represents the covariance matrix of the predicted box; Σ2 represents the covariance matrix of the real box; introduce the normalization process, and normalize the 2-Wasserstein distance to obtain NWD, as shown in the formula:

[0084] C represents the normalization constant.

[0085] To address the common problems of missed and false detections in manual borescope inspections, as well as the characteristics of civil aviation engine damage in terms of background structural correlation and random anisotropy, a bidirectional step-wise pooling pyramid module (BSPP) is designed. This module utilizes a directional alternating pooling strategy (MA-Block) to extract structural semantic information and anisotropic features, thereby enhancing the accuracy of damage localization. Furthermore, to address the multi-scale and diverse nature of damage morphology, a two-dimensional decoupled feature matching aggregation network (DFM-PAN) is constructed. By embedding a hierarchical semantic-spatial decoupled attention mechanism (HSDA), this network effectively alleviates the problem of inconsistent feature matching during the fusion of multi-scale features. Furthermore, to address the imbalanced data sample distribution in civil aviation engine damage detection, a loss function based on the normalized Wasserstein distance (NWD Loss) is introduced to enhance the model's discriminative ability under conditions of small sample sizes and class imbalance. Experimental results demonstrate that this method has good adaptability and application potential for damage detection in civil aviation engine borescope images. Description of the drawings:

[0086] Attachment Figure 1 It is a schematic diagram of the overall architecture of the model of the present invention.

[0087] Attachment Figure 2 This is the architecture diagram of the dual-guard decoupled feature matching aggregation network DFM-PAN in the present invention.

[0088] Attachment Figure 3 This is the overall architecture diagram of the hierarchical semantic-spatial decoupled attention mechanism HSDA in this invention.

[0089] Attachment Figure 4 2 is a visualization diagram of Grad-CAM executed on the BID-DET dataset according to an embodiment of the present invention.

[0090] Attachment Figure 5 1 is an overall architecture diagram of the bidirectional step-by-step pooling pyramid module BSPP in an embodiment of the present invention.

[0091] Attachment Figure 6 2 is a visualization diagram of Grad-CAM executed on the BID-DET dataset according to an embodiment of the present invention. Specific implementation method:

[0092] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0093] Example:

[0094] The brief architecture of the borescope image damage recognition method proposed in this example based on directional alternating pooling and decoupled feature matching is as follows: Figure 1As shown in the figure. First, in order to deal with the background structure correlation and random anisotropy of damage, a bidirectional stepwise pooling pyramid module (BSPP) is designed. This module adopts a bidirectional stepwise pooling operation mechanism (Max-Average Block, MA-Block), which can not only effectively extract the semantic information of the target, but also capture anisotropic features that match the target semantics from different directions, while mining the global structural information associated with these anisotropic features. This design can provide richer and more diverse feature information for subsequent modules and enhance the model's ability to accurately locate damage. Then, in order to further capture the multi-scale characteristics of damage and optimize the feature matching and coordination process of the model, a dual-dimensional feature matching path aggregation network (DFM-PAN) is designed. The network integrates the hierarchical semantic-spatial decoupled attention mechanism (HSDA) originally designed in this example into the original PAN path, effectively optimizing the feature mismatch problem that exists in the model during the fusion of multi-scale damage features of the engine, and significantly improving the generalization ability and robustness of the model in dealing with large-scale and small damage scenarios. Finally, in order to further improve the robustness of the model under unbalanced and small sample conditions, the normalized Wasserstein distance loss function (NWD Loss) is introduced to enhance the model's discrimination ability in complex scenarios. Through these improvements, it can meet the actual needs of damage detection in civil aviation engine borescope images and has good adaptability and practical value.

[0095] The model can be summarized as a model formed by a stack of various components, including three modules: the backbone network (Backbone), the neck fusion network (Neck) and the detection head (Head).

[0096] The overall workflow is as follows: First, the model preprocesses the input image, resizing it to a uniform 640×640 resolution for both training and inference. Data augmentation is then performed, merging the augmented images into a single augmented image to enrich the dataset. Backbone then performs convolution operations on the image layer by layer, outputting feature maps containing rich semantic information for use by subsequent modules. After Backbone processing, the feature maps enter the BSPP module. This module uses bidirectional alternating pooling to capture multi-scale context and high-level directional semantic information in the feature maps. This extracted information is then fed into the C2PSA module for key feature screening. The processed feature maps are then fed into the Neck module, which includes both FPN and PAN mechanisms. The FPN combines high-resolution low-level features with low-resolution high-level features. The two fused features are then fed into the DFM-PAN pathway, which embeds a hierarchical semantic-spatial decoupled attention (HSDA) mechanism to further enhance and align features at different scales, ensuring efficient information flow between layers. Finally, the NWD loss is introduced to improve the model's sensitivity to imbalanced samples and small object impairments. Ultimately, the model integrates, transforms, and enhances the three feature maps of different sizes through the bottom-up DFM-PAN pathway, outputting them to the Head module to complete the object detection task.

[0097] To address the problem of insufficient recognition of multi-scale and complex morphological damage during borescope damage identification of civil aviation engines due to unbalanced feature matching when the model fuses features, this solution redesigns the PAN structure of YOLO, namely, a two-dimensional decoupled feature matching aggregation network (DFM-PAN). This structure can effectively alleviate the matching deviation and high-level directional feature extraction in the feature fusion process, thereby improving the detection accuracy of damage of different scales and complex morphologies.

[0098] (1) Design Reasons for the Two-Dimensional Decoupled Feature Matching Aggregation Network (DFM-PAN)

[0099] The FPN and PAN of the YOLO11 neck network refer to top-down and bottom-up feature fusion paths. The FPN network primarily transfers deep, semantically strong features to higher-resolution shallow features through top-down upsampling and lateral connections. Its design goal is to supplement semantic information in shallow features, enabling them to better participate in object classification and thereby improving the model's classification capabilities. Because the upsampling process uses standard bilinear interpolation or nearest neighbor interpolation methods, the network is relatively stable when processing high-level semantic information. Furthermore, in lateral connections, the scale and number of channels of the feature maps are transformed using pre-designed transformations, making feature matching and fusion within this network relatively easy to control.

[0100] But the feature fusion of the PAN path is different. The PAN path mainly transfers the details in the shallow features from the bottom up to make up for the precise spatial information that may be lost when the deep features are positioned, thereby enhancing the model's ability to locate the target. However, since the shallow features themselves contain a large number of local details, and the deep features are more abstract in semantics and sparse in space, feature mismatch problems are prone to occur during the downsampling and fusion process, that is, feature maps at different levels cannot fully adapt to each other's characteristics when fused, which leads to inconsistent information expression or partial information loss. This problem is more obvious when the model detects targets with complex shapes or fuzzy edges, which directly affects the regression accuracy of the target boundary. Therefore, special attention needs to be paid to and the feature matching and alignment mechanism for the PAN path is designed. Specifically, feature mismatch is mainly manifested as follows:

[0101] I. Resolution difference leads to information loss. In the traditional PAN path, when low-level high-resolution features are downsampled and fused with high-level low-resolution features, feature alignment errors and boundary offsets are easily generated due to the inconsistency between the sampling method and the original resolution, thereby weakening spatial details and affecting positioning accuracy.

[0102] II. Semantic level differences lead to the coverage of detailed information. Although traditional PAN aims to enhance the spatial details in high-level semantic features, during the fusion process, high-level features tend to cover low-level detailed information due to their stronger global representation capabilities, making it difficult to fully express small objects and fine-grained features, thereby reducing detection accuracy.

[0103] III. Scale Mismatch Leads to Information Imbalance. In the traditional PAN pipeline, the direct fusion of features at different scales can easily lead to information imbalance. This is because high-level features are suitable for large objects, while low-level features are primarily used for small objects. Simply fusing the two cannot effectively reconcile the scale differences. Consequently, small objects are easily overwhelmed by high-level semantic features during detection, while large objects are interfered with by redundant low-level detail features during detection. This leads to an imbalance in information utilization between large and small objects, which in turn impairs the robustness and generalization ability of the model.

[0104] Overall, while the PAN approach enhances the ability to express spatial details, its bottom-up information fusion approach is also more susceptible to interference from factors such as resolution degradation, semantic differences, and scale mismatch. These issues pose significant limitations in the detection and identification of internal damage in civil aviation engines, especially for large-scale damage and small damage with random anisotropy and complex morphology:

[0105] I. Detecting the coverage and extent of large-scale damage typically relies on the synergy of high- and low-level features. If the scale and semantic information are not properly matched during the feature fusion process, the model may experience blurred boundaries when detecting large-scale damage, especially for large-scale damage with complex multi-directional morphology. Low-level features are affected by the mismatch between feature scale and semantics during the fusion process, and cannot fully express the global damage outline. This makes it difficult for the model to accurately locate the damage boundary, thereby reducing the detection accuracy of large-scale damage.

[0106] II. The characteristics of minor damage are often only visible in high-resolution, low-level feature maps. The traditional PAN pipeline suffers from feature overlap during feature fusion, causing the global information of high-level features to suppress or blur the details of low-level features. This causes minor damage details to be overlooked and weakened by the overlay of high-level semantic information. In particular, if minor damage details are not specifically addressed during the feature fusion process, the model can easily miss these critical details. Furthermore, when low-level detail information is over-smoothed, the model may mistake some normal features for damage.

[0107] Therefore, when constructing a recognition network for borescope damage detection of civil aviation engines, full attention should be paid to the optimization of the PAN path fusion strategy and the rationality of feature alignment. In particular, when processing multi-scale and multi-level damage features, the matching problem of feature scale and semantic information should be focused on, and the effective processing of directional features of the damage area and cross-scale feature information should be strengthened.

[0108] (2) Design of the two-dimensional decoupled feature matching aggregation network (DFM-PAN) structure

[0109] This case believes that in order to effectively solve the problem of feature information mismatch and easy loss of position and detail information in the PAN path, the key lies in implementing dynamic screening and weighting strategies for important features in the fusion process, thereby achieving adaptive feature enhancement, providing support for effective feature matching, and avoiding the coverage effect and information conflict problems in traditional feature fusion. In view of this, this case, based on the reference to the current mainstream attention mechanism, according to the complex morphological characteristics of civil aviation engine borehole detection damage such as random anisotropy, diversity and multi-scale, proposes a hierarchical semantic-spatial decoupled attention mechanism (HSDA), and integrates it into the traditional PAN path to form a two-dimensional decoupled feature matching aggregation network (DFM-PAN), such as Figure 2 shown.

[0110] By enhancing the model's ability to collaboratively process global and local information about directional and multi-scale features during bottom-up information transfer, DFM-PAN effectively mitigates information loss caused by feature differences, supports subsequent multi-level feature matching, and refines damage detection. DFM-PAN still uses a bottom-up information transfer approach, but it establishes an effective collaborative processing strategy between global and local information through the Hierarchical Semantic-Spatial Decoupled Attention (HSDA) mechanism. During the transfer process, each layer not only dynamically weights and filters features from the previous layer in different dimensions, but also deeply explores directional information at different levels.

[0111] The specific working principle of DFM-PAN is as follows: B1-3 are the three-scale feature maps output by the Backbone. First, the bottom-level features B1 of the FPN pathway are fused with the mid- and high-level semantic features F3 to generate the N3 feature. This feature combines shallow spatial location information with deep semantic understanding. N3 is then passed to the bottom layer of the DFM-PAN pathway and processed by the HSDA module. HSDA adaptively weights the input features in the channel and spatial dimensions, extracting key directional features from different layers, effectively enhancing the representation of small objects and heterogeneous lesion areas. After HSDA dynamically enhances the features, the resulting features are concatenated with N4 to further incorporate contextual information from the intermediate layers. This fused feature preserves semantic consistency in the FPN pathway and strengthens directional spatial response in the DFM-PAN pathway, thereby improving the matching between multi-scale features and alleviating feature mismatch issues such as inconsistent semantic hierarchies and spatial positioning bias in traditional PAN. This fused feature is then input into the C3k2 module for feature extraction, resulting in the D2 feature, which is used in the small and medium-sized object detection layer. Finally, the D2 features are passed to the HSDA module for further optimization to enhance their adaptability to high-level semantic representations. These optimized features are then fused with the B3 feature map to generate a new feature map. This is then fed into the C3k2 module for feature extraction, resulting in the D3 features that serve as the prediction layer for medium-to-large-scale objects. By continuously enhancing directional features and aligning semantics during this transfer and fusion process, DFM-PAN significantly improves the robustness and accuracy of multi-scale object detection.

[0112] This design enables the model to extract and enhance key features in a targeted manner when coordinating and matching multi-scale damage of civil aviation engines with random anisotropy, thereby effectively solving the feature mismatch problem faced by traditional PAN and improving the efficiency and accuracy of the feature fusion process. The design of the hierarchical semantic-spatial decoupled attention mechanism (HSDA) mainly consists of two stages, such as Figure 3 shown.

[0113] I. The first stage: Figure 3 As shown in the first stage, this stage aims to perform comprehensive feature extraction on the input feature map, optimizing DFM-PAN's mining of damage directional and multi-scale information, and providing sufficiently comprehensive and rich feature information for adaptive feature screening and enhancement. Considering that DFM-PAN needs to compress feature representations and increase the receptive field during information transmission, a convolution operation is first performed on the input feature map. Through the convolution kernel, the feature map more clearly displays damage characteristics, enhancing the module's subsequent sensitivity to damage morphological features at different scales and orientations.

[0114] Subsequently, the input feature maps are average pooled and maximum pooled along the X-axis and Y-axis directions, respectively, and then concatenated along the X-axis and Y-axis directions to form feature maps of size C×H×2 and C×2×H. The purpose of this is to establish a global receptive field in different dimensions and enhance HSDA's attention to spatial information, especially the differences in the direction of damage morphological features. Since maximum pooling can highlight strong response areas and capture significant features, average pooling can suppress noise and retain overall information. Therefore, the feature maps constructed by different pooling operations in this step can ensure that HSDA mines the overall and local structural information of directional damage, thereby improving the model's overall attention to diverse damage features and multi-scale damage features in subsequent detection tasks.

[0115] Next, the different feature maps are concatenated using a concatenation operation to form a feature map of size C × (H + W) × 2. This step converts information from different directions into a single feature map. While retaining key positional information, the module can simultaneously utilize features from both directions in subsequent processing, improving computational and feature utilization efficiency.

[0116] Then, a convolution operation is applied to the concatenated feature map, processing features in both directions simultaneously. The local receptive field and hierarchical stacking properties of convolution achieve a balance between global and local information, constructing a compact and focused feature map. The convolution kernel integrates directional features, automatically adapting and focusing on key damage locations, removing redundant information, and enabling precise characterization of damage features.

[0117] Finally, to mitigate the impact of image conditions such as lighting and angle on feature processing, a batch normalization (BN) layer is added after the convolution operation to maintain consistent distribution across different images during feature extraction. Furthermore, given the complex nonlinear structural characteristics of certain damages, such as the curved morphology of cracks and the irregular edges of wear, this example further feeds the features processed by the BN layer into an activation function. This nonlinear mapping enables the module to more fully focus on local details in the image, ensuring that key features are not missed or lost when detecting complex damage.

[0118] The specific calculation formula for this stage is as follows:

[0119] A. Input feature map. The input feature map is represented as X∈R C×H×W , where C represents the number of channels, H represents the height, and W represents the width.

[0120] B. Global pooling in horizontal and vertical directions. Perform global average pooling in horizontal and vertical directions on the input feature map. The calculation formula is as follows:

[0121]

[0122] Where c represents the number of channels of the current feature map;

[0123] h——represents the height of the current feature map;

[0124] w——represents the width of the current feature map.

[0125] At the same time, global maximum pooling in the horizontal and vertical directions is performed, which is calculated as follows:

[0126]

[0127] The average pooling and maximum pooling feature maps of the two are concatenated along the X direction and the Y direction respectively to form two new feature maps, as shown in the following formula:

[0128] F concat,h =[X avg,h (c,w);X max,h (c,w)]∈R C×2×W (5)

[0129] F concat,w =[X avg,w (c,h);X max,w (c,h)]∈R C×2×W (6)

[0130] C. Global pooling splicing operation. After completing the above splicing, another splicing operation is performed along the Y direction to prepare for global convolution. The calculation formula is as follows:

[0131] F concat1 =[F concat,h ; F concat,w ]∈R C×(H+W)×2 (7),

[0132] Where, F concat,h ——Represents the global pooling information in the X direction;

[0133] F concat,w ——Represents the global pooling information in the Y direction.

[0134] D. Convolution operation. Apply a 1×1 convolution operation to the newly formed feature map to generate an attention map:

[0135] M′ F =Conv 1×1 (F concat1 ) (8),

[0136] Where M′ F ——represents the generated attention map;

[0137] F concat1 ——Represents the global pooling feature of the splicing.

[0138] E. Batch normalization and activation function processing: Process the generated attention map:

[0139] M F =φ(BN(M′ F )) (10),

[0140] Where γ represents a learnable scale parameter; ε represents a constant; BN(·) represents the batch normalization layer operation; μ represents the mean of each channel in the attention map; σ 2 represents the variance of each channel of the attention map; β represents the offset parameter; φ(·) represents the activation function calculation.

[0141] The second stage: Figure 3 As shown in the second stage, based on the comprehensive acquisition of image spatial structure and feature information, the weight generation and application stage is entered. The core purpose of this stage is to generate attention weights based on the extracted features and apply these weights to the original input features, giving key features a higher response to optimize the accuracy of feature expression, thereby improving the robustness and accuracy of feature matching between high-level semantic information and low-level detail information.

[0142] In the feature map, the C, H, and W dimensions represent channels, height, and width, respectively. Generally, the channel dimension represents different feature layers in the feature map, reflecting different semantic information. The height and width dimensions of the feature map together constitute the spatial resolution of the feature map. Given this, HSDA, building on the features acquired in the first stage, addresses the issues mentioned in Section (1) at both the semantic and spatial levels based on the functionality of each dimension.

[0143] First, addressing the issue of detail information being obscured due to differences in semantic hierarchy at the semantic level requires focusing on important information at different levels within the feature map. The integration and alignment of high-level semantic information and low-level detail information depends on the coordination between semantic features in the channel dimension and detail features in the spatial dimension. Therefore, this example decouples the feature map output by the first stage in terms of spatial direction. The C×(H+W)×1 feature map is decomposed into C×H×1 and C×1×W convolution operations, respectively, to capture local detail features in specific directions. An activation function is then used to generate feature weights for each direction. This step is designed primarily based on the significant differences in detail granularity and information density between feature maps at different semantic levels. This directional decoupling allows the model to independently adjust feature responses for each direction at different semantic levels, thereby improving its ability to perceive details in different directions and comprehensively extracting important semantic information and detail features in each direction. Furthermore, independent convolution operations are used to effectively align features in each direction, ensuring information consistency across different directions. On this basis, adaptive directional feature enhancement is achieved through feature weighting, which enables the model to separately adjust the information distribution between feature maps at different semantic levels, so as to better retain and enhance the directional local details in the low-level semantic level, and avoid the masking and interference of high-level semantic information on low-level detail features.

[0144] Next, we further address the problem of spatial information loss caused by resolution differences and scale mismatches at the spatial level. Resolution differences are typically caused by a reduction or change in spatial size, leading to a loss of feature detail. The fundamental reason for information loss due to resolution differences during feature fusion is a lack of attention to spatial structural information. Given this, this example focuses on alignment and weighting in the spatial dimension. The feature maps output from the first stage are average pooled and max pooled along the channel dimension and then concatenated along the channel dimension. Convolution operations are used to further refine features, forming feature maps of size 1×(H+W)×1. Weights are then obtained through activation functions, and the results are then split to form feature maps of size 1×H×1 and 1×1×W. The purpose of this step is to preserve the salient feature details in the channel by pooling and concatenating the channel dimension, while more fully presenting the feature information in the spatial dimension, effectively avoiding the interference of feature differences at different levels between channels on feature focus during weight calculation. Subsequently, an activation function is directly applied to the feature map with a shape of 1×(H+W)×1 to obtain weights, which further enhances the model’s sensitivity to directional features. This allows the importance of high-resolution and low-resolution features in different directions to be dynamically adjusted during feature fusion, effectively ensuring the fusion of spatial details and global information as well as the fidelity of details when fusing high- and low-resolution features.

[0145] Finally, the weighted feature maps generated at the semantic and spatial levels are multiplied to obtain the final calculation, and the feature map with each feature weight is obtained as the final output of the hierarchical semantic-spatial decoupled attention mechanism (HSDA).

[0146] The specific calculation steps in this stage are as follows:

[0147] A. Channel dimension pooling: In order to further extract global information, based on the attention map generated above, the pooling information extraction in the channel dimension is performed again. The calculation formula is as follows:

[0148]

[0149] Where, ——Represents the feature map information in the channel dimension;

[0150] F avg ——Represents the average pooling information in the channel dimension;

[0151] F max (i,j)——represents the maximum pooling information in the channel dimension.

[0152] Then do the splicing:

[0153] F concat2 =[F avg; F max ]∈R 1×(H+W)×2 (13),

[0154] B. Weight acquisition in the channel dimension: Apply a 1×1 convolution operation to the newly formed feature map to generate attention features in the channel dimension:

[0155] M C =Conv 1×1 (F concat2 )(14), A C =σ(M C )(15)

[0156] Where A C ——Represents the weight information on the channel dimension;

[0157] F concat2 ——Represents the pooling information in the channel dimension of the splicing.

[0158] C. Segmentation operation: Two segmentation operations are required here. The first segmentation operation is to segment the pooled attention features in the channel dimension in the horizontal and vertical directions respectively. The second segmentation operation is to segment the attention map obtained in the first stage D step. The first segmentation operation is as follows:

[0159] A h,C =A C [:,:H]∈R 1×H×1 (16), A w,C =A C [:,W:]∈R 1×1×W (17), where A C represent

[0160] Weight information in the channel dimension; H represents height and W represents width.

[0161] The second split operation looks like this:

[0162] M h,F =M F [:,:H]∈R C×H×1 (18), M w,F =M F [:,W:]∈R C×1×W (19),

[0163] Where M F represents the attention map generated by the first stage D step; M h,F Represents the feature map in the vertical direction of the cut; M w,F Represents the feature map in the horizontal direction of the cut;

[0164] Obtaining weights in the horizontal and vertical directions: After obtaining the segmentation, horizontal and vertical feature maps, the weights in the horizontal and vertical directions are obtained through the activation function. The specific calculation process is as follows:

[0165] First, we need to perform a convolution operation:

[0166] M h =Conv 1×1 (M h,F )(20), M w =Conv 1×1 (M w,F ) (twenty one),

[0167] Where M w ——Represents the feature map in the horizontal direction generated by convolution;

[0168] M h ——Represents the feature map in the vertical direction generated by convolution.

[0169] Then, the weights in the horizontal and vertical directions are obtained through the activation function:

[0170] A h =σ(M h )(22), A w =σ(M w )(twenty three),

[0171] Where A w Represents the weight information obtained in the horizontal direction; A h Represents the obtained weight information in the vertical direction.

[0172] Feature fusion: Multiply the obtained weight information of the horizontal, vertical and channel dimensions by the input feature map to finally obtain a feature map with importance distinction.

[0173]

[0174] Where Y represents the final generated feature map with importance distinction; X represents the input feature map; A C Represents the weight information on the channel dimension.

[0175] Discussion on the properties of dual-dimensional decoupled feature matching aggregation network (DFM-PAN)

[0176] DFM-PAN is designed to mitigate the loss of detail information caused by inconsistent feature matching by embedding the HSDA mechanism. This mechanism dynamically enhances the discriminative spatial position and channel dimensionality responses of low-level features before feature fusion. This further strengthens the directional structure and salient region representation of low-level features, enabling them to have stronger representational power and engagement when fused with high-level semantic features. This in turn mitigates the dilution and overlay issues of multi-scale target features during information transfer. DFM-PAN effectively improves the model's sensitivity to weakly characterized areas such as tiny defects and ablation edges, enhancing its detection accuracy and robustness in borescope inspection environments.

[0177] In order to further verify the effectiveness of DFM-PAN in feature fusion, this example compares and analyzes the feature response distribution of DFM-PAN and traditional PAN paths on the civil aviation engine borescope damage detection dataset (BID-DET) based on the Grad-CAM visualization method. Figure 4 As shown in the figure, DFM-PAN is able to more clearly focus on the target area, with a more concentrated response area and clearer boundaries, significantly reducing activation noise in non-target areas, reflecting its improved spatial modeling capabilities. Furthermore, when faced with input images with densely distributed multiple targets or blurred target boundaries, DFM-PAN demonstrates greater robustness and generalization capabilities, able to simultaneously capture multiple key areas and maintain good response balance, avoiding the feature mismatch and detection offset issues that are common with traditional PAN in such scenarios.

[0178] From the visualization results, it can be seen that DFM-PAN not only optimizes the transmission path of feature information, but also significantly improves the accuracy and stability of the model in complex detection tasks, verifying its effectiveness and practical value as the core path of model feature fusion in damage identification tasks.

[0179] When training a civil aviation engine damage recognition model, the traditional pooling pyramid module's single pooling operation makes it difficult for the model to simultaneously learn the structural associations between local anomalies in the damaged area and the background environment, neglecting the acquisition of some key spatial information, thus affecting model performance. In this example, a bidirectional step-wise pooling pyramid module (BSPP) is designed to specifically extract and fuse local extreme values ​​and global background information to supplement the spatial distribution characteristics of high-level semantic feature maps, thereby improving the ability to accurately recognize damage edges and diverse morphologies.

[0180] (1) Design Reasons for Bidirectional Step-by-Step Pooling Pyramid Module (BSPP)

[0181] In the YOLO model, the SPPF (Spatial Pyramid Pooling-Fast) module achieves fast and efficient multi-scale feature fusion through cascaded maximum pooling operations. Its core mechanism is to perform repeated maximum pooling operations with a fixed-size pooling kernel, gradually expanding the receptive field to obtain local and larger area features. On this basis, the module splices the initial feature map with the pooling results of each stage along the channel dimension to form a rich composite feature representation, which significantly improves the model's detection accuracy for multi-scale targets. Usually, this module is deployed at the end of the backbone network to provide more discriminative multi-scale feature input for the subsequent detection head. However, in this case, it was found that although the SPPF module optimized the extraction of multi-scale features, it only used maximum pooling to extract features in the pooling window, and the pooling operations in all directions were the same, which easily led to the omission of some key details during information transmission and weakened the perception of the global context. In particular, in the identification of damage in civil aviation engine borehole detection, it is more likely to affect the accurate capture of damage edges and morphological features. Specifically, there are the following problems:

[0182] I. The single pooling method has limitations. Compared with the mobile target detection task, the detection targets in the civil aviation engine borescope images are usually in a static state and are mostly distributed in a specific detection environment. In addition, the damage types corresponding to different core components have significant differences in morphological structure and performance characteristics. Therefore, the background distribution information around the damaged area is of great significance for understanding the nature of the damage and achieving accurate type discrimination. However, the SPPF module only uses a single maximum pooling operation, which easily ignores the non-salient areas within the pooling window during the feature extraction process, thereby losing high-order semantic features such as the spatial structural trend of the target edge, the weak response of the blurred area, and the discriminative background semantic clues. These fine-grained structural features are crucial for subsequent feature fusion and matching, especially in improving the accuracy of damage location and the reliability of boundary and morphological recognition.

[0183] II. Lack of effective capture of anisotropic information. Civil aviation engine borehole inspection damage has significant anisotropic characteristics, that is, there are random differences in the way certain damages appear and their intensity in different directions. This difference is usually reflected in the size, area and distribution of the damage. In high-level semantic feature maps, this anisotropy is usually reflected in the feature response intensity or distribution trend in different directions. However, the SPPF structure is based on the standard maximum pooling operation and lacks a modeling and enhancement mechanism for directional features. It is difficult to effectively capture the weak but critical directional features caused by anisotropy. This is particularly insufficient when modeling damage edges and fine structures, which can easily lead to missed detections and reduce the model's adaptability to complex damage. It is worth mentioning that compared with other models in the YOLO series, YOLO11 adds a C2PSA module to the backbone network. This module aims to enhance feature extraction through a cross-stage partial self-attention mechanism and has high requirements for the richness of input features. This example found that when this module is introduced, if the features output by the SPPF module lack diverse background information and key details of the damage characteristics, then the C2PSA module may lack certain spatial distribution and contextual clues when focusing on the damage characteristics, resulting in an incomplete understanding of the damage area.

[0184] Therefore, considering the above problems, when designing a network for damage identification of civil aviation engine borescopes, the pooling pyramid module in the backbone network should be designed specifically to ensure that the module is more suitable for the characteristics of civil aviation engine borescope images, so as to improve the accuracy of the model in damage identification.

[0185] (2) Bidirectional step-by-step pooling pyramid module (BSPP) structure design

[0186] This paper argues that the key to solving the problems of information loss and lack of directionality in the SPPF module lies in redesigning the pooling operation mode when performing multi-scale pooling operations. In view of this, this paper proposes a bidirectional step-by-step pooling pyramid module (BSPP) to enhance the backbone network's ability to extract features of damage at different scales in different directions while maintaining its ability to perceive global context information. Its structure is as follows: Figure 5 shown.

[0187] The BSPP module designs a bidirectional step-by-step pooling mechanism, named Max-Average Block, abbreviated as MA-Block. Specifically, the mechanism consists of two parts, each of which guides modeling information in different directions:

[0188] I. Part 1. First, the input feature map X is max-pooled along the X-axis, taking the maximum value in each 1×k window. This operation compresses the information along the X-axis while maintaining the Y-axis resolution, resulting in a feature map of size C×H×W / k. Next, the result of the X-axis max-pooling is average-pooled along the Y-axis, taking the average value in each k×1 window, where the height k is the same as the previous X-axis max-pooling. The resulting feature map is now of size C×H / k×W / k.

[0189] II. Part 2. First, the original input feature map X is re-max-pooled along the Y-axis with a window size of k×1. This compresses the information along the Y-axis while maintaining the X-axis resolution, resulting in an output feature map of size C×H / k×W. Next, the results of the previous Y-axis max-pooling are average-pooled along the X-axis, taking the average value in each 1×k window, also with a window size of k. The output feature map of this step becomes C×H / k×W / k.

[0190] Finally, the results of parts (1) and (1) are added element by element and the average is taken to ensure the stability of model training and obtain the final feature map. The size of the feature map obtained in this way is C×H / k×W / k, which is consistent with the pooling output size in the original SPPF. The purpose of this bidirectional step-by-step pooling operation, namely the MA-Block design, is to preferentially capture feature information with local extreme values ​​in multiple directions, that is, the most significant damage area, thereby enhancing the model's perception and representation ability of directional damage texture. Subsequently, average pooling is performed in the vertical direction of these significant directional features to integrate and retain the global background information around these significant damage areas, so that the model can understand the relationship between background and damage based on the significant damage area, and supplement the perception information of context and spatial distribution at the high-level semantic level. Finally, the results in different directions are added together. While fusing the feature information of each direction and enhancing the complete expression of the feature, it provides the model with the ability to extract some damage features that are not significant in some directions, avoiding information loss.

[0191] The specific calculation formula for the multi-directional alternating pooling operation is as follows:

[0192] A. Input feature map: denoted as F, of size C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively.

[0193] BX-axis maximum pooling + Y-axis average pooling: Take the maximum value in each 1×k window of the input feature map in step A, set the stride to 1, and keep the output width unchanged through appropriate padding.

[0194]

[0195] Where, F max_x ——Represents the feature map after the maximum pooling of the X axis, F max_x ∈R C×H×W ;

[0196] F max_x Average pooling is performed along the Y-axis, taking the average value in each k×1 window with a stride of 1, and ensuring the output height remains unchanged through appropriate padding.

[0197]

[0198] Where, F avg_yx ——Represents the feature map after average pooling on the Y axis, F avg_yx ∈R C×H×W ;

[0199] CY axis maximum pooling + X axis average pooling: Take the maximum value in each 1×k window of the original A step input feature map, set the stride to 1, and keep the output width unchanged through appropriate padding.

[0200]

[0201] Where, F max_y ——Represents the feature map after the maximum pooling of the X axis, F max_y ∈R C×H×W ;

[0202] F max_y Average pooling is performed along the Y-axis, taking the average value in each k×1 window with a stride of 1, and ensuring the output height remains unchanged through appropriate padding.

[0203]

[0204] Where, F avg_yx ——Represents the feature map after the maximum pooling of the X axis, F avg_yx ∈R C×H×W ;

[0205] D. Bidirectional pooling fusion: Add the output results of steps B and C element by element and take the average to obtain the final output feature map F out (c,h,w):

[0206] F out (c,h,w)=[F avg_yx (c,h,w)+F avg_xy (c,h,w)] / 2(29),

[0207] The size of the pooling kernel k will remain consistent with the original SPPF pooling kernel size, with values ​​of 5, 9, and 13. The adaptive pooling pyramid module still uses multi-scale pooling, performing multi-directional alternating pooling operations at each scale. At the same time, the stride value is set to 1, and padding is used to ensure that the size of the output feature map after pooling is the same as the input feature map.

[0208] (3) Discussion on the properties of the Bidirectional Step Pooling Pyramid Module (BSPP)

[0209] Inadequate representation of high-level semantic information in aircraft engine borescope damage recognition models severely limits their performance in complex backgrounds and multi-scale damage conditions. Analysis of the feature extraction process within the YOLO11 backbone network revealed that the lack of discriminative high-level semantic information stems from insufficient capture of global context during feature aggregation. The SPPF module, as the core link in information aggregation and transformation, directly determines the integrity of high-level semantic representation.

[0210] Based on the above situation, this paper proposes to design a bidirectional step-wise pooling pyramid module (BSPP) while maintaining the stability of the overall network architecture, aiming to enhance the depth and directional representation capabilities of feature fusion. Compared with the traditional SPPF module, the BSPP module shows significant advantages in multi-scale semantic integration and directional information preservation. Its bidirectional distributed pooling strategy not only improves the collaborative modeling capabilities of local details and global context, but also optimizes the response intensity and spatial consistency of the damaged area in the backbone network feature pathway, significantly alleviating the spatial information degradation problem caused by the pooling operation in the original structure, thereby achieving more accurate multi-scale damage localization in complex scenes.

[0211] This example uses the Grad-CAM visualization method to compare and analyze the characteristic response distributions of BSPP and SPPF on the civil aviation engine borescope damage detection dataset (BID-DET). Figure 6 As shown in the figure, the BSPP module covers more discriminative areas and provides richer semantic structural feature information for subsequent modules. In contrast, the original SPPF module's attention in borescope images is relatively scattered, resulting in unclear target feature extraction. This fully demonstrates that the improved BSPP module in this example can fully represent salient features while avoiding information omission.

[0212] In view of the characteristics of small targets and imbalanced sample categories in civil aviation engine borescope damage, this example introduces the Normalized Wasserstein Distance (NWD) as a regression loss function to replace the traditional detection box regression loss function. Generally speaking, the traditional detection box regression loss function relies on the geometric overlap between the predicted box and the true box for optimization. However, when the target size is very small, even if the predicted box is relatively close to the target area, the intersection-over-union ratio may still be extremely small, resulting in extremely sparse gradient information, affecting the model's convergence speed for small targets and the final positioning accuracy. At the same time, the damage category samples of civil aviation engine internal damage are also extremely unbalanced. During training, the traditional loss function is more likely to converge to the frequently appearing main class, while the spatial characteristics of rare damage classes cannot be further explored.

[0213] In view of this, this example introduces the NWD loss to improve the traditional detection box regression loss function. NWD models the bounding box as a two-dimensional Gaussian distribution and measures the difference between the predicted box and the true box by calculating the Wasserstein distance between the two Gaussian distributions. This distribution-based measurement method does not rely on the intersection-over-union ratio and naturally has stronger gradient stability and continuity. It is particularly suitable for the detection of small targets and targets with uncertain boundaries. At the same time, by measuring the spatial structural differences between the predicted box and the true box distribution, NWD can still provide significant optimization directions for the model even if the number of samples is small, as long as there are certain regularities or characteristics in the spatial positions. This mechanism enables the model to maintain a high detection sensitivity for low-frequency damage categories.

[0214] The specific design is as follows:

[0215] First, each target box is considered as a two-dimensional Gaussian distribution as shown in Eq.

[0216] N = (μ, Σ) (30), where μ represents the two-dimensional mean vector, which is used to describe the center position of the target box, and Σ represents the covariance matrix corresponding to the target box. Then, the 2-Wasserstein distance between the two-dimensional Gaussian distribution is used to measure the difference between the predicted box and the true box. Let the predicted box and the true box be N(μ1, Σ1) and N(μ2, Σ2) respectively. The square formula of their 2-Wasserstein distance is shown in the formula.

[0217]

[0218] Where μ1 represents the center coordinate vector of the prediction box;

[0219] μ2——represents the center coordinate vector of the real frame;

[0220] Σ1 — represents the covariance matrix of the predicted box;

[0221] Σ2 — represents the covariance matrix of the ground-truth box.

[0222] In order to make the distance more comparable for objects of different scales, this example introduces a normalization process to normalize the 2-Wasserstein distance to obtain NWD, as shown in the formula.

[0223] C represents the normalization constant. The introduction of this loss function can effectively solve the gradient vanishing problem faced by traditional loss functions in small target detection. At the same time, when the sample categories are extremely unbalanced, it can still maintain high sensitivity to rare damage, providing strong support for damage identification in civil aviation engines.

Claims

1. A borescope image damage recognition method based on directional alternating pooling and decoupled feature matching, characterized in that: Establishing a borescope image damage recognition model, and using the borescope image damage recognition model to perform borescope image damage recognition, wherein establishing the borescope image damage recognition model comprises the following steps: Step 1: To address the background structure correlation and random anisotropy of damage, a bidirectional step-by-step pooling pyramid module (BSPP) is proposed. This module uses a bidirectional step-by-step pooling mechanism to effectively extract the semantic information of the target while capturing anisotropic features that match the target semantics from different directions. It also mines the global structural information associated with these anisotropic features. Step 2: To further capture the multi-scale characteristics of damage and optimize the model's feature matching and coordination processes, a two-dimensional decoupled feature matching aggregation network (DFM-PAN) is proposed. This integrates a hierarchical semantic-spatial decoupled attention mechanism (HSDA) into the existing PAN path to effectively address feature mismatches in the fusion of multi-scale engine damage features. This significantly improves the model's generalization and robustness in handling both large-scale and small-scale damage scenarios. Step 3: The normalized Wasserstein distance loss function (NWD Loss) is introduced to enhance the model's discrimination ability in complex scenes.

2. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 1 is characterized in that: The borescope image damage recognition model is established to include three parts: the backbone network Backbone, the neck fusion network Neck, and the detection head Head. The borescope image damage recognition model is used to perform borescope image damage recognition specifically including: First, the borescope image damage recognition model preprocesses the input images, resizing them to a uniform 640×640 resolution to make them suitable for model training and inference. Data augmentation is then performed, combining the enhanced images into a single enhanced image to enrich the dataset. Subsequently, the image is convolved layer by layer through Backbone, outputting feature maps containing rich semantic information for use by subsequent modules. After Backbone processing is completed, the feature maps enter the BSPP module, which captures multi-scale context and high-level directional semantic information in the feature maps through bidirectional alternating pooling operations, and inputs the extracted information into the C2PSA module for key feature screening. Next, the processed feature maps are input into the Neck module, which includes the FPN and PAN mechanisms. The FPN combines high-resolution low-level features with low-resolution high-level features through the FPN, and then outputs the two fused features to the DFM-PAN path. The DFM-PAN path embeds the hierarchical semantic-spatial decoupling attention mechanism HSDA to enhance and match features of different scales and ensure that information can flow efficiently between different layers. Finally, the NWD loss is introduced to improve the model's sensitivity to imbalanced samples and small target damage. Finally, the model integrates, transforms, and enhances the three feature maps of different sizes through the bottom-up DFM-PAN path, outputting them to the Head module to complete the target detection task.

3. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 1 is characterized in that: The bidirectional step-by-step pooling pyramid module BSPP described in step 1 designs a bidirectional step-by-step pooling operation mechanism named Max-Average Block, abbreviated as MA-Block. It consists of two parts, which guide modeling of information in different directions: Part 1: First, the input feature map X is max-pooled along the X-axis, that is, the maximum value is taken in each 1×k window. This operation compresses the information in the X-axis direction while keeping the Y-axis resolution unchanged. The size of the generated feature map is C×H×W / k. Then, the result of the X-axis max-pooling is averaged along the Y-axis, that is, the average value is taken in each k×1 window. The height k of the window is the same as the previous X-axis max-pooling. The size of the feature map generated in this step becomes C×H / k×W / k; Part 2: First, re-perform maximum pooling on the original input feature map X along the Y axis with a window size of k×1. This will compress the information in the Y axis direction while maintaining the X axis resolution, and the output feature map size is C×H / k×W. Then, the result of the previous step of maximum pooling on the Y axis is averaged along the X axis, that is, the average value is taken in each 1×k window, and the window size is also k. The output feature map size of this step becomes C×H / k×W / k; Finally, the results of the two parts are added element by element and the average is taken to ensure the stability of model training to obtain the final feature map. The size of the feature map obtained is C×H / k×W / k, which is consistent with the pooling output size in the original SPPF.

4. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 1, characterized in that: The specific calculation formula for the multi-directional alternating pooling operation is as follows: A: Input feature map: denoted as F, size is C×H×W, where C is the number of channels, H and W are the height and width of the feature map respectively; B: X-axis maximum pooling + Y-axis average pooling: Take the maximum value in each 1×k window of the input feature map in step A, set the stride to 1, and keep the output width unchanged by padding. Where F max_x Represents the feature map after the maximum pooling of the X axis, F max_x ∈R C×H×W ; Perform average pooling of Fmax_x along the Y axis, taking the average value in each k×1 window with a stride of 1, and padding to ensure that the output height remains unchanged; Where F avg_yx Represents the feature map after average pooling on the Y axis, F avg_yx ∈R C×H×W ; C: Y-axis maximum pooling + X-axis average pooling: Take the maximum value in each 1×k window of the input feature map in the original A step, set the stride to 1, and keep the output width unchanged by padding; Where, F max_y Represents the feature map after the maximum pooling of the X axis, F max_y ∈R C×H×W , perform average pooling of Fmax_y along the Y axis, take the average value in each k×1 window, with a stride of 1, and ensure the output height remains unchanged through appropriate padding; Where, F avg_yx Represents the feature map after the maximum pooling of the X axis, F avg_yx ∈R C×H×W ; D: Bidirectional pooling fusion: Add the output results of steps B and C element by element and take the average to obtain the final output feature map F out (c,h,w): F out (c,h,w)=[F avg_yx (c,h,w)+F avg_xy (c,h,w)] / 2 (29), The size of the above pooling kernel k will remain consistent with the original SPPF pooling kernel size, with values ​​of 5, 9, and 13. The adaptive pooling pyramid module still uses multi-scale pooling, performing multi-directional alternating pooling operations at each scale. At the same time, the stride value is 1, and the padding operation is used to ensure that the size of the output feature map after pooling is the same as the size of the input feature map.

5. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 1, characterized in that: In step 2, the dual-dimensional disentangled feature matching aggregation network (DFM-PAN) continues the bottom-up information transfer approach, but establishes an effective collaborative processing strategy between global and local information through the Hierarchical Semantic-Spatial Disentangled Attention (HSDA) mechanism. During the transfer process, each layer not only dynamically weights and filters the features from the previous layer in different dimensions, but also deeply explores directional information at different levels. The specific working method includes: First, the bottom-level feature B1 of the FPN path is fused with the mid- and high-level semantic features F3 to generate the N3 feature, which combines shallow spatial position information with deep semantic understanding capabilities. Subsequently, N3 is passed to the bottom layer of the DFM-PAN path and processed by the HSDA module. HSDA adaptively weights the input features in the channel dimension and spatial dimension, and extracts key directional features from different layers, thereby enhancing the expression capabilities of small targets and heterogeneous damage areas. After HSDA dynamically enhances the features, the resulting features are concatenated with N4 to further integrate the contextual information from the intermediate layer. Then, the fused features are input to the C3k2 module for feature extraction to obtain the D2 feature, which is used for the small and medium target detection layer. Finally, the D2 feature is passed to the HSDA module for further optimization to enhance its ability to adapt to high-level semantic expression. The optimized features are fused with B3 to generate a new feature map, which is input to the C3k2 module for feature extraction to obtain the D3 feature, which is used as the prediction layer for medium and large targets.

6. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 5, characterized in that: The proposed Hierarchical Semantic-Spatial Decoupled Attention (HSDA) mechanism consists of two stages. The first stage aims to perform comprehensive feature extraction on the input feature map, optimize the DFM-PAN to mine the directional and multi-scale information of the damage, and provide sufficient comprehensive and rich feature information for adaptive feature screening and enhancement. First, a convolution operation is performed on the input feature map. Through the action of the convolution kernel, the feature map can more clearly display the damage characteristics, and the subsequent sensitivity of the enhancement module to the morphological characteristics of damage at different scales and directions is enhanced. Subsequently, the input feature maps are average pooled and maximum pooled along the X-axis and Y-axis respectively, and are spliced ​​along the X-axis and Y-axis respectively to form feature maps of size C×H×2 and C×2×H; then, the different feature maps are spliced ​​together through the splicing operation to form a feature map of size C×(H+W)×2; then, the convolution operation is applied to the spliced ​​feature map to process features in two directions at the same time, and the balance between global information and local information is achieved through the local receptive field and hierarchical stacking characteristics of the convolution, so as to construct a compact and concentrated feature map; finally, a batch normalization (BN) layer is added after the convolution operation to keep the distribution of different images consistent during the feature extraction process.

7. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 6, characterized in that: The specific steps of the first stage of the hierarchical semantic-spatial decoupled attention mechanism (HSDA) are as follows: Step (1) Input feature map, and represent the input feature map as X∈R C×H×W , where C represents the number of channels, H represents the height, and W represents the width; Step (2): Global pooling in horizontal and vertical directions. Perform global average pooling in horizontal and vertical directions on the input feature map. The formula is as follows: In the formula, c represents the number of channels of the current feature map, h represents the height of the current feature map, and w represents the width of the current feature map. The global maximum pooling in the horizontal and vertical directions is calculated as follows: The average pooling and maximum pooling feature maps of the two are connected along the X direction and the Y direction respectively to form two new feature maps, as shown in the following formula: F concat,h =[X avg,h (c,w);X max,h (c,w)]∈R C×2×W (5), F concat,w =[X avg,w (c,h);X max,w (c,h)]∈R C×2×W (6); Step (3): Global pooling splicing operation. After completing the above splicing, another splicing operation is performed along the Y direction to prepare for global convolution. The calculation formula is as follows: F concat1 =[F concat,h ;F concat,w ]∈R C×(H+W)×2 (7), Where, F concat,h Represents the global pooling information in the X direction, F concat,w Represents the global pooling information in the Y direction; Step (4): Convolution operation, apply a 1×1 convolution operation to the newly formed feature map to generate an attention map: M′ F =Conv 1×1 (F concat1 ) (8), Where M′ F represents the generated attention map; F concat1 Represents the global pooling feature of the splicing; Step (5): Batch normalization and activation function processing to process the generated attention map: M F =φ(BN(M′ F )) (10), Where γ represents a learnable scale parameter; ε represents a constant, BN(·) represents the batch normalization layer operation, μ represents the mean of each channel in the attention map, and σ 2 represents the variance of each channel of the attention map, β represents the offset parameter, and φ(·) represents the activation function calculation.

8. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 7, characterized in that: The second stage of the hierarchical semantic-spatial decoupling attention mechanism (HSDA) includes: decoupling the feature map output by the first stage in the spatial direction, decomposing the feature map of size C×(H+W)×1 into C×H×1 and C×1×W, and performing convolution operations on them respectively to capture local detail features in a specific direction, and generating feature weights in different directions through activation functions; then, further solving the problem of spatial information loss caused by resolution difference and scale mismatch at the spatial level, performing average pooling and maximum pooling on the feature map output by the first stage along the channel dimension, and splicing them along the channel dimension, using convolution operations to further refine features to form The feature map of size 1×(H+W)×1 is used, and the weights are obtained through the activation function. The result is then split to form feature maps of size 1×H×1 and 1×1×W. Subsequently, the activation function is directly used to obtain the weights of the feature map of shape 1×(H+W)×1, which further enhances the model's sensitivity to directional features. Finally, the weighted feature maps generated at the semantic level and the spatial level are multiplied to obtain the final calculation, and the feature map with the weights of each feature is obtained as the final output of the hierarchical semantic-spatial decoupled attention mechanism (HSDA).

9. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 8, characterized in that: The specific steps of the second stage of the hierarchical semantic-spatial decoupled attention mechanism (HSDA) are as follows: Step a: Channel dimension pooling. In order to further extract global information, based on the attention map generated above, the pooling information extraction on the channel dimension is performed again. The calculation formula is as follows: Where, Represents the feature map information in the channel dimension; F avg Represents the average pooling information in the channel dimension; F max (i, j) represents the maximum pooling information in the channel dimension; then splicing is performed: F concat2 =[F avg ;F max ]∈R 1×(H+W)×2 (13); Step b: Weight acquisition in the channel dimension: Apply a 1×1 convolution operation to the newly formed feature map to generate attention features in the channel dimension: M C =Conv 1×1 (F concat2 ) (14),A C =σ(M C ) (15), Where A C Represents the weight information on the channel dimension; F concat2 Represents the pooling information in the channel dimension of the splicing; Step c: Segmentation operation: Two segmentation operations are required here. The first segmentation operation is to segment the pooled attention features in the channel dimension in the horizontal and vertical directions respectively. The second segmentation operation is to segment the attention map obtained in the first stage D step. The first segmentation operation is as follows: A h,C =A C [:,:H]∈R 1×H×1 (16),A w,C =A C [:,W:]∈R 1×1×W (17), Where A C Represents the weight information on the channel dimension, H represents height, and W represents width; the second split operation is as follows: M h,F =M F [:,:H]∈R C×H×1 (18),M w,F =M F [:,W:]∈R C×1×W (19), Where M F Represents the attention map generated by the first stage D step, M h,F Represents the feature map in the vertical direction of the cut; M w,F Represents the feature map in the horizontal direction of the cut; Step d: Obtaining weights in the horizontal and vertical directions: After obtaining the segmentation, horizontal and vertical feature maps, the weights in the horizontal and vertical directions are obtained respectively through the activation function. The specific calculation process is as follows: First, a convolution operation is required: M h =Conv 1×1 (M h,F )(20),M w =Conv 1×1 (M w,F )(21), Where M w Represents the feature map in the horizontal direction generated by convolution; M h Representatives passed The feature map in the vertical direction generated by convolution; Through the activation function, the weights in the horizontal and vertical directions are obtained: A h =σ(M h ) (22),A w =σ(M w ) (23), Where A w Represents the weight information obtained in the horizontal direction; A h Represents the obtained weight information in the vertical direction; Step e: Feature fusion: Multiply the obtained weight information in the horizontal, vertical and channel directions by the input feature map, and finally obtain a feature map with importance distinction: In the formula, Y represents the final generated feature map with importance distinction; X represents the input feature map; A C Represents the weight information on the channel dimension.

10. The borescope image damage recognition method based on directional alternating pooling and decoupled feature matching according to claim 1, characterized in that: In step 3, the specific design of introducing NWD loss is as follows: First, consider each target box as a two-dimensional Gaussian distribution: N=(μ,Σ) (30), Where μ represents the two-dimensional mean vector, which is used to describe the center position of the target box; Σ represents the covariance matrix corresponding to the target box; then, the 2-Wasserstein distance between the two-dimensional Gaussian distribution is used to measure the difference between the predicted box and the true box. Let the predicted box and the true box be N(μ1,Σ1) and N(μ2,Σ2) respectively. The square formula of the 2-Wasserstein distance is shown as follows: μ1 represents the center coordinate vector of the predicted box, μ2 represents the center coordinate vector of the real box; Σ1 represents the covariance matrix of the predicted box; Σ2 represents the covariance matrix of the real box; introduce the normalization process, and normalize the 2-Wasserstein distance to obtain NWD, as shown in the formula: C represents the normalization constant.

Images