Engine casting surface defect detection method based on deep learning

Abstract

The present application relates to a deep learning-based engine casting surface defect detection method, comprising: collecting a target engine casting surface image to be detected; inputting the image to be detected into an improved RT-DETR-R18 model for defect detection, and outputting the category probability and boundary box coordinates of all defects, wherein the improved RT-DETR-R18 model replaces the AIFI module in the original RT-DETR-R18 model with a spatial cognition enhanced coding structure, and at the same time reconstructs a cross-scale feature fusion framework with frequency domain perception and spatial dependence enhancement capability and introduces a sparse scale perception compression and enhancement fusion module. The present application can adapt to the detection requirements of sparse distribution and small scale defects under industrial manufacturing conditions and meet the current industrial product detection requirements.

Description

Technical Field

[0001] This invention relates to the field of industrial quality inspection technology, and in particular to a method for detecting surface defects in engine castings based on deep learning. Background Technology

[0002] In the industrial manufacturing sector, defect detection, as a crucial link in the quality assurance system, is of great significance for ensuring product reliability, reducing production risks, and improving overall production efficiency. With the continuous improvement of manufacturing processes and quality control levels, the occurrence of obvious or large-scale defects in workpieces is gradually decreasing. In actual production environments, defects often exhibit characteristics of small size and highly discrete spatial distribution. Simultaneously, the surfaces of workpieces in industrial settings typically have highly regular and repetitive texture structures, making defect areas highly similar to the background in local appearance, further exacerbating the difficulty of distinguishing defects from the background. Therefore, the problem of "small scale, sparse distribution, and weak contrast," jointly determined by improved manufacturing levels and data distribution characteristics, has become a representative technical challenge in the current field of industrial defect detection.

[0003] Early industrial defect detection relied primarily on manual inspection and traditional non-destructive testing (NDT) methods. Manual inspection typically involved visual examination, and the results were highly dependent on operator experience, easily affected by subjective factors and fatigue, making it difficult to meet the demands of modern large-scale, high-efficiency production. Traditional NDT methods, such as ultrasonic, X-ray, and microwave testing, while effective in detecting surface defects in specific scenarios, generally suffer from high dependence on hardware and testing conditions, complex procedures, and insufficient adaptability to online testing, limiting their widespread application in complex industrial environments. To address the NDT problem in industrial defects, researchers have proposed various detection methods based on physical signals such as microwaves and ultrasound, but all suffer from high dependence on hardware and testing conditions, poor adaptability to online testing, and susceptibility to industrial environmental influences.

[0004] With the rapid development of deep learning and computer vision technologies, deep learning-based industrial defect detection methods have gradually become a focal point of attention in both academia and industry. Neural network models, with their powerful feature learning and representation capabilities, have to some extent overcome the insufficient generalization ability of traditional non-destructive testing methods. Simultaneously, continuous optimization of model structure and inference processes has significantly improved inference efficiency while maintaining detection accuracy, providing a feasible technical path for real-time defect detection in industrial scenarios. Existing research has explored various aspects, including network structure design, multi-scale feature fusion, and model lightweighting, focusing on different industrial scenarios and specific datasets, achieving significant progress. However, from an overall research paradigm perspective, existing methods primarily focus on improving detection performance by enhancing feature fusion and attention modeling capabilities, with relatively limited attention paid to the sparse spatial distribution of defect targets. From the perspective of practical industrial applications, as manufacturing precision continues to improve, the frequency of large-scale, dense defects in the production process is gradually decreasing, and defect targets tend to exist in a small-scale, sparsely distributed form. Therefore, re-examining industrial defect detection methods from the perspective of sparse small-target modeling, whether from a theoretical research or engineering application standpoint, has significant research value. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based method for detecting surface defects in engine castings, in order to meet the detection requirements of sparsely distributed, small-scale defects under industrial manufacturing conditions and to satisfy current industrial product testing requirements.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A deep learning-based method for detecting surface defects in engine castings includes:

[0008] Images of the surface of the target engine casting are acquired for inspection.

[0009] The image to be detected is input into the improved RT-DETR-R18 model for defect detection, and the class probabilities and bounding box coordinates of all defects are output. The improved RT-DETR-R18 model is obtained by replacing the AIFI module in the original RT-DETR-R18 model with a spatial cognitive enhancement coding structure, and reconstructing a cross-scale feature fusion framework with frequency domain awareness and spatial dependency enhancement capabilities, and introducing a sparse scale awareness compression and enhancement fusion module.

[0010] Optionally, inputting the image to be detected into the improved RT-DETR-R18 model for defect detection includes:

[0011] The image to be detected is input into the backbone network to extract multi-scale features and obtain feature maps S2~S5 of different sizes;

[0012] The feature map S5 is input into the spatial cognitive enhancement coding structure for processing to obtain high-level semantic features;

[0013] The feature map S2, feature map S3, feature map S4 and the high-level semantic features are fused using a cross-scale feature fusion framework to output multi-scale aggregated features.

[0014] The multi-scale aggregated features are flattened into a 1D token sequence, and cross-attention iteratively updated with the learnable query vector of the decoder. The MLP prediction head outputs the class probabilities and bounding box coordinates of all defects.

[0015] Optionally, the feature map S5 is input into the spatial cognitive enhancement coding structure for processing to obtain high-level semantic features, including:

[0016] The feature map S5 is input into the position-aware attention module to calculate the attention weights and obtain the enhanced features after position-aware attention aggregation.

[0017] The enhanced features are then subjected to residual connection and adaptive normalization and input into the semantic reorganization and enhancement module for semantic reorganization and enhancement to obtain the reorganized features.

[0018] The recombined features are subjected to a discriminative nonlinear transformation to obtain the transformed features, and residual connection and adaptive normalization are performed to obtain the high-level semantic features.

[0019] Optionally, the fusion processing of feature map S2, feature map S3, feature map S4 and the high-level semantic features through a cross-scale feature fusion framework includes:

[0020] The feature map S3, feature map S4 and the high-level semantic features are fused by the high-frequency perception module and the spatial dependency perception module to obtain the first fused feature of feature map S4 and the high-level semantic features, and the second fused feature of feature map S3, feature map S4 and the high-level semantic features.

[0021] The feature map S2 of the P2 feature layer is input into a spatial depth convolution to perform spatial slicing, and feature fusion and mapping are completed through convolution to obtain compressed output features.

[0022] The compressed output features and the second fused features are fused using a spatial dependency perception module, and then processed using a CSP-based multi-core refinement module to obtain the third fused features.

[0023] The first fusion feature, the second fusion feature, and the third fusion feature are input into the multi-scale feature aggregation structure FAR for feature aggregation to obtain the multi-scale aggregated features.

[0024] Optionally, obtaining the first fusion feature and the second fusion feature includes:

[0025] The feature map S3, the feature map S4, and the high-level semantic features are input into the high-frequency perception module for processing, and the high-frequency response features are output. , , ;

[0026] High frequency response characteristics and The first fused feature is obtained by performing feature fusion through the spatial dependency perception module;

[0027] The high-frequency response characteristics The second fused feature is obtained by fusing the first fused feature with the spatial dependency perception module.

[0028] Optionally, the feature aggregation performed by the multi-scale feature aggregation structure FAR includes:

[0029] The fusion features at different scales are aggregated using an element-wise addition method. The aggregated features are then calibrated using 1×1 convolutions. Finally, the calibrated features are reconstructed using independent depthwise separable convolutions.

[0030] Optionally, the spatial dependency perception module performs feature fusion using the following method:

[0031] ;

[0032] ;

[0033] in, , and They represent the first The corresponding query, key, and value features in each spatial feature block. For intra-block attention weights, Indicates the first The output feature representation of a spatial feature block after introducing cross-scale spatial dependencies, with the superscript T indicating transpose.

[0034] The beneficial effects of this invention are as follows: Based on the RT-DETR-18 model, this invention introduces a spatial cognitive enhancement coding structure and reconstructs an improved scheme for the cross-scale feature fusion stage, which is validated on the NEU-DET steel surface defect public dataset. The improved model achieves an overall accuracy (P) of 80% and an average precision (mAP50%) of 77%, which are 5% and 4% higher than the baseline model RT-DETR-18, respectively, significantly enhancing the ability to identify sparsely distributed small defects. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a structural diagram of the SCE-Encoding Module according to an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the CCFF feature fusion stage in an embodiment of the present invention;

[0038] Figure 3 This is a structural diagram of the SCEF-SDP according to an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the improved RT-DETR-R18 model according to an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] This embodiment proposes a deep learning-based method for detecting surface defects in engine castings, including:

[0043] Images of the surface of the target engine casting are acquired for inspection.

[0044] The image to be detected is input into the improved RT-DETR-R18 model for defect detection, and the model outputs the class probability and bounding box coordinates of all defects, such as... Figure 4 As shown, the improved RT-DETR-R18 model is obtained by replacing the AIFI module in the original RT-DETR-R18 model with a spatial cognitive enhancement coding structure, while reconstructing a cross-scale feature fusion framework with frequency domain awareness and spatial dependency enhancement capabilities, and introducing a sparse scale awareness compression and enhancement fusion module.

[0045] Furthermore, the defect detection process involves inputting the image to be detected into the improved RT-DETR-R18 model, including:

[0046] The image to be detected is input into the backbone network to extract multi-scale features and obtain feature maps S2~S5 of different sizes;

[0047] The feature map S5 is input into the spatial cognitive enhancement coding structure for processing to obtain high-level semantic features;

[0048] The cross-scale feature fusion framework is used to fuse feature map S2, feature map S3, feature map S4 and high-level semantic features to output multi-scale aggregated features.

[0049] The multi-scale aggregated features are flattened into a 1D token sequence, which is then iteratively updated with cross-attention to the learnable query vector of the decoder. The MLP prediction head outputs the class probabilities and bounding box coordinates of all defects.

[0050] Specifically, the input data is a 640×640 image to be detected. After multi-scale feature extraction by the backbone network, feature maps S3~S5 of different sizes are obtained. Among them, S5 is a high-level semantic feature map with a downsampling ratio of 1 / 32 of the input image. Compared with other feature layers, it contains higher-level semantic information and global contextual information due to its higher downsampling ratio and larger receptive field. Then, it is input into the proposed Spatial Cognition Enhancement Coding Module (SCE-Encoding Module). The main function of this structure is to model the relationship between different spatial locations in the feature map, strengthen the contrast relationship between target-related features and background features, and thus improve the discriminative power of high-level semantic features at different spatial locations.

[0051] Furthermore, the feature map S5 is processed into the spatial cognitive enhancement coding structure to obtain high-level semantic features, including:

[0052] The feature map S5 is input into the position-aware attention module to calculate the attention weights and obtain the enhanced features after position-aware attention aggregation.

[0053] After performing residual connection and adaptive normalization on the enhanced features, they are input into the semantic reorganization and enhancement module for semantic reorganization and enhancement, and the reorganized features are obtained.

[0054] Discriminative nonlinear transformation is performed on the recombined features to obtain the transformed features, and residual connection and adaptive normalization are performed to obtain high-level semantic features.

[0055] Specifically, the SCE-Encoding Module structure diagram is as follows: Figure 1 As shown, the process of input data from entry to output is as follows:

[0056] Under a position-aware attention strategy, the input features first enter the information interaction stage. In this stage, when updating its own representation, each feature unit no longer indiscriminately aggregates contextual information from the global context, but rather selectively absorbs information from other locations based on content relevance and spatial dependency. Its position-awareness can be abstracted as formulas (1) and (2):

[0057] ;

[0058] ;

[0059] in Indicates the first The feature unit for the first Attention weights for each feature unit; and They represent the first With the Feature representation corresponding to each spatial location; and They represent the first With the Location information corresponding to each spatial location; Indicates the indices of all candidate feature units participating in attention normalization; and They represent the first Feature representations of each feature unit and their spatial location information; The first after location-aware attention aggregation An enhanced feature representation. Calculate content similarity. Calculate location similarity so that location information can be used as part of the attention weights, thus affecting the output token. It carries location information.

[0060] After completing the spatial perception information redistribution, the features must undergo a semantic-level reorganization and enhancement (Mona) to strengthen the discriminative representation of the target region. The process is shown in formula (3):

[0061] ;

[0062] in This represents the input features, which in turn represent the output token. , This represents a lightweight feature reorganization process. This indicates the characteristics after the reorganization is completed.

[0063] Based on this, further analysis of features Discriminative nonlinear transformation (ED-FFN) is performed to strengthen key dimensions of the model, amplify the weak target response, and prevent it from being smoothed or weakened in conventional feedforward mapping. The transformation can be expressed by formula (4):

[0064] ;

[0065] in Indicates input features, This represents the feature after transformation. and It is a learnable projection matrix. It is a non-linear activation function. This represents the discriminative feedforward transform, emphasizing the feature dimensions that are crucial for representing sparse objects.

[0066] To protect the original feature information and improve training stability, two residual connections (Add & Dropout) and adaptive normalization (DyT) are introduced throughout the process.

[0067] After this step, the high-level semantic feature output (denoted as S'5) has the same size and number of channels as S5, but with enhanced spatial discrimination ability. It is then passed to the cross-scale fusion stage (CCFF) for further processing.

[0068] Furthermore, the fusion of feature map S2, feature map S3, feature map S4, and high-level semantic features through a cross-scale feature fusion framework includes:

[0069] The feature map S3, feature map S4 and high-level semantic features are fused by the high-frequency perception module and the spatial dependency perception module to obtain the first fused feature of feature map S4 and high-level semantic features, and the second fused feature of feature map S3, feature map S4 and high-level semantic features.

[0070] The feature map S2 of the P2 feature layer is input into the spatial depth convolution to perform spatial slicing, and feature fusion and mapping are completed through convolution to obtain the compressed output features.

[0071] The compressed output features and the second fused features are fused by the spatial dependency perception module, and then processed by the CSP-based multi-core refinement module to obtain the third fused features.

[0072] The first, second, and third fusion features are input into the multi-scale feature aggregation structure FAR for feature aggregation to obtain multi-scale aggregated features.

[0073] Furthermore, obtaining the first fusion feature and the second fusion feature includes:

[0074] Feature map S3, feature map S4, and high-level semantic features are input into the high-frequency perception module for processing, and the high-frequency response features are output. , , ;

[0075] High frequency response characteristics and Feature fusion is performed using a spatial dependency awareness module to obtain the first fused feature;

[0076] High frequency response characteristics The second fused feature is obtained by fusing the first fused feature with the spatially dependent perception module.

[0077] Specifically, in the cross-scale feature fusion (CCFF) stage, the S3, S4, and S'5 feature layers are first processed by the high-frequency sensing module (HFP) to extract the high-frequency components in their frequency domain representation and map them back to the spatial domain, outputting a more discriminative high-frequency feature representation. The high-frequency feature generation can be represented by formula (5):

[0078] ;

[0079] in and These represent the Discrete Cosine Transform (DCT) and its inverse transform, respectively. This is a high-pass filter operator used to suppress low-frequency background information. The process takes the input features... High-frequency details closely related to defect identification are explicitly extracted and mapped back to the spatial domain to obtain high-frequency response features. (The spatial domain high-frequency response characteristics corresponding to S3, S4, and S'5 are respectively represented by...) , , express).

[0080] Then, a spatial dependency-aware module (SDP) is used to perform cross-scale feature fusion on features at various scales, mitigating the spatial misalignment problem caused by scale transformation during multi-scale feature fusion. Specifically, and The output obtained by feature fusion through SDP is input to FAR on one hand, and on the other hand... Feature fusion (denoted as S') is performed via SDP and then input into FAR. The core formula of SDP can be expressed by formulas (6) and (7):

[0081] ;

[0082] ;

[0083] in, , and They represent the first The corresponding query, key, and value features in each spatial feature block. For intra-block attention weights, Indicates the first The output feature representation obtained by incorporating cross-scale spatial dependencies in each spatial feature block. By modeling spatial dependencies at the feature block level, SDP can explicitly capture the spatial correspondences between cross-scale features, introducing richer contextual information for small targets. This stage of the operation is as follows: Figure 2 As shown.

[0084] To fully utilize the spatial detail information in the low-level high-resolution features, this algorithm introduces a P2 layer in CCFF and designs a sparse scale-aware compression and enhancement fusion module (SCEF-SDP), the structure of which is as follows: Figure 3 As shown.

[0085] Specifically, using the feature map S2 of the P2 feature layer as input, spatial depthwise convolution (SPDconv) is used to perform spatial slicing with a stride of 2, dividing the original feature map into four non-overlapping sub-features and concatenating them along the channel dimension. Then, a standard 3×3 convolution is used to complete feature fusion and mapping, resulting in the compressed output features. SPDconv can be abstracted as formula (8):

[0086] ;

[0087] in This represents the output of SPDconv. This represents non-overlapping spatial blocks. This represents a 3×3 convolution. This indicates channel splicing.

[0088] Then this feature is processed via SDP. The features of S' are fused, and a multi-kernel refinement module (CSP-MKR) based on CSP is used to refine the model, minimizing the computational load. The model is then fed into FAR, giving FAR a total of 3 inputs.

[0089] Furthermore, the multi-scale feature aggregation structure FAR performs feature aggregation including:

[0090] The fusion features at different scales are aggregated using an element-wise addition method. The aggregated features are then calibrated using 1×1 convolutions. Finally, the calibrated features are reconstructed using independent depthwise separable convolutions.

[0091] FAR (Focused Arithmetic Array) is a concise and efficient multi-scale feature aggregation structure. Its function is to further integrate and calibrate fused features, thereby improving the stability and consistency of feature representation. Specifically, FAR first performs residual aggregation on fused features at different scales (i.e., the three inputs mentioned above) using an element-wise addition method. While maintaining a fixed channel dimension of 256, it then performs channel calibration on the aggregated features using 1×1 convolutions. Finally, for the calibrated features, three sets of independent depthwise separable convolutions are used for feature reconstruction, decoupling spatial feature extraction from channel feature aggregation and effectively controlling parameter scale while maintaining expressive power.

[0092] The final FAR output is first flattened into a 1D token sequence. Then, a "minimum uncertainty query selection" process is used to filter out high-quality, low-uncertainty initial query vectors. These queries are then fed into the decoder, where features at each scale are iteratively updated through cross-attention, continuously optimizing to obtain higher-quality queries. Finally, the MLP prediction head directly outputs the class probabilities and bounding box coordinates of all defects from the final queries, thus completing the detection.

[0093] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for detecting surface defects in engine castings based on deep learning, characterized in that, include: Images of the surface of the target engine casting are acquired for inspection. The image to be detected is input into the improved RT-DETR-R18 model for defect detection, and the class probabilities and bounding box coordinates of all defects are output. The improved RT-DETR-R18 model is obtained by replacing the AIFI module in the original RT-DETR-R18 model with a spatial cognitive enhancement coding structure, and reconstructing a cross-scale feature fusion framework with frequency domain awareness and spatial dependency enhancement capabilities, and introducing a sparse scale awareness compression and enhancement fusion module.

2. The method for detecting surface defects in engine castings based on deep learning according to claim 1, characterized in that, The defect detection process involves inputting the image to be detected into the improved RT-DETR-R18 model, which includes: The image to be detected is input into the backbone network to extract multi-scale features and obtain feature maps S2~S5 of different sizes; The feature map S5 is input into the spatial cognitive enhancement coding structure for processing to obtain high-level semantic features; The feature map S2, feature map S3, feature map S4 and the high-level semantic features are fused using a cross-scale feature fusion framework to output multi-scale aggregated features. The multi-scale aggregated features are flattened into a 1D token sequence, and cross-attention iteratively updated with the learnable query vector of the decoder. The MLP prediction head outputs the class probabilities and bounding box coordinates of all defects.

3. The method for detecting surface defects in engine castings based on deep learning according to claim 2, characterized in that, The feature map S5 is input into the spatial cognitive enhancement coding structure for processing to obtain high-level semantic features, including: The feature map S5 is input into the position-aware attention module to calculate the attention weights and obtain the enhanced features after position-aware attention aggregation. The enhanced features are then subjected to residual connection and adaptive normalization and input into the semantic reorganization and enhancement module for semantic reorganization and enhancement to obtain the reorganized features. The recombined features are subjected to a discriminative nonlinear transformation to obtain the transformed features, and residual connection and adaptive normalization are performed to obtain the high-level semantic features.

4. The method for detecting surface defects in engine castings based on deep learning according to claim 2, characterized in that, The fusion process of feature map S2, feature map S3, feature map S4, and the high-level semantic features through a cross-scale feature fusion framework includes: The feature map S3, feature map S4 and the high-level semantic features are fused by the high-frequency perception module and the spatial dependency perception module to obtain the first fused feature of feature map S4 and the high-level semantic features, and the second fused feature of feature map S3, feature map S4 and the high-level semantic features. The feature map S2 of the P2 feature layer is input into a spatial depth convolution to perform spatial slicing, and feature fusion and mapping are completed through convolution to obtain compressed output features. The compressed output features and the second fused features are fused using a spatial dependency perception module, and then processed using a CSP-based multi-core refinement module to obtain the third fused features. The first fusion feature, the second fusion feature, and the third fusion feature are input into the multi-scale feature aggregation structure FAR for feature aggregation to obtain the multi-scale aggregated features.

5. The method for detecting surface defects in engine castings based on deep learning according to claim 4, characterized in that, Obtaining the first fusion feature and the second fusion feature includes: The feature map S3, the feature map S4, and the high-level semantic features are input into the high-frequency perception module for processing, and the high-frequency response features are output. , , ; High frequency response characteristics and The first fused feature is obtained by performing feature fusion through the spatial dependency perception module; The high-frequency response characteristics The second fused feature is obtained by fusing the first fused feature with the spatial dependency perception module.

6. The method for detecting surface defects in engine castings based on deep learning according to claim 4, characterized in that, The multi-scale feature aggregation structure FAR performs feature aggregation as follows: The fusion features at different scales are aggregated using an element-wise addition method. The aggregated features are then calibrated using 1×1 convolutions. Finally, the calibrated features are reconstructed using independent depthwise separable convolutions.

7. The method for detecting surface defects in engine castings based on deep learning according to claim 5, characterized in that, The method for feature fusion in the spatial dependency perception module is as follows: ； ； in, , and They represent the first The corresponding query, key, and value features in each spatial feature block. For intra-block attention weights, Indicates the first The output feature representation of a spatial feature block after introducing cross-scale spatial dependencies, with the superscript T indicating transpose.

Images