Transformer-based multi-frequency boundary perception industrial CT image defect detection system, method and device

Abstract

The application provides a multi-frequency boundary perception industrial CT image defect detection system, method and device based on a Transformer, and belongs to the field of image processing; a segmentation model of a three-stage progressive feature modeling architecture is provided, a first stage of the three-stage architecture adopts a light convolution shallow encoder module for shallow local focusing, high resolution is maintained at the shallow layer, and low-level features such as edges and textures are densely extracted; a frequency decomposition double-branch Transformer module is adopted at the second stage for global-local collaborative modeling, and high-frequency details and low-frequency structures are separated and processed; a boundary perception refining module is adopted at the third stage to perform boundary enhancement and edge smoothing on a rough segmentation result; under the premise of maintaining linear computational complexity, the application realizes organic fusion of local details and global context, enhances the perception ability of a fuzzy boundary of a defect in a low-contrast CT image, and maintains good generalization performance under a small sample condition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a multi-frequency boundary sensing industrial CT image defect detection system, method and apparatus based on Transformer. Background Technology

[0002] 1. Existing industrial CT image defect segmentation methods and their limitations:

[0003] Industrial computed tomography (CT) technology is an important tool for non-destructive testing of internal defects in key components in high-end manufacturing, aerospace, and precision machinery. Its goal is to accurately extract defect regions (such as porosity, cracks, inclusions, and looseness) from CT images, providing quantitative evidence for quality assessment and process improvement. With the development of deep learning technology, industrial CT defect segmentation methods have evolved from traditional image processing techniques to deep learning methods based on convolutional neural networks (CNNs), and in recent years, to those based on the Transformer architecture.

[0004] 2. Defect segmentation methods based on CNN and their limitations:

[0005] Encoder-decoder architectures, exemplified by U-Net, have become a common method for defect segmentation in industrial CT due to their excellent performance. These methods extract hierarchical features through stacked convolutional layers, demonstrating advantages in local texture modeling and edge detail preservation. However, CNNs are limited by their finite receptive field, causing convolutional operations to aggregate features only within local neighborhoods, making it difficult for the model to effectively capture long-range dependencies in images. Although the receptive field can theoretically be expanded by stacking deep networks, in practical modeling, establishing contextual relationships between distant pixels remains difficult.

[0006] This limitation is particularly pronounced for industrial CT defect detection tasks. Many defects (such as microcracks and low-contrast inclusions) appear in CT images as having blurred boundaries and minimal grayscale differences from the surrounding matrix material. Accurate defect segmentation relies not only on local edge information but also on integrating global contextual information, such as the spatial relationship between the defect and its surrounding structure (e.g., material density distribution, part geometry). CNN models that rely solely on local features are prone to oversegmentation (misclassifying noise or artifacts as defects) or undersegmentation (missing true defect areas) when processing weak-contrast defects.

[0007] Furthermore, while the inherent inductive bias of CNNs—that is, the locality and translational variability processing methods—is beneficial for model convergence under limited data conditions, it also limits its ability to model the global structure of images. When defects have irregular shapes and significant size differences (from micron-level fine cracks to millimeter-level large pores), CNNs struggle to represent multi-scale features within the same framework.

[0008] 3. The classic Transformer segmentation method and its limitations:

[0009] To address the context-constrained nature of CNNs, researchers have introduced the Transformer architecture from natural language processing into computer vision tasks. Vision Transformer (ViT) divides images into sequential image patches and utilizes a self-attention mechanism to model global dependencies between these patches, theoretically capable of capturing pixel-to-pixel relationships at arbitrary distances. However, directly applying the original ViT to 3D industrial CT image segmentation reveals the following key issues:

[0010] The computational complexity is too high: the computational complexity of the self-attention mechanism is proportional to the square of the length of the input sequence. For high-resolution 3D industrial CT volumetric data, the sequence length generated after dividing it into image blocks is extremely large, causing the memory consumption and computational overhead required for model training and inference to far exceed the carrying capacity of conventional industrial inspection hardware.

[0011] The lack of local inductive bias: Unlike CNNs, Transformers lack prior assumptions about the local structure of images. This characteristic makes pure Transformer models prone to overfitting when there are limited samples of industrial CT defect annotations (defect annotation requires experienced inspection engineers to invest a lot of time and effort, which is costly). Their feature learning ability for small sample data is actually inferior to that of CNNs.

[0012] Insufficient sensitivity to detail texture: Accurate segmentation of defects in industrial CT relies heavily on capturing detailed information such as defect edges and minute crack textures. The global attention mechanism of the pure Transformer tends to focus on the overall structure, which may dilute the response to local details, resulting in positioning deviations at fine boundaries in the segmentation results.

[0013] 4. Addressing the specific challenges of industrial CT defect detection:

[0014] In addition to the limitations of the general methods mentioned above, accurate 3D segmentation of defects in industrial CT scans also faces the following specific difficulties:

[0015] Low contrast and blurred boundaries: Many defects (such as microcracks and inclusions of similar density) show very little difference in grayscale from the matrix material in CT images, making it difficult to clearly define defect boundaries. Existing segmentation methods often exhibit segmentation bias when distinguishing between true defect boundaries and material noise or artifacts.

[0016] Heterogeneity in Defect Morphology and Size: Industrial defects exhibit highly irregular morphologies, ranging from micropores to long cracks, with volume spanning several orders of magnitude. This places stringent demands on the multi-scale feature representation capabilities of segmentation models. Existing single-scale segmentation models struggle to simultaneously achieve high detection rates for micro-defects and complete boundary delineation for large-volume defects within the same framework.

[0017] The scarcity of high-quality labeled data: Industrial CT defect annotation requires deep expertise in materials science and non-destructive testing, and the definition of defect boundaries is subject to subjective differences. This makes it difficult to acquire high-quality 3D labeled data, limiting the performance of data-driven deep learning methods.

[0018] To overcome the aforementioned technical bottlenecks, researchers have attempted to fuse CNNs and Transformers, aiming to simultaneously leverage the local modeling capabilities of CNNs and the global context capture capabilities of Transformers. Typical hybrid architectures (such as TransUNet) insert Transformer modules into the encoder portion of a U-Net, or employ a parallel dual-branch structure to extract local and global features separately. However, the following difficulties were encountered during implementation:

[0019] The design challenges of feature fusion mechanisms: Designing an efficient cross-modal feature fusion mechanism is a major challenge. Simple feature concatenation or element-wise addition often leads to feature conflicts, where local details may be overwhelmed by global noise, or global structural information may be fragmented by local details. Especially in complex industrial parts, interference from material density fluctuations and scattering artifacts makes it difficult for models to distinguish between real defect boundaries and irrelevant noise.

[0020] The challenge of balancing computational efficiency and accuracy: While some studies have proposed sparse attention or window-based attention (such as SwingTransformer) to address the high computational cost of Transformers, designing an attention mechanism that maintains 3D spatial continuity while significantly reducing computation remains a hot research topic in 3D industrial CT. Simple 3D window expansion can lead to the loss of window boundary information, difficulties in cross-window interaction, and discontinuities or step-like artifacts in the Z-axis direction of segmented defects.

[0021] Overfitting risk under small sample conditions: Mixed models often have a larger number of parameters. In reality, given the limited amount of industrial CT defect annotation data, these models are prone to overfitting, leading to a decline in generalization ability. How to mitigate this problem by combining strategies such as transfer learning and self-supervised pre-training still requires further exploration. Summary of the Invention

[0022] To address the following issues in existing technologies for processing industrial CT images: 1. the trade-off between computational complexity and segmentation accuracy; 2. the fusion of local detail features and global contextual features; 3. the problem of accurate segmentation of defect boundaries in low-contrast CT images; 4. the model generalization problem under small sample conditions; and 5. the adaptation problem to the heterogeneity of defect morphology and size, this application proposes a Transformer-based multi-frequency boundary sensing industrial CT image defect detection system, method, and apparatus.

[0023] The technical solution adopted in this application is as follows: a multi-frequency boundary-aware industrial CT image defect detection system based on Transformer. The system includes a 3D industrial CT image segmentation model, which includes a lightweight convolutional shallow encoder module, a frequency decomposition dual-branch Transformer module, a multi-scale feature aggregation gating mechanism module, a decoder module, and a boundary-aware refinement module. The lightweight convolutional shallow encoder module is used to extract multi-scale features from the original 3D industrial CT data and input the deep features output at the end into the frequency decomposition dual-branch Transformer module. The frequency decomposition dual-branch Transformer module consists of multiple sequentially connected frequency decomposition dual-branch blocks. Each frequency decomposition dual-branch block includes a frequency decomposition unit, a high-frequency branch, a low-frequency branch, and a frequency fusion unit. The multi-scale feature aggregation gating mechanism module is located between the encoder and the decoder and is used to dynamically select encoder features that match the decoder layer. The boundary-aware refinement module is used to refine the boundary of the coarse segmentation result output by the decoder module.

[0024] Furthermore, the lightweight convolutional shallow encoder module consists of three sequentially connected lightweight convolutional blocks. Each lightweight convolutional block contains a depth-separable 3D convolutional layer, an instance normalization layer, a GELU activation function, and an inverted residual structure that are connected in sequence. The convolutional kernel size is 3×3×3 and the stride is 2.

[0025] Furthermore, the frequency decomposition dual-branch Transformer module consists of four sequentially connected frequency decomposition dual-branch blocks. The frequency decomposition unit uses 3D discrete cosine transform to decompose the input feature map into high-frequency and low-frequency components. The high-frequency branch contains a 3D windowed multi-head self-attention layer with a fixed window size of 4×4×4, used to extract high-frequency local detail features. The low-frequency branch contains an improved 3D Mamba state space model, used to achieve global context modeling with linear complexity. The frequency fusion unit fuses the output features of the high-frequency and low-frequency branches through a learnable weighted fusion strategy.

[0026] Furthermore, the multi-scale feature aggregation gating mechanism module generates gating weights for encoder features at each scale based on the semantic features of the current layer of the decoder; upsamples encoder features at different scales to the same size as the decoder layer; performs weighted summation of multi-scale features according to the gating weights to achieve dynamic feature selection; and concatenates the weighted fused features with the decoder features and inputs them into subsequent layers.

[0027] Furthermore, the boundary-aware refinement module includes a boundary extraction unit, a boundary consistency constraint unit, and a 3D trainable guided filter layer. The boundary extraction unit extracts the boundary mask from the coarse segmentation result through differentiable morphological gradient operations. The boundary consistency constraint unit is used to calculate the boundary loss function in the region covered by the boundary mask, forcing the model to focus on boundary pixels. The 3D trainable guided filter layer uses the original industrial CT image as the guide map to perform edge-preserving smoothing processing on the coarse segmentation result. The linear coefficients of the guided filter are obtained through end-to-end learning of the neural network.

[0028] Furthermore, the system also includes a self-supervised pre-training module for pre-training model parameters on large-scale unlabeled 3D industrial CT images.

[0029] Furthermore, the self-supervised pre-training module includes a volume data fusion task and a mask autoencoder task. The volume data fusion task selects two 3D image patches from different regions and fuses them at a random ratio, requiring the model to predict the fusion coefficient of each voxel. The mask autoencoder task randomly masks some voxels, requiring the model to reconstruct the masked region.

[0030] A Transformer-based multi-frequency boundary-aware industrial CT image defect detection method, implemented using the aforementioned Transformer-based multi-frequency boundary-aware industrial CT image defect detection system, includes the following steps:

[0031] Step 1: Obtain large-scale unlabeled and a small amount of labeled 3D industrial CT data as a dataset for self-supervised pre-training;

[0032] Step 2: Construct a 3D industrial CT image segmentation model;

[0033] Step 3: Train the 3D industrial CT image segmentation model to obtain the trained model;

[0034] Step 4: Use the trained model to process the 3D industrial CT data and output the 3D segmentation results of the defects.

[0035] Furthermore, step 3 specifically includes:

[0036] During the pre-training phase, the encoder part of the lightweight convolutional shallow encoder module and the frequency decomposition dual-branch Transformer module are jointly trained using two tasks in the self-supervised pre-training module: the volume data fusion task and the masked autoencoder task, to obtain the pre-trained weights.

[0037] The volume data fusion task involves randomly cropping two 3D blocks from different locations in the same image, generating a random fusion coefficient map, and then fusing them proportionally. The model is required to predict the fusion coefficient of each voxel. Simultaneously, a mask autoencoder task is performed, which randomly masks a portion of the voxel region and requires the model to reconstruct the original signal of the masked region.

[0038] Then, pre-trained weights are loaded to initialize the 3D industrial CT image segmentation model. Subsequently, the entire model is trained end-to-end using a small amount of labeled 3D industrial CT data.

[0039] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0040] The advantages of this application over existing technologies are as follows: The DefectFormer model proposed in this application, through its unique "three-stage progressive feature modeling" architecture and the collaborative design of five innovative modules, has the following advantages compared to existing technologies:

[0041] 1. A dual improvement in computational efficiency and segmentation accuracy:

[0042] Existing Transformer-based 3D segmentation models (such as ViT and Swin Transformer) face a trade-off between computational complexity and segmentation accuracy. The quadratic complexity of global self-attention leads to excessive memory consumption, while sparse attention or window attention, although reducing computation, may sacrifice the continuity of 3D space, resulting in segmentation results with inter-layer discontinuities or step-like artifacts.

[0043] This application reduces the computational complexity from O(N) by frequency decomposing the low-frequency Mamba branch in the dual-branch Transformer module and replacing the traditional self-attention model with a state-space model.2 The computational complexity is reduced to O(N), significantly decreasing the GPU memory consumption for model training and inference. Simultaneously, the window attention of the high-frequency branch works in conjunction with the Mamba modeling of the low-frequency branch, ensuring the integrity of the 3D spatial structure through frequency fusion units, thus maintaining segmentation accuracy while reducing computational burden.

[0044] The frequency decomposition strategy separates high-frequency details from low-frequency structures, enabling the model to adopt the most suitable modeling method for different frequency components. This avoids redundant computation of global attention and ensures semantic coherence in three-dimensional space, achieving synergistic optimization of computational efficiency and segmentation accuracy.

[0045] 2. The organic integration of local details and global context:

[0046] Existing hybrid architectures (such as TransUNet and CoTr) employ simple feature concatenation or element-wise addition to fuse CNN and Transformer features, which may result in local details being overwhelmed by global noise or the global structure being fragmented by local details. Especially in complex industrial parts, interference such as material density fluctuations and scattering artifacts make it difficult for the fused features to distinguish between real defect boundaries and irrelevant noise.

[0047] This application separates features into high-frequency (edges, textures) and low-frequency (structure, contours) features in the frequency domain using a frequency decomposition dual-branch Transformer module. Window attention and Mamba modeling are employed respectively, followed by learnable weighted fusion. This "separate first, then fuse" strategy avoids direct conflicts between features of different properties, allowing high-frequency details and low-frequency structures to coexist harmoniously. Simultaneously, the multi-scale feature aggregation gating module dynamically calculates the fusion weights of encoder features at each scale based on the semantic information of the current layer of the decoder, rather than fixedly concatenating them, effectively suppressing the influence of artifact regions.

[0048] Frequency domain separation eliminates the root cause of feature conflicts at the source, while dynamic gating enables adaptive feature selection—when the decoder needs detailed information, the gating weights favor the high-frequency branch; when structural information is needed, the weights favor the low-frequency branch. This adaptive mechanism allows the model to intelligently adjust the fusion strategy according to the image content, obtaining clearer and more semantically complete defect segmentation results.

[0049] 3. Ability to accurately segment defect boundaries in low-contrast CT images:

[0050] In industrial CT images, many defects (such as microcracks and low-density inclusions) have very little grayscale difference from the matrix material, resulting in blurred boundaries. The global attention mechanism of pure Transformer tends to focus on the overall structure, which may dilute the response to local boundaries; pure CNN, due to its limited receptive field, has difficulty integrating the spatial relationship between defects and the surrounding material structure (such as density distribution and geometric contours), leading to boundary localization errors.

[0051] This application explicitly extracts defect boundary regions through differentiable morphological gradient operations in the boundary-aware refinement module and applies additional boundary loss constraints to force the model to focus on boundary pixels. Simultaneously, a 3D trainable guided filter layer, guided by the original industrial CT image, performs edge-preserving smoothing on the coarse segmentation results. The filter parameters are learned end-to-end through the network, enabling them to adapt to the morphological characteristics of different defects.

[0052] Boundary extractors enable the model to "know" which pixels require special attention, while guided filtering ensures the geometric smoothness and structural rationality of the segmentation results. The combination of the two simulates the judgment process of inspection engineers: first identifying suspicious areas, and then finely delineating the boundaries, thereby obtaining defect segmentation results with clearer boundaries and more accurate positioning.

[0053] 4. Strong generalization ability under small sample conditions:

[0054] Industrial CT defect annotation requires deep expertise in materials science and non-destructive testing, and obtaining high-quality 3D annotation data is difficult. Existing hybrid models have a large number of parameters, are prone to overfitting with limited annotation data, and exhibit significantly reduced generalization ability.

[0055] This application utilizes a self-supervised pre-training module, leveraging 3D volumetric data fusion and mask reconstruction tasks, to learn general structural representations on large-scale unlabeled industrial CT images. The volumetric data fusion task forces the model to learn the relative positional relationships and content consistency between image patches, while the mask reconstruction task helps the model understand the complete 3D structural morphology. After pre-training, the model can achieve good performance through fine-tuning on a small amount of labeled data.

[0056] These two pre-training tasks together endow the model with the ability to understand the deep structure of industrial CT images, enabling it to maintain robust feature extraction even with limited labeled samples. This effectively mitigates the risk of overfitting under small sample conditions and improves the model's generalization ability.

[0057] 5. Robustness to adapting to heterogeneity in defect morphology and size:

[0058] Industrial defects are extremely irregular in shape, ranging from micron-level fine cracks to millimeter-level large pores, with a large volume range, which places stringent demands on the multi-scale feature representation capabilities of segmentation models. Single-scale segmentation models struggle to simultaneously achieve high detection rates for minute defects and complete boundary delineation for large-volume defects.

[0059] This application employs a lightweight convolutional shallow encoder to maintain high-resolution feature maps at shallow layers, avoiding detail loss due to premature downsampling; a frequency decomposition dual-branch Transformer performs global modeling at deeper layers, forming a multi-scale feature pyramid from fine to coarse. The trainable guided filter in the boundary-aware refinement module, whose filter parameters are learned through data-driven learning, can adaptively adjust the processing strategy according to the actual shape of the defect.

[0060] Shallow high-resolution features preserve the details of tiny defects, deep global features capture the overall structure of large-volume defects, and adaptive filtering bridges the processing differences between different scales, enabling the model to adapt to defects of different sizes and shapes and achieve robust segmentation performance.

[0061] 6. Practical value in industrial applications:

[0062] Existing models are either too computationally demanding to be deployed on conventional industrial testing hardware, or lack sufficient accuracy to meet quality inspection standards, thus limiting their practical industrial applications.

[0063] This application employs a lightweight design to reduce the model's computational resource requirements, making it easier to deploy on conventional industrial inspection hardware. The boundary mask output by the boundary-aware refinement module visually displays the areas of interest for the model, providing inspection engineers with decision-making support and enhancing industrial acceptance. By improving segmentation accuracy and efficiency, this invention is expected to assist inspection engineers in shortening defect identification time, reducing the false negative and false positive rates of manual interpretation, and improving inspection consistency.

[0064] The design of this application balances algorithm performance and industrial practicality, pursuing both technological advancement and the feasibility of practical applications, and has good prospects for industrial application. Attached Figure Description

[0065] The following description, in conjunction with the accompanying drawings, further illustrates this application:

[0066] Figure 1 This is a diagram illustrating the overall architecture of the segmentation model provided in the embodiments of this application.

[0067] Figure 2 A structural diagram of a lightweight convolutional shallow encoder module provided in an embodiment of this application;

[0068] Figure 3 This is a structural diagram of the frequency decomposition dual-branch Transformer module provided in an embodiment of this application;

[0069] Figure 4 This is a structural diagram of the multi-scale feature aggregation gating module provided in an embodiment of this application;

[0070] Figure 5 This is a structural diagram of the boundary awareness refinement module provided in an embodiment of this application. Detailed Implementation

[0071] like Figures 1 to 5 As shown, this application provides a Transformer-based multi-frequency boundary-aware industrial CT image defect detection system, and proposes a 3D industrial CT image segmentation model called DefectFormer (Defect Transformer with Multi-Frequency Boundary Awareness). This model adopts a "three-stage progressive feature modeling" architecture and achieves high-precision 3D segmentation of industrial CT images through the collaborative work of five innovative modules.

[0072] like Figure 1 As shown, the overall architecture of the DefectFormer model proposed in this application includes: a Lightweight Convolutional Shallow Encoder (LCSE) module, a Frequency Decomposition Dual-Branch Transformer (FDDB-T) module, a Multi-Scale Feature Aggregation Gate (MSFAG) module, a decoder module, a Boundary-Aware Refinement Module (BARM) module, and a self-supervised pre-training module. The model takes raw 3D industrial CT data as input, processes it through the above modules sequentially, and outputs accurate 3D segmentation results of industrial defects.

[0073] The first stage of the "three-stage progressive feature modeling" architecture uses a lightweight convolutional shallow encoder module to perform shallow local focusing, maintaining high resolution in the shallow layer and densely extracting low-level features such as edges and textures; the second stage uses a frequency decomposition dual-branch Transformer module to perform global-local collaborative modeling, separating and processing high-frequency details and low-frequency structures; the third stage uses a boundary-aware refinement module to enhance the boundaries of the coarse segmentation results and keep the edges smooth.

[0074] First, the input image is fed into a lightweight convolutional shallow encoder module, which consists of three consecutive 3D depthwise separable convolutions to extract shallow local texture and edge features, while reducing the number of model parameters and computational complexity. After encoding, the network generates multi-scale feature maps.

[0075] Subsequently, the multi-scale feature maps are input into a frequency decomposition dual-branch Transformer module, which consists of a high-frequency branch and a low-frequency branch. The high-frequency branch uses Window Attention to model local texture and edge details, while the low-frequency branch uses the Mamba Module to capture global semantic information and long-range dependencies. The features extracted from the two branches are fused through a frequency fusion unit to form a global-local fused feature that combines local details with global contextual information.

[0076] Subsequently, the fused features are fed into the multi-scale feature aggregation gating mechanism module. This module dynamically weights and fuses features at different scales through dynamic weighted skip connections, thereby enhancing multi-scale semantic expression capabilities and reducing redundant information interference.

[0077] The fused features are then input into the decoding network. The decoder employs three consecutive 3D Transposed Convolution Upsampling operations to progressively restore the spatial resolution of the feature map, and combines multi-scale features from skip connections to reconstruct the target region structure, ultimately generating a preliminary coarse segmentation map.

[0078] Finally, the coarse segmentation result is sent to the boundary-aware refinement module to refine and optimize the boundary region, thereby improving the boundary continuity and detail recovery capability, and finally generating the final 3D segmentation output.

[0079] like Figure 2 As shown, the LCSE module consists of three sequentially connected lightweight convolutional blocks. Each block contains: a depthwise separable 3D convolution layer with a kernel size of 3×3×3 and a stride of 2, used for downsampling and feature extraction; an inverted residual structure, which first expands the number of channels and then compresses them to enhance feature representation; and an instance normalization layer and the GELU activation function.

[0080] The input is a raw 3D industrial CT image with dimensions H×W×D (height×width×depth) and 1 channel. The first convolutional block outputs a size of H / 2×W / 2×D / 2 with 32 channels (C1). The second convolutional block outputs a size of H / 4×W / 4×D / 4 with 64 channels (C2). The third convolutional block outputs a size of H / 8×W / 8×D / 8 with 128 channels (C3). The final output is a multi-scale feature map {F1, F2, F3}, corresponding to feature representations at three different scales. F3 is passed as input to the frequency decomposition dual-branch Transformer module, while F1 and F2 are reserved for subsequent skip connections.

[0081] The LCSE module, through a shallow CNN design, compensates for the shortcomings of the Transformer in local feature extraction, and enhances the model's sensitivity to details such as defect edges and textures (corresponding to technical issues 2 and 3).

[0082] like Figure 3 As shown, the FDDB-T module consists of four sequentially connected frequency decomposition dual-branch blocks. Each block contains: a Frequency Decomposition Unit (FDUnit), which uses 3D Discrete Cosine Transform (DCT) to decompose the input feature map into high-frequency and low-frequency components; a high-frequency branch, which includes a 3D windowed multi-head self-attention layer with a fixed window size of 4×4×4, used to extract high-frequency detail features; a low-frequency branch, which includes an improved 3D Mamba StateSpace Modeling (3D Mamba StateSpace Modeling) to achieve linear complexity global context modeling; and a Frequency Fusion Unit (FFUUnit), which fuses the output features of the high-frequency and low-frequency branches using a learnable weighted fusion strategy.

[0083] The input is a feature map F3 from the LCSE module, with dimensions H / 8×W / 8×D / 8 and 128 channels. After processing through four frequency decomposition bi-branch blocks, the output feature map size remains unchanged, while the number of channels gradually increases to 256. The output of each block serves as the input to the next block, forming a hierarchical feature extraction.

[0084] The frequency decomposition unit performs a 3D discrete cosine transform on the input feature map X, and the resulting frequency domain representation is as follows:

[0085] .

[0086] Learnable threshold parameters F is divided into high-frequency regions. and low frequency region Then to and Perform an inverse transform to obtain high-frequency spatial features. and low-frequency spatial characteristics It is expressed as follows:

[0087] ;

[0088] .

[0089] The high-frequency branching structure will input high-frequency spatial features First, window partitioning is performed, dividing the feature map into non-overlapping 3D windows of size 4×4×4. Self-attention is computed independently within each window using a multi-head self-attention mechanism. Window movement is used to alternate between regular and offset window partitioning, enabling cross-window information interaction. Finally, high-frequency enhanced features are output. .

[0090] The low-frequency branch structure will input low-frequency spatial features State-space modeling is performed using an improved 3DMamba module, selectively scanning three spatial dimensions and updating the state in each dimension. The final output is low-frequency global features. .

[0091] Frequency fusion unit and Calculate the channel attention weights separately and The fused features are represented as follows:

[0092] ;

[0093] in This indicates element-wise multiplication.

[0094] Linear complexity is achieved through Mamba branching, thus solving the problem of excessive computational complexity (corresponding to technical problem 1).

[0095] Frequency decomposition separates high-frequency and low-frequency information to avoid feature conflicts during feature fusion (corresponding to technical problem 2).

[0096] The high-frequency branch is specifically designed to enhance edge response and improve boundary awareness (corresponding to technical issue 3).

[0097] like Figure 4As shown, the MSFAG module is located between the encoder and decoder, and is used to dynamically select encoder features that match the decoder layer. The gate generation unit generates attention weights for encoder features at each scale based on the semantic features of the current decoder layer. The feature selection unit performs weighted selection of output features from each encoder layer. The feature transformation unit adjusts the number of channels of the selected features through a 1×1×1 convolution to match them with the decoder layer.

[0098] The input to this module consists of the multi-scale coding features F1 (H / 4×W / 4×D / 4, 64) and F2 (H / 8×W / 8×D / 8, 128) output from the LCSE module, the deep features F3 (H / 8×W / 8×D / 8, 256) output from the FDDB-T module, and the current layer features of the decoder. Composition. (The rest of the text appears to be incomplete and requires further context.) After global average pooling and a two-layer fully connected network, a gated vector g=[g1, g2, g3] is generated, satisfying g1+g2+g3=1. Then, F1, F2, and F3 are upsampled to the same level as... Same size, obtained Then the result of the special selection can be expressed as follows:

[0099] ;

[0100] Finally, the weighted and fused features are... The data is then passed to the corresponding layer of the decoder for splicing.

[0101] By using dynamic weighted fusion, feature conflicts caused by simple splicing are avoided (corresponding to technical problem 2).

[0102] Gating mechanisms can suppress the effects of irrelevant noise such as material density fluctuations and scattering artifacts (corresponding to technical issue 3).

[0103] like Figure 5 As shown, the BARM module, located after the decoder, refines the boundaries of the coarse segmentation results. The boundary extraction unit extracts the boundary regions of the coarse segmentation results through differentiable morphological gradient operations. The boundary consistency constraint unit calculates the cross-entropy loss of the boundary regions, forcing the model to focus on boundary pixels. The 3D trainable guided filter layer uses industrial CT images as a guide map to smooth the coarse segmentation results while preserving the edges.

[0104] The input is the coarse segmentation result. and raw industrial CT images All dimensions are H×W×D. During boundary extraction, After 3D expansion and erosion operations, its boundary mask can be represented as follows:

[0105] ;

[0106] Subsequently, at the boundary mask The cross-entropy loss between the segmentation result and the true label is calculated for the covered region. .by For the guiding diagram, Filtering is performed, and the filter kernel parameters are learned autonomously by the network through training. Finally, the refined segmentation result is output. .

[0107] For output voxels The filtering result is expressed as:

[0108] ;

[0109] in and These are linear coefficients learned by minimizing the prediction error within a local window, specifically implemented through trainable 3D convolutional layers.

[0110] The boundary extractor forces the model to focus on the boundary region, thus solving the problem of boundary blurring in low-contrast images (corresponding to technical problem 3).

[0111] Guided filtering maintains edge sharpness and adapts to defects of different shapes and sizes (corresponding to technical issue 5).

[0112] The self-supervised pre-training module is used to pre-train model parameters on large-scale unlabeled 3D industrial CT images. The volume data fusion task involves selecting two different 3D image patches and fusing them at random proportions, requiring the model to predict the fusion coefficients for each voxel. The masking autoencoder task involves randomly masking partial voxels, requiring the model to reconstruct the masked regions.

[0113] During the pre-training phase, the encoder part of the model is jointly trained using the two tasks mentioned above, without the need for label segmentation. In the downstream task phase, the pre-trained weights are loaded, and the model is fine-tuned on a small amount of labeled data.

[0114] During volumetric data fusion, two 3D blocks of the same size but located at different positions are first randomly cropped from the same image. and Generate random fusion coefficient map Each voxel If the values ​​are uniformly distributed in the interval [0,1], the fused image can be represented as follows.

[0115] ;

[0116] Model input is The output is a map of the predicted fusion coefficients. The loss function can be expressed as follows:

[0117] .

[0118] By pre-training on unlabeled data, the dependence of downstream tasks on labeled data is reduced (corresponding to technical issue 4).

[0119] Learn generalized anatomical structures to improve generalization ability under small sample conditions.

[0120] Finally, the model training and inference process is performed: In the training phase, the encoder parts of the LCSE and FDDB-T modules are first trained using a self-supervised pre-training module on a large number of unlabeled 3D industrial CT images to obtain pre-trained weights. Next, the pre-trained weights are loaded to initialize the complete DefectFormer model. Then, the entire model is trained end-to-end using a small number of labeled 3D industrial CT images.

[0121] Loss function design:

[0122] ;

[0123] in The segmentation loss function is based on the Dice similarity coefficient. Let cross-entropy be the loss function. This comes from the BARM module and represents the boundary loss function. The weights for the cross-entropy loss are typically set to 0.1~1.0. The weights for the boundary loss are typically set to 0.1 to 0.5, and need to be adjusted according to the output scale of the BARM module.

[0124] During the inference phase, the input 3D industrial CT image to be segmented is first normalized to H×W×D, and then passed through the LCSE module, FDDB-T module, MSFAG module, decoder, and BARM module in sequence, finally outputting accurate 3D segmentation results of industrial CT defects.

[0125] This application, through the collaborative work of the above five innovative modules, forms a complete 3D industrial CT defect segmentation technology solution. The characteristics and technical problems solved by each module are shown in Table 1 below, and the corresponding beneficial effects of each module are shown in Table 2.

[0126] Table 1 Module Summary Table:

[0127]

[0128] Table 2 Summary of the beneficial effects of the modules:

[0129]

[0130] The method of this application achieves organic integration of local details and global context while maintaining linear computational complexity, enhances the ability to perceive the blurred boundaries of defects in low-contrast industrial CT images, and maintains good generalization performance under small sample conditions, thus comprehensively solving the technical problems existing in the prior art.

[0131] Based on the above system, this application also proposes a Transformer-based multi-frequency boundary-aware industrial CT image defect detection method, which includes the following steps:

[0132] Step 1: Obtain a large number of unlabeled and a small number of labeled 3D industrial CT images as a dataset for self-supervised pre-training;

[0133] Step 2: Construct a 3D industrial CT image segmentation model;

[0134] Step 3: Train the 3D industrial CT image segmentation model to obtain the trained model;

[0135] Step 4: Use the trained model to process the 3D industrial CT image and output the 3D segmentation result of the defect.

[0136] A specific embodiment of this application also proposes a computer device comprising: a memory and a processor, wherein the memory stores instructions executable on the processor. When the processor executes the instructions, it implements the methods described in the above embodiments. The number of memories and processors can be one or more. This computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0137] The computer device may also include a communication interface for communicating with external devices and exchanging data. The devices are interconnected using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor processes instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). The bus can be divided into address buses, data buses, control buses, etc.

[0138] Optionally, in a specific implementation, if the memory, processor, and communication interface are integrated on a single chip, then the memory, processor, and communication interface can communicate with each other through an internal interface.

[0139] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting advanced RISC machines (ARM) architecture.

[0140] This application provides a computer-readable storage medium (such as the memory described above) that stores computer instructions, which, when executed by a processor, implement the method provided in this application.

[0141] Optionally, the memory may include a stored program area and a stored data area, wherein the stored program area may store the operating system and application programs required for at least one function; the stored data area may store data created based on the use of the computer device for mapping. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the computer device for mapping via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A Transformer-based multi-frequency boundary-aware industrial CT image defect detection system, characterized in that: The system includes a 3D industrial CT image segmentation model, which comprises a lightweight convolutional shallow encoder module, a frequency decomposition dual-branch Transformer module, a multi-scale feature aggregation gating mechanism module, a decoder module, and a boundary-aware refinement module. The lightweight convolutional shallow encoder module extracts multi-scale features from the raw 3D industrial CT data and inputs the deep features output from the end into the frequency decomposition dual-branch Transformer module. The frequency decomposition dual-branch Transformer module consists of multiple sequentially connected frequency decomposition dual-branch blocks, each of which includes a frequency decomposition unit, a high-frequency branch, a low-frequency branch, and a frequency fusion unit. The multi-scale feature aggregation gating mechanism module is located between the encoder and decoder and is used to dynamically select encoder features that match the decoder layer. The boundary-aware refinement module refines the boundaries of the coarse segmentation results output by the decoder module.

2. The Transformer-based multi-frequency boundary sensing industrial CT image defect detection system according to claim 1, characterized in that: The lightweight convolutional shallow encoder module consists of three sequentially connected lightweight convolutional blocks. Each lightweight convolutional block contains a depth-separable 3D convolutional layer, an instance normalization layer, a GELU activation function, and an inverted residual structure that are connected in sequence. The convolutional kernel size is 3×3×3 and the stride is 2.

3. The Transformer-based multi-frequency boundary-aware industrial CT image defect detection system according to claim 1, characterized in that: The frequency decomposition dual-branch Transformer module consists of four sequentially connected frequency decomposition dual-branch blocks. The frequency decomposition unit uses 3D discrete cosine transform to decompose the input feature map into high-frequency components and low-frequency components. The high-frequency branch contains a 3D window multi-head self-attention layer with a fixed window size of 4×4×4, which is used to extract high-frequency local detail features. The low-frequency branch incorporates an improved 3D Mamba state-space model for achieving global context modeling with linear complexity; The frequency fusion unit fuses the output features of the high-frequency branch and the low-frequency branch through a learnable weighted fusion strategy.

4. The Transformer-based multi-frequency boundary-aware industrial CT image defect detection system according to claim 2, characterized in that: The multi-scale feature aggregation gating mechanism module generates gating weights for encoder features at each scale based on the semantic features of the current layer of the decoder; it upsamples encoder features at different scales to the same size as the decoder layer; it performs weighted summation of multi-scale features according to the gating weights to achieve dynamic feature selection; and it concatenates the weighted fused features with the decoder features and inputs them into subsequent layers.

5. The Transformer-based multi-frequency boundary-aware industrial CT image defect detection system according to claim 1, characterized in that: The boundary-aware refinement module includes a boundary extraction unit, a boundary consistency constraint unit, and a 3D trainable guided filter layer. The boundary extraction unit extracts the boundary mask from the coarse segmentation result through differentiable morphological gradient operations. The boundary consistency constraint unit is used to calculate the boundary loss function in the region covered by the boundary mask, forcing the model to focus on boundary pixels. The 3D trainable guided filter layer uses the original industrial CT image as the guide map to perform edge-preserving smoothing on the coarse segmentation result. The linear coefficients of the guided filter are obtained through end-to-end learning of the neural network.

6. A Transformer-based multi-frequency boundary-aware industrial CT image defect detection system according to any one of claims 1-5, characterized in that: The system also includes a self-supervised pre-training module for pre-training model parameters on large-scale unlabeled 3D industrial CT images.

7. The Transformer-based multi-frequency boundary-aware industrial CT image defect detection system according to claim 6, characterized in that: The self-supervised pre-training module includes a volume data fusion task and a mask autoencoder task. The volume data fusion task selects two 3D image patches from different regions and fuses them at a random ratio, requiring the model to predict the fusion coefficient of each voxel. The mask autoencoder task randomly masks some voxels, requiring the model to reconstruct the masked region.

8. A multi-frequency boundary-aware industrial CT image defect detection method based on Transformer, characterized in that: The method is implemented using the Transformer-based multi-frequency boundary-aware industrial CT image defect detection system as described in any one of claims 1-7, and includes the following steps: Step 1: Obtain large-scale unlabeled and a small amount of labeled 3D industrial CT data as a dataset for self-supervised pre-training; Step 2: Construct a 3D industrial CT image segmentation model; Step 3: Train the 3D industrial CT image segmentation model to obtain the trained model; Step 4: Use the trained model to process the 3D industrial CT data and output the 3D segmentation results of the defects.

9. The method for detecting defects in industrial CT images based on Transformer multi-frequency boundary sensing according to claim 8, characterized in that: Step 3 specifically includes: During the pre-training phase, the encoder part of the lightweight convolutional shallow encoder module and the frequency decomposition dual-branch Transformer module are jointly trained using two tasks in the self-supervised pre-training module: the volume data fusion task and the masked autoencoder task, to obtain the pre-trained weights. The volume data fusion task involves randomly cropping two 3D blocks from different locations in the same image, generating a random fusion coefficient map, and then fusing them proportionally. The model is required to predict the fusion coefficient of each voxel. Simultaneously, a mask autoencoder task is performed, which randomly masks a portion of the voxel region and requires the model to reconstruct the original signal of the masked region. Then, pre-trained weights are loaded to initialize the 3D industrial CT image segmentation model. Subsequently, the entire model is trained end-to-end using a small amount of labeled 3D industrial CT data.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 8 or 9.

Images