Radiographic detection image defect identification system and method based on RE-Ground DINO

Abstract

The invention relates to the technical field of industrial nondestructive testing and computer vision, in particular to a ray detection image defect recognition system and method based on RE-Ground DINO, and the system comprises a data input module, a multi-modal feature extraction module, a feature enhancement module, a coding and decoding module, a multi-level recall enhancement module and an output integration module. The core of the system is that a multi-level recall enhancement module is used for carrying out layer-by-layer iteration processing on object query features of each layer output by a coding and decoding module, sequentially executing bounding box prediction, spatial semantic enhancement, double-path decoding and confidence coefficient calibration, and finally integrating and outputting a detection result. The technical problems that in X-ray image defect detection of an existing deep learning model, the recall rate of difficult defects such as tiny defects and low-contrast defects is insufficient, and high precision is difficult to maintain at the same time are solved. Compared with the prior art, the method has the advantages that the sensing and distinguishing capability of difficult case defects is enhanced, the detection recall rate can be remarkably improved, high precision can be effectively maintained, and the dual requirements for zero leak detection and high reliability in industrial detection are met.

Description

Technical Field

[0001] This invention relates to the fields of industrial nondestructive testing and computer vision technology, specifically to a defect recognition system and method for X-ray inspection images based on RE-GroundingDINO. Background Technology

[0002] X-ray imaging technology, with its penetrating power and material resolution capabilities, has become a core tool in fields such as industrial non-destructive testing, medical diagnosis, and security inspection. In industrial inspection, traditional defect assessment mainly relies on manual interpretation by professionals or computer-aided analysis. However, manual methods are easily affected by subjective experience, film quality, and fatigue factors. When faced with high-resolution, large-volume data, the risk of misjudgment and omission is significant, which may lead to serious consequences in high-risk fields such as aerospace.

[0003] In recent years, deep learning has significantly propelled the development of this field. Object detection models based on convolutional neural networks (CNNs), such as the single-stage RetinaNet and the two-stage Mask R-CNN, have been widely applied to defect identification. Subsequently, cross-modal detection models based on the Transformer architecture and integrating visual and linguistic information (such as GroundingDINO) have demonstrated stronger semantic understanding and open-set recognition potential. By deeply fusing image and text descriptions at the feature level, the GroundingDINO model can achieve general object detection guided by text without the need for predefined categories, providing a new approach for industrial defect detection.

[0004] However, even advanced models like GroundingDINO still have inherent limitations when directly applied to industrial X-ray defect detection scenarios with high reliability requirements, failing to systematically solve the following key issues:

[0005] 1. High rate of missed detection of difficult defects: The model is not sensitive enough to defects with low contrast, small size or occlusion, and lacks an adaptive attention mechanism for potential missed detection areas;

[0006] 2. Poor training stability of high-resolution images: Inputting high-resolution images (such as 2000×2000 or higher) leads to a sharp increase in computational complexity, unstable gradients, and high memory usage, which affects model convergence and detection performance.

[0007] 3. Scarcity of samples and long-tail distribution: The serious defects are the scarcity of samples and the high cost of annotation, which leads to the model's weak generalization ability on classes with few or zero samples;

[0008] 4. Precision and recall are difficult to balance: Existing methods lack adaptive mechanisms and cannot intelligently balance recall and precision according to sample complexity, often resulting in one being sacrificed for the other.

[0009] 5. Difficulty in engineering implementation: Complex models (especially Transformer-type models) face challenges such as complex parameter configuration, unclear training strategies, and low memory efficiency when deployed. There is a lack of end-to-end optimization solutions for high-resolution X-ray detection.

[0010] 6. Recall Bottleneck for "Zero Missed Detections": Existing cross-modal detection models (such as GroundingDINO) are designed with universality and openness in mind. Their detection heads and loss functions are not optimized for the extreme reliability goal of "zero missed detections" in industrial scenarios. When dealing with difficult defects in industrial X-ray images, there is still a bottleneck in improving recall, especially for small, low-contrast defects, making it difficult to meet the "zero tolerance" detection standards for defects in aerospace, precision manufacturing, and other fields.

[0011] Therefore, the question is how to design a recall-enhanced (RE-) improvement scheme, namely RE-GroundingDINO, for high-resolution X-ray images based on advanced architectures such as GroundingDINO, with the core objective of systematically and significantly improving the detection recall rate, while ensuring a defect detection system and method with high accuracy and engineering feasibility. Summary of the Invention

[0012] One of the objectives of this invention is to provide a defect recognition system for X-ray inspection images based on RE-GroundingDINO, which solves the problem that existing deep learning-based defect detection models are difficult to achieve high recall for difficult defects such as small and low-contrast defects in complex industrial X-ray images, and often fail to effectively maintain detection accuracy while improving recall.

[0013] To achieve the above objectives, a radiographic image defect recognition system based on RE-GroundingDINO is provided, comprising the following modules:

[0014] Data input module: used to acquire X-ray imaging image data and corresponding defect text from CT / DR inspection equipment; the image data includes three-dimensional CT image data, two-dimensional CT image data and DR image data;

[0015] Multimodal feature extraction module: used to input image data and defective text into a hierarchical convolutional neural network and a pre-trained language model respectively for feature extraction, extracting image features and text features;

[0016] Feature enhancement module: Used to achieve deep interaction and fusion of visual and linguistic features through a cross-modal attention mechanism to obtain fused features;

[0017] The encoding and decoding module is used to perform deep encoding and decoding of fused features. The encoder is used to extract global context features of the image, and the decoder takes learnable object queries and text features as input and generates hidden state sequences and reference point coordinates of object queries through iterative optimization.

[0018] Multi-level recall enhancement module: used to perform layer-by-layer iterative processing on the object query features output by the encoding and decoding module, corresponding to each layer of the decoder; the layer-by-layer iterative processing includes: receiving the object query features output by the decoder at each layer; for each layer of features, sequentially performing bounding box coordinate prediction, spatial semantic enhancement, and dual-path decoding processing; generating classification logical values ​​for the processed features and performing confidence calibration; integrating the outputs of all layers to generate the final detection result;

[0019] Output integration module: Used to output detection results, which include bounding box coordinates, category, and confidence level.

[0020] Furthermore, the iterative processing of the multi-level recall enhancement module specifically includes the following steps:

[0021] (a) Receive object query features containing multiple hidden states from the decoder output in the encoding / decoding module;

[0022] (b) For each layer, generate the bounding box coordinate prediction for the current layer based on the object query features of that layer;

[0023] (c) Based on the bounding box coordinate prediction, spatial semantic enhancement and dual-path decoding are sequentially performed on the object query features of the current layer to obtain the enhanced object query features;

[0024] (d) Based on the enhanced object query features, generate the classification logic value of the current layer and calibrate it through the confidence calibration unit;

[0025] (e) For each layer output of the decoder, repeat steps (b) to (d) for iterative processing and enhancement;

[0026] (f) Finally, integrate the outputs of all layers to generate the final detection result.

[0027] Furthermore, the calculation formula for the bounding box coordinate prediction is as follows:

[0028]

[0029] in, This represents the coordinate offset of the l-th bounding box. This indicates bounding box prediction MLP. This represents the bounding box coordinates predicted at layer l. Let be the coordinates of the reference point at layer l, and σ be the sigmoid function. It is the inverse sigmoid function. The object query feature is the output of the l-th layer decoder.

[0030] Furthermore, the multi-level recall enhancement module includes a spatial semantic enhancement unit. This unit employs location encoding based on bounding box coordinates and concatenates it with text features to form key-value pairs. It also enhances object query features through an attention mechanism. The calculation formula is as follows:

[0031]

[0032]

[0033] in, Indicates spatial semantic query, , Represents spatial semantic key-value pairs, , and This represents the spatial semantic projection matrix, PositionEncode represents the position encoding function, and Concat represents the concatenation operation. Represents spatial semantic attention weights, Let α represent the spatial semantic enhancement feature, and let α represent the learnable scaling factor. Enhanced object query features after spatial semantic enhancement; Represents the features of the projected text. for The transpose of the matrix, This represents the dimension of the key vector.

[0034] Furthermore, the multi-level recall enhancement module includes a dual-path detection head unit that implements dual-path decoding processing; the dual-path detection head unit includes a parallel positioning path and a classification path, where the positioning path handles positioning tasks and the classification path handles classification tasks; the outputs of the two paths are fused using adaptive weights to obtain enhanced object query features; the calculation formula is as follows:

[0035]

[0036] in, This represents the output features of the l-th layer localization path. This represents the output features of the classification path at layer l. , These represent localization and classification MLPs, respectively. Let represent the dual-path fusion feature of the l-th layer, i.e., the enhanced object query feature, and β be the adaptive weight parameter.

[0037] Furthermore, the formula for calculating the classification logical value of the current layer is as follows:

[0038]

[0039] in, This represents the original similarity score of the l-th layer. The logits represent the classification of the l-th layer, where τ is the temperature parameter; for The transpose of the matrix;

[0040] The confidence calibration unit is based on a confidence calibration mechanism for uncertainty estimation. First, it estimates the uncertainty of the prediction through network branching, and then uses the uncertainty information to adaptively adjust the classification logits, as shown in the following formula:

[0041]

[0042] in, This represents the uncertainty estimate at level l. MLP represents uncertainty estimation. Indicates calibration weights MLP, This represents the classification logits after calibration at level l, i.e., the confidence level.

[0043] Furthermore, the multi-level recall enhancement module also includes an adaptive query expansion unit, used to generate additional object query features during the training phase to expand attention coverage of hard-to-detect regions; the calculation formula is as follows:

[0044]

[0045] in, Representing salient features, AttentionPooling represents the attention pooling function. This indicates an expanded query vector, and GlobalAvgPool indicates global average pooling. This represents the expanded initial query. This represents the l-th feature projection layer, belonging to which... This represents the first feature projection layer. This indicates the initial query.

[0046] Furthermore, the total loss function of the system is a composite loss function, which includes at least bounding box regression loss, classification cross-entropy loss, and a recall-enhanced specificity loss; the formula for calculating the composite loss function is as follows:

[0047]

[0048] in, For the total loss function, For bounding box regression loss, For classification cross-entropy loss, To enhance specificity and reduce losses during recall, , , , All are loss weighting coefficients;

[0049] The superscript ^l indicates the l-th layer. .

[0050] Furthermore, the system employs a dynamic threshold adjustment strategy during the inference phase. This strategy adaptively adjusts the detection threshold based on the calibrated confidence level and the estimated uncertainty to filter the final detection results. The dynamic threshold adjustment includes:

[0051]

[0052] in, This represents the adjusted detection threshold. ΔT represents the basic detection threshold, and ΔT represents the threshold adjustment range. Based on confidence level And the adaptive threshold function for uncertainty U, The final detection result is represented by yi, where yi represents the set of detection results and pi represents the confidence level.

[0053] The second objective of this invention is to provide a method for defect identification in X-ray inspection images based on RE-GroundingDINO, which utilizes the aforementioned system.

[0054] Principles and advantages:

[0055] 1. This solution achieves a synergistic improvement in both recall and precision. Specifically, through iterative processing of bounding box coordinate prediction, spatial semantic enhancement, dual-path decoding, and confidence calibration, the algorithm dynamically strengthens its perception and discrimination capabilities for potential defective regions (especially small, low-contrast regions) during inference. This significantly reduces the false negative rate (improves recall) while effectively suppressing false positives that may be introduced by relaxing detection conditions, maintaining high detection precision and overcoming the dilemma of traditional methods where both are difficult to achieve simultaneously. It also improves the systematic capture capability for difficult defects: the entire system, from multimodal feature fusion and context encoding to a dedicated recall enhancement process, forms a targeted processing chain, systematically enhancing the robust detection capability for various difficult defects.

[0056] 2. By introducing a multi-level recall enhancement module, especially its spatial semantic enhancement unit and adaptive query expansion unit, the model can dynamically enhance feature extraction and attention allocation for difficult defect regions with low background contrast, small size, or occlusion, thereby effectively reducing the risk of missed detections. While ensuring high accuracy, it achieves a high recall target of near "zero missed detections" and significantly improves the recall rate for difficult defects.

[0057] 3. This scheme optimizes the processing flow of high-resolution features through the collaborative design of the feature enhancement fusion module and the Transformer encoder-decoder module. It also employs a dual-path detection head unit to decouple the localization and classification tasks, mitigating gradient instability and convergence difficulties caused by high input resolution. This allows the model to be stably trained on high-resolution X-ray images and maintain excellent detection performance, thus enhancing training stability and detection performance under high-resolution images.

[0058] 4. This scheme utilizes a cross-modal attention mechanism to deeply fuse image and textual descriptive information. By guiding visual feature understanding through textual semantics, it enhances the feature learning and discrimination capabilities for scarce defect samples and unknown defect types, thereby improving the algorithm's generalization robustness in scenarios with imbalanced data distribution. It also improves the model's generalization ability under small sample and long-tailed distributions.

[0059] 5. This scheme uses a confidence calibration unit to perform adaptive calibration of predicted scores based on uncertainty, and combines this with a dynamic threshold adjustment strategy during the inference stage. This enables the system to intelligently adjust the judgment criteria according to the complexity of specific samples, automatically achieving the optimal balance between high recall and low false alarm rate without manual intervention. This overcomes the systemic dilemma of traditional methods where these two aspects are difficult to balance simultaneously, thus achieving an intelligent balance between detection accuracy and recall.

[0060] 6. This solution provides an end-to-end engineering optimization scheme to improve the feasibility of implementation: The system has been specifically designed from model architecture (such as hierarchical design and dual-path decoupling), training strategy (composite loss function and recall perception optimization) to inference mechanism (dynamic threshold), forming a complete optimization scheme from training stability to inference efficiency, which significantly reduces the engineering threshold and resource consumption of deploying complex Transformer models in actual industrial inspection environments. Attached Figure Description

[0061] Figure 1 This is a logic block diagram of a radiographic image defect recognition system based on RE-GroundingDINO, according to an embodiment of the present invention. Detailed Implementation

[0062] The following detailed description illustrates the specific implementation method:

[0063] Example

[0064] A X-ray inspection image defect recognition system based on RE-GroundingDINO, basically as follows: Figure 1 As shown, it includes a server, specifically comprising the following modules:

[0065] Data input module: used to acquire X-ray imaging image data and corresponding defect text from CT / DR inspection equipment; the image data includes 3D CT image data, 2D CT image data and DR image data; acquire slice images of 3D CT image data, 2D CT image data or DR image data and their corresponding annotation files, the pixel size of the slice images is about 1000×1000, and the defect types are classified and graded as needed. The images and corresponding annotation files are randomly divided into training set and test set at a ratio of 8:2, and then 10% of the data in the training set is randomly divided as validation set.

[0066] A multimodal feature extraction module is used to input image data and defective text into a hierarchical convolutional neural network and a pre-trained language model, respectively, for feature extraction, extracting image features and text features. This module is responsible for extracting multi-level visual features and deep semantic features from the input image and text, respectively, providing a high-quality input representation for subsequent cross-modal fusion. The multimodal feature extraction module includes:

[0067] The visual feature extraction path is used to extract image features from the input image through a hierarchical convolutional neural network (such as ResNet, Swin Transformer, etc.). Specifically, the input image I is processed through four levels of convolutional blocks to generate a feature map set V. Subsequently, dimensionality projection is performed through 1×1 convolution and group normalization to obtain the standardized visual features V_pro.

[0068] Visual feature extraction:

[0069]

[0070] in, This represents the feature map output by the i-th level convolutional layer. This represents the i-th level convolutional neural network. Represents the feature projection layer. GN represents a 1×1 convolution operation, and GN represents a group normalization operation.

[0071] The language feature extraction path is used to extract text features from the input defective text using pre-trained language models (such as BERT, RoBERTa, etc.). The input text T is first converted into a word sequence by a tokenizer, then passed through a multi-layer Transformer encoder to obtain a context-aware text representation H_text, and finally aligned to the visual feature dimension through a linear projection layer.

[0072] Language feature extraction is as follows:

[0073] Text input is processed through a pre-trained language model:

[0074]

[0075] in, The tokenizer represents the sequence of words after word segmentation. Represents the initial word embedding vector. This represents the hidden state of the Lth layer BERT output. Represents the features of the projected text. This indicates a unified dimension for the model.

[0076] Feature Enhancement Module: This module enables deep interaction and fusion of visual and linguistic features through a cross-modal attention mechanism, resulting in fused features. It achieves deep interaction and fusion of visual and linguistic features, establishes semantic associations between image regions and text concepts through a cross-modal attention mechanism, and enhances the discriminative power of features.

[0077] It adopts a three-tiered structure: a cross-modal attention layer, a self-attention layer, and a feedforward neural network layer.

[0078] Cross-modal attention layer: Using image features as queries and text features as keys and values, it calculates the relevance weights between image regions and text concepts to achieve semantically guided visual feature enhancement; the calculation formula is as follows:

[0079]

[0080] Where Q represents the query matrix, K represents the key matrix, and V represents the value matrix. Let represent the learnable projection matrices, This represents the cross-attention weight matrix. Represents the dimension of the key vector. Represents the attention mask matrix; Indicates fusion characteristics;

[0081] Self-attention layer: Applying a self-attention mechanism to the fused features after cross-modal fusion to capture long-range dependencies within the image and enhance the context-awareness of the features; the calculation formula is as follows:

[0082]

[0083] in, represents the self-attention output feature, MultiHeadSelfAttention represents the multi-head self-attention mechanism, and LayerNorm represents the layer normalization operation;

[0084] Feedforward neural network layer: This layer further refines and fuses features through two fully connected layers and a non-linear activation function; the calculation formula is as follows:

[0085]

[0086] in, This represents the output features of the feedforward network; FNN stands for feedforward neural network. This represents the final enhanced feature, i.e., the refined fused feature.

[0087] The encoding / decoding module is used for deep encoding and decoding of fused features. The encoder is used to extract global context features of the image, and the decoder takes learnable object query and text features as input and generates the hidden state sequence and reference point coordinates of the object query through iterative optimization. The encoding / decoding module is a Transformer encoder-decoder, based on the standard Transformer architecture, which performs deep encoding and decoding of the enhanced multimodal features to generate the hidden state sequence and reference point coordinates of the object query.

[0088] The encoder section consists of L_enc identical encoder layers stacked together. Each layer contains a multi-head self-attention mechanism and a feedforward neural network to extract global contextual features of the image. The encoder layers are as follows:

[0089]

[0090] in, This represents the encoder's initial input. This represents the output of the l-th encoder layer, and TransformerEncoderLayer represents the Transformer encoder layer. Indicates the number of encoder layers;

[0091] The decoder section consists of L_dec identical decoder layers stacked together. Each layer contains masked multi-head self-attention, encoder-decoder cross-attention, and a feedforward network. The decoder takes a learnable object query as input and generates the final detection result through iterative optimization. The decoder layers are as follows:

[0092]

[0093] in, Indicates the initial query of the decoder. This indicates a learnable target query, where PE stands for location code. The output of the l-th layer decoder is represented by `TransformerDecoderLayer`, which represents the Transformer decoder layer. Indicates the number of decoder layers.

[0094] Multi-level recall enhancement module: used to perform layer-by-layer iterative processing on the object query features output by the encoding and decoding module, corresponding to each layer of the decoder; the layer-by-layer iterative processing includes: receiving the object query features output by the decoder at each layer; for each layer of features, sequentially performing bounding box coordinate prediction, spatial semantic enhancement, and dual-path decoding processing; generating classification logical values ​​for the processed features and performing confidence calibration; integrating the outputs of all layers to generate the final detection result;

[0095] The iterative processing of the multi-level recall enhancement module specifically includes the following steps:

[0096] (a) Receive object query features containing multiple hidden states from the decoder output in the encoding / decoding module;

[0097] (b) For each layer, based on the object query features of that layer, generate the bounding box coordinate prediction for the current layer; the calculation formula for the bounding box coordinate prediction is as follows:

[0098]

[0099] in, This represents the coordinate offset of the l-th bounding box. This indicates bounding box prediction MLP. This represents the bounding box coordinates predicted at layer l. Let be the coordinates of the reference point at layer l, and σ be the sigmoid function. It is the inverse sigmoid function. The object query feature is the output of the l-th layer decoder.

[0100] (c) Based on the bounding box coordinate prediction, spatial semantic enhancement and dual-path decoding are sequentially performed on the object query features of the current layer to obtain the enhanced object query features; the multi-level recall enhancement module includes a spatial semantic enhancement unit, which uses positional encoding based on bounding box coordinates and concatenates it with text features to form key-value pairs, and enhances the object query features through an attention mechanism; the calculation formula is as follows:

[0101]

[0102]

[0103] in, Indicates spatial semantic query, , Represents spatial semantic key-value pairs, , and This represents the spatial semantic projection matrix, PositionEncode represents the position encoding function, and Concat represents the concatenation operation. Represents spatial semantic attention weights, Let α represent the spatial semantic enhancement feature, and let α represent the learnable scaling factor. This refers to the enhanced object query features after spatial semantic enhancement.

[0104] The multi-level recall enhancement module includes a dual-path detection head unit that implements dual-path decoding processing. The dual-path detection head unit includes a parallel localization path and a classification path; the localization path handles localization tasks, and the classification path handles classification tasks. The outputs of the two paths are fused using adaptive weights to obtain enhanced object query features. The calculation formula is as follows:

[0105]

[0106] in, This represents the output features of the l-th layer localization path. This represents the output features of the classification path at layer l. , These represent localization and classification MLPs, respectively. Let represent the dual-path fusion feature of the l-th layer, i.e., the enhanced object query feature, and β be the adaptive weight parameter.

[0107] (d) Based on the enhanced object query features, generate the classification logical value for the current layer, and calibrate it via the confidence calibration unit; the calculation formula for generating the classification logical value for the current layer is as follows:

[0108]

[0109] in, This represents the original similarity score of the l-th layer. The logits represent the classification of the l-th layer, where τ is the temperature parameter; for The transpose of the matrix;

[0110] The confidence calibration unit is based on a confidence calibration mechanism for uncertainty estimation. First, it estimates the uncertainty of the prediction through network branching, and then uses the uncertainty information to adaptively adjust the classification logits, as shown in the following formula:

[0111]

[0112] in, Indicates uncertainty estimation, MLP represents uncertainty estimation. Indicates calibration weights MLP, This represents the calibrated classification logits, i.e., the confidence level.

[0113] (e) For each layer output of the decoder, repeat steps (b) to (d) for iterative processing and enhancement;

[0114] (f) Finally, integrate the outputs of all layers to generate the final detection result.

[0115] The multi-level recall enhancement module also includes an adaptive query expansion unit, used to generate additional object query features during the training phase to expand attention coverage of hard-to-detect regions; the calculation formula is as follows:

[0116]

[0117] in, Representing salient features, AttentionPooling represents the attention pooling function. This indicates an expanded query vector, and GlobalAvgPool indicates global average pooling. This represents the expanded initial query; This represents the l-th feature projection layer, belonging to which... This represents the first feature projection layer. This indicates the initial query.

[0118] Output integration module: used to output detection results, which include bounding box (selection of defect parts), category (defect category) and confidence level;

[0119]

[0120] Where I represents the input image data, T represents the input defect text, and Y represents the set of detection results {y_i}= {(p_i, b_i, c_i)}, which includes confidence, bounding box and category.

[0121] The multi-level output of confidence score, bounding box, and category is as follows:

[0122]

[0123] The total loss function of the system is a composite loss function, which includes at least bounding box regression loss, classification cross-entropy loss, and a recall-enhanced specificity loss; the formula for calculating the composite loss function is as follows:

[0124]

[0125] in, For the total loss function, For bounding box regression loss, For classification cross-entropy loss, To enhance specificity and reduce losses during recall, , , , All of these are loss weighting coefficients.

[0126] The system employs a dynamic threshold adjustment strategy during the inference phase. This strategy adaptively adjusts the detection threshold based on the calibrated confidence level and the estimated uncertainty to filter the final detection results. The dynamic threshold adjustment includes:

[0127]

[0128] in, This represents the adjusted detection threshold. ΔT represents the basic detection threshold, and ΔT represents the threshold adjustment range. An adaptive threshold function based on confidence and uncertainty. This represents the final detection result. yi represents the set of detection results, and pi represents the confidence level.

[0129] In all formulas, the superscript ^l represents the l-th layer, the subscript _l represents the l-th level feature, the superscript ^⊤ represents matrix transpose, σ represents the sigmoid function, Concat represents the concatenation operation, and Softmax represents the softmax normalization function.

[0130] The core technology of this solution is based on the open-source "GroundingDINO" model. Addressing the specific requirement of "zero missed detections" in industrial inspection, the structure of the GroundingDINO model is specifically improved, proposing a ray-based image defect recognition algorithm based on RE-GroundingDINO. The "RE-" prefix stands for "Recall-Enhanced," which embodies a specialized modification of the original GroundingDINO architecture. This is achieved by systematically embedding an innovative module called a "multi-level recall enhancement module" into the standard GroundingDINO inspection process. This module is not simply added but deeply integrated into its encoder-decoder framework, customizing the query features of each layer of the decoder output. It aims to overcome the bottleneck problem of insufficient recall rate in industrial defect detection scenarios, especially the missed detection of difficult defects. The effectiveness of the algorithm is verified on a railway casting defect detection dataset, and the comparative experimental results with other models are as follows:

[0131]

[0132] Wherein, mAP is the mean precision and AR is the mean recall. All of the above metrics were tested at a confidence level of 50%.

[0133] A defect identification method based on ray-detected images using RE-GroundingDINO includes the following steps:

[0134] Data input steps: Obtain X-ray imaging data and corresponding defect text from the CT / DR inspection equipment; the image data includes 3D CT image data, 2D CT image data, and DR image data;

[0135] Multimodal feature extraction steps: Input image data and defective text into a hierarchical convolutional neural network and a pre-trained language model respectively for feature extraction, extracting image features and text features;

[0136] Feature enhancement steps: Image features and text features are combined and fused through a cross-modal attention mechanism to achieve deep interaction between visual and linguistic features, resulting in fused features;

[0137] Encoding and decoding steps: Deep encoding and decoding of fused features. The encoder is used to extract global context features of the image. The decoder takes learnable object queries and text features as input and generates hidden state sequences and reference point coordinates of object queries through iterative optimization.

[0138] Multi-level recall enhancement steps: The object query features output by the encoding and decoding module, corresponding to each layer of the decoder, are processed iteratively layer by layer; the iterative processing layer by layer includes: receiving the object query features output by the decoder for each layer; for each layer of features, performing bounding box coordinate prediction, spatial semantic enhancement and dual-path decoding processing in sequence; generating classification logical values ​​for the processed features and performing confidence calibration; integrating the outputs of all layers to generate the final detection result;

[0139] The iterative processing of the multi-level recall enhancement module specifically includes the following steps:

[0140] (a) Receive object query features containing multiple hidden states from the decoder output in the encoding / decoding module;

[0141] (b) For each layer, based on the object query features of that layer, generate the bounding box coordinate prediction for the current layer; the calculation formula for the bounding box coordinate prediction is as follows:

[0142]

[0143] in, This represents the coordinate offset of the l-th bounding box. This indicates bounding box prediction MLP. This represents the bounding box coordinates predicted at layer l. Let be the coordinates of the reference point at layer l, and σ be the sigmoid function. It is the inverse sigmoid function. The object query feature is the output of the l-th layer decoder.

[0144] (c) Based on the bounding box coordinate prediction, spatial semantic enhancement and dual-path decoding are sequentially performed on the object query features of the current layer to obtain the enhanced object query features; spatial semantic enhancement is performed through a spatial semantic enhancement unit, which uses positional encoding based on bounding box coordinates and concatenates it with text features to form key-value pairs, and enhances the object query features through an attention mechanism; the calculation formula is as follows:

[0145]

[0146]

[0147] in, Indicates spatial semantic query, , Represents spatial semantic key-value pairs, , and This represents the spatial semantic projection matrix, PositionEncode represents the position encoding function, and Concat represents the concatenation operation. Represents spatial semantic attention weights, Let α represent the spatial semantic enhancement feature, and let α represent the learnable scaling factor. Enhanced object query features after spatial semantic enhancement; Represents the features of the projected text. for The transpose of the matrix, This represents the dimension of the key vector.

[0148] Dual-path decoding is performed using a dual-path detection head unit. This dual-path detection head unit includes a parallel localization path and a classification path; the localization path handles localization tasks, and the classification path handles classification tasks. The outputs of the two paths are fused using adaptive weights to obtain enhanced object query features. The calculation formula is as follows:

[0149]

[0150] in, This represents the output features of the l-th layer localization path. This represents the output features of the classification path at layer l. , These represent localization and classification MLPs, respectively. Let represent the dual-path fusion feature of the l-th layer, i.e., the enhanced object query feature, and β be the adaptive weight parameter.

[0151] (d) Based on the enhanced object query features, generate the classification logical value for the current layer, and calibrate it via the confidence calibration unit; the calculation formula for generating the classification logical value for the current layer is as follows:

[0152]

[0153] in, This represents the original similarity score of the l-th layer. The logits represent the classification of the l-th layer, where τ is the temperature parameter;

[0154] The confidence calibration unit is based on a confidence calibration mechanism for uncertainty estimation. First, it estimates the uncertainty of the prediction through network branching, and then uses the uncertainty information to adaptively adjust the classification logits, as shown in the following formula:

[0155]

[0156] in, Indicates uncertainty estimation, MLP represents uncertainty estimation. Indicates calibration weights MLP, This represents the calibrated classification logits.

[0157] (e) For each layer output of the decoder, repeat steps (b) to (d) for iterative processing and enhancement;

[0158] (f) Finally, integrate the outputs of all layers to generate the final detection result.

[0159] The multi-level recall enhancement module also includes an adaptive query expansion unit, used to generate additional object query features during the training phase to expand attention coverage of hard-to-detect regions; the calculation formula is as follows:

[0160]

[0161] in, Representing salient features, AttentionPooling represents the attention pooling function. This indicates an expanded query vector, and GlobalAvgPool indicates global average pooling. This represents the expanded initial query.

[0162] Output integration step: Output the detection results, which include bounding box coordinates, category, and confidence level.

[0163]

[0164] Where I represents the input image data, T represents the input defect text, and Y represents the set of detection results {y_i}= {(p_i, b_i, c_i)}, which includes confidence, bounding box and category.

[0165] The multi-level output of confidence score, bounding box, and category is as follows:

[0166]

[0167] The system's total loss function is a composite loss function, which includes at least bounding box regression loss, classification cross-entropy loss, and a recall-enhanced specificity loss; the formula for calculating the composite loss function is as follows:

[0168]

[0169] in, For the total loss function, For bounding box regression loss, For classification cross-entropy loss, To enhance specificity and reduce losses during recall, , , , All of these are loss weighting coefficients.

[0170] The system employs a dynamic threshold adjustment strategy during the inference phase. This strategy adaptively adjusts the detection threshold based on the calibrated confidence level and the estimated uncertainty to filter the final detection results. The dynamic threshold adjustment includes:

[0171]

[0172] in, This represents the adjusted detection threshold. ΔT represents the basic detection threshold, and ΔT represents the threshold adjustment range. An adaptive threshold function based on confidence and uncertainty. This is the final test result.

[0173] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A X-ray inspection image defect recognition system based on RE-GroundingDINO, characterized in that, Includes the following modules: Data input module: used to acquire X-ray imaging image data and corresponding defect text from CT / DR inspection equipment; the image data includes three-dimensional CT image data, two-dimensional CT image data and DR image data; Multimodal feature extraction module: used to input image data and defective text into a hierarchical convolutional neural network and a pre-trained language model respectively for feature extraction, extracting image features and text features; Feature enhancement module: Used to achieve deep interaction and fusion of visual and linguistic features through a cross-modal attention mechanism to obtain fused features; The encoding and decoding module is used to perform deep encoding and decoding of fused features. The encoder is used to extract global context features of the image, and the decoder takes learnable object queries and text features as input and generates hidden state sequences and reference point coordinates of object queries through iterative optimization. Multi-level recall enhancement module: used to iteratively process the object query features output by the encoding and decoding module, corresponding to each layer of the decoder; The layer-by-layer iterative processing includes: receiving the object query features of each layer output by the decoder; for each layer of features, sequentially performing bounding box coordinate prediction, spatial semantic enhancement, and dual-path decoding; generating classification logical values ​​for the processed features and performing confidence calibration; and integrating the outputs of all layers to generate the final detection result. Output integration module: Used to output detection results, which include bounding box coordinates, category, and confidence level.

2. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 1, characterized in that: The iterative processing of the multi-level recall enhancement module specifically includes the following steps: (a) Receive object query features containing multiple hidden states from the decoder output in the encoding / decoding module; (b) For each layer, generate the bounding box coordinate prediction for the current layer based on the object query features of that layer; (c) Based on the bounding box coordinate prediction, spatial semantic enhancement and dual-path decoding are sequentially performed on the object query features of the current layer to obtain the enhanced object query features; (d) Based on the enhanced object query features, generate the classification logic value of the current layer and calibrate it through the confidence calibration unit; (e) For each layer output of the decoder, repeat steps (b) to (d) for iterative processing and enhancement; (f) Finally, integrate the outputs of all layers to generate the final detection result.

3. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 2, characterized in that: The formula for calculating the bounding box coordinate prediction is as follows: in, This represents the coordinate offset of the l-th bounding box. This indicates bounding box prediction MLP. This represents the bounding box coordinates predicted at layer l. Let be the coordinates of the reference point at layer l, and σ be the sigmoid function. It is the inverse sigmoid function. The object query feature is the output of the l-th layer decoder.

4. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 3, characterized in that: The multi-level recall enhancement module includes a spatial semantic enhancement unit, which uses location encoding based on bounding box coordinates and concatenates it with text features to form key-value pairs, and enhances object query features through an attention mechanism. The calculation formula is as follows: in, Indicates spatial semantic query, , Represents spatial semantic key-value pairs, , and This represents the spatial semantic projection matrix, PositionEncode represents the position encoding function, and Concat represents the concatenation operation. Represents spatial semantic attention weights, Let α represent the spatial semantic enhancement feature, and let α represent the learnable scaling factor. Enhanced object query features after spatial semantic enhancement; Represents the features of the projected text. for The transpose of the matrix, This represents the dimension of the key vector.

5. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 4, characterized in that: The multi-level recall enhancement module includes a dual-path detection head unit that implements dual-path decoding processing; the dual-path detection head unit includes a positioning path and a classification path set in parallel, the positioning path processes the positioning task, and the classification path processes the classification task; the outputs of the two paths are fused through adaptive weights to obtain enhanced object query features. The calculation formula is as follows: in, This represents the output features of the location path at layer l. This represents the output features of the classification path at layer l. , These represent localization and classification MLPs, respectively. Let represent the dual-path fusion feature of the l-th layer, i.e., the enhanced object query feature, and β be the adaptive weight parameter.

6. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 5, characterized in that: The formula for calculating the classification logical value of the current layer is as follows: in, This represents the original similarity score of the l-th layer. The logits represent the classification of the l-th layer, where τ is the temperature parameter; for The transpose of the matrix; The confidence calibration unit is based on a confidence calibration mechanism for uncertainty estimation. First, it estimates the uncertainty of the prediction through network branching, and then uses the uncertainty information to adaptively adjust the classification logits, as shown in the following formula: in, This represents the uncertainty estimate at level l. MLP represents uncertainty estimation. Indicates calibration weights MLP, This represents the classification logits after calibration at level l, i.e., the confidence level.

7. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 6, characterized in that: The multi-level recall enhancement module also includes an adaptive query expansion unit, used to generate additional object query features during the training phase to expand attention coverage of hard-to-detect regions; the calculation formula is as follows: in, Representing salient features, AttentionPooling represents the attention pooling function. This indicates an expanded query vector, and GlobalAvgPool indicates global average pooling. This represents the expanded initial query. This represents the l-th feature projection layer, belonging to which... This represents the first feature projection layer. This indicates the initial query.

8. The X-ray inspection image defect recognition system based on RE-GroundingDINO according to claim 7, characterized in that: The total loss function of the system is a composite loss function, which includes at least bounding box regression loss, classification cross-entropy loss, and a recall-enhanced specificity loss; the formula for calculating the composite loss function is as follows: in, For the total loss function, For bounding box regression loss, For classification cross-entropy loss, To enhance specificity and reduce losses during recall, , , , All are loss weighting coefficients; The superscript ^l indicates the l-th layer. .

9. A radiographic image defect recognition system based on RE-GroundingDINO according to claim 8, characterized in that: The system employs a dynamic threshold adjustment strategy during the inference phase. This strategy adaptively adjusts the detection threshold based on the calibrated confidence level and the estimated uncertainty to filter the final detection results. The dynamic threshold adjustment includes: in, This represents the adjusted detection threshold. ΔT represents the basic detection threshold, and ΔT represents the threshold adjustment range. Based on confidence level And the adaptive threshold function for uncertainty U, The final detection result is represented by yi, where yi represents the set of detection results and pi represents the confidence level.

10. A method for defect identification in X-ray inspection images based on RE-GroundingDINO, characterized in that: The X-ray inspection image defect recognition system based on RE-GroundingDINO, as described in any one of claims 1-9, is employed.

Images