Enhancement-guided X-Ray image defect segmentation method and related equipment

Abstract

This invention relates to the field of artificial intelligence technology, specifically to an enhancement-guided X-Ray image defect segmentation method and related equipment. The method includes: inputting the X-Ray image to be segmented into the defect segmentation model; extracting features from the X-Ray image to be segmented using the image encoder to obtain an image embedding; generating a cue embedding using the cue encoder; performing bidirectional interactive attention calculation on the image embedding and the cue embedding using the image decoder to obtain query features and image features; obtaining the unnormalized score and quality score of the mask based on the query features and image features; and generating a target defect mask based on the unnormalized score and quality score of the mask, which serves as the defect segmentation result of the X-Ray image to be segmented. This invention can improve the accuracy and generalization ability of X-Ray image defect segmentation while taking into account the lightweight nature and real-time performance of the model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an enhanced guided X-ray image defect segmentation method and related equipment. Background Technology

[0002] With the rapid development of new energy vehicles and the energy storage industry, the demand for lithium-ion batteries has increased dramatically, placing higher demands on their manufacturing quality. As a core component of batteries, the surface defects of battery electrodes (such as uneven coatings and scratches) directly affect the battery's electrochemical and safety performance. Currently, the industry widely uses X-ray imaging technology for non-destructive testing of electrodes to identify internal defects.

[0003] However, existing defect segmentation methods in related technologies often suffer from insufficient ability to capture complex defect features and limited segmentation accuracy when processing defects in X-ray images. Furthermore, they are slow to segment, making them unsuitable for real-time industrial inspection scenarios. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an enhanced guided X-Ray image defect segmentation method and related equipment to solve one or more technical problems existing in the prior art and to provide at least one beneficial option or create conditions.

[0005] On one hand, embodiments of the present invention provide a method for X-Ray image defect segmentation based on enhancement guidance, the method comprising the following steps:

[0006] Acquire the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder, and an image decoder.

[0007] The X-Ray image to be segmented is input into the defect segmentation model. The image encoder extracts features from the X-Ray image to be segmented to obtain an image embedding. The cue encoder generates a cue embedding. The image decoder performs bidirectional interactive attention calculation on the image embedding and the cue embedding to obtain query features and image features. Based on the query features and image features, the unnormalized score and quality score of the mask are obtained.

[0008] A target defect mask is generated based on the unnormalized score and quality score of the mask, and is used as the defect segmentation result of the X-Ray image to be segmented.

[0009] Optionally, the step of extracting features from the X-Ray image to be segmented using the image encoder to obtain the image embedding includes:

[0010] Obtain the X-Ray image to be segmented, and convert the X-Ray image to be segmented into a preprocessed image in tensor format; the X-Ray image to be segmented is a single-channel grayscale image;

[0011] Convolutional layers are used to reduce the dimensionality of the preprocessed image to obtain an initial feature map. The initial feature map is then input into a Transformer encoder based on the ViTDeT architecture. Deep features are extracted from the initial feature map through a windowed attention mechanism and a periodically inserted global attention mechanism, and a token map containing multi-scale contextual information is output.

[0012] The token image is sequentially subjected to 1×1 convolution channel adjustment, LayerNorm2d activation function processing, and 3×3 convolution feature optimization to obtain an image embedding containing batch dimension, channel dimension, height dimension, and width dimension.

[0013] Optionally, generating the cue embedding through the cue encoder includes:

[0014] Obtain sparse and dense tooltips, where sparse tooltips include point tooltips and bounding box tooltips, and dense tooltips are segmentation masks;

[0015] The sparse cue is encoded by mapping the 2D coordinates of the point cue to a position encoding vector of a preset dimension through random Fourier feature mapping. The bounding box cue is embedded by superimposing the corresponding corner points on the two corner points respectively to obtain the sparse cue embedding.

[0016] The dense cue is encoded by downsampling the segmentation mask to the same resolution as the image embedding through a convolutional layer, adjusting the number of channels through convolutional mapping, and then combining the dense position encoding generated by the random Fourier mechanism to obtain the dense cue embedding.

[0017] The sparse cue embedding and the dense cue embedding are fused to generate a cue embedding.

[0018] Optionally, the image decoder includes a bidirectional Transformer block and a dynamic masking block; the step of performing bidirectional interactive attention calculation on the image embedding and cue embedding through the image decoder to obtain query features and image features, and obtaining the unnormalized score and quality score of the mask based on the query features and image features, includes:

[0019] The image embedding and prompt embedding are input into a bidirectional Transformer block. After performing self-attention computation on the query vector to establish the association between different types of queries, the image tags with added image location encoding are used as keys. Cross-attention computation is performed on the image tags through the query vector, and the query vector is updated through layer normalization and a feedforward neural network of the query vector. The updated query vector is used as keys. Cross-attention computation is performed on the query vector through the image tags, and the image tags are updated through layer normalization and a feedforward neural network of the image.

[0020] After stacking multiple bidirectional Transformer blocks, the output query token and image token are used as dynamic masking inputs for query features and image features. The query token contains K mask tokens and 1 IoU tag.

[0021] The image token is reshaped into a feature map, and after being added element-wise to the dense cue embedding, it is upsampled to a preset resolution to obtain F1. Each mask token is processed by a multilayer perceptron to obtain F2. At the pixel position, the inner product of the channel features corresponding to F1 and F2 is generated to produce the mask unnormalized score. At the same time, the IoU marker is processed by a multilayer perceptron to output the mask quality score.

[0022] Optionally, the prompt encoder is trained in the following manner:

[0023] The X-Ray image to be segmented is converted into a preprocessed image in tensor format and used as a training sample;

[0024] The preprocessed images are input into the prompt encoder one by one, and the coordinate information of the bounding boxes is stored in a dictionary. The key of the dictionary is the name of the image after removing the suffix. For multiple masks in the preprocessed image, the corresponding multi-boxes are converted into single boxes and then sent to the prompt encoder.

[0025] After training all single frames of a single image, backpropagation is performed. The loss of each single frame is calculated by combining the full intersection-union ratio loss and the binary cross-entropy loss and accumulated to obtain the loss value of the cue encoder. The model parameters are updated according to the loss value of the cue encoder and the optimal model parameters are saved according to the average intersection-union ratio on the validation set.

[0026] Create a tensor identical to the predicted mask, assign 1 to the coordinate region within the box, and obtain the real mask region by finding the intersection with the real mask. The first total loss function is calculated by jointly using binary cross-entropy loss and Focal loss.

[0027] During training, the model parameters are adjusted to minimize the first total loss function, resulting in a fine-tuned cue encoder.

[0028] Optionally, the defect segmentation model is trained in the following manner:

[0029] Obtain a training dataset containing X-Ray images, segmentation masks, and bounding box cues;

[0030] The training dataset is input into the enhanced guided defect segmentation model, the image encoder and cue encoder are frozen, only the decoder part is trained, and the optimal model weights are saved on the validation set according to the average intersection-over-union ratio evaluation metric to obtain the trained model.

[0031] The trained model is pruned using an L1 channel pruning strategy. The pruned model is then fine-tuned. The optimal model weights are saved based on the average intersection-over-union ratio (IoU) evaluation metric on the validation set, resulting in the pruned and fine-tuned model.

[0032] Using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model, knowledge distillation is performed. The differences between the student model output, the teacher model output, and the true label are monitored. The optimal student model weights are saved based on the indicators on the validation set, resulting in a well-trained defect segmentation model.

[0033] Optionally, the process of using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model for knowledge distillation, supervising the differences between the student model output, the teacher model output, and the true labels, and saving the optimal student model weights based on the metrics on the validation set to obtain a trained defect segmentation model includes:

[0034] The original defect segmentation model is used as the teacher model, and the pruned and fine-tuned model is used as the student model. The parameters of the teacher model are frozen so that it only provides forward output.

[0035] The training dataset is input into the student model, and forward computation is performed through the student model to obtain the first mask unnormalized score and the first mask quality score output by the student model; the same training data is input into the teacher model to obtain the second mask unnormalized score and the second mask quality score output by the teacher model, which are used as soft labels.

[0036] The KL divergence between the unnormalized score of the first mask and the unnormalized score of the second mask is calculated as the distillation loss, and the binary cross-entropy loss between the unnormalized score of the first mask and the true label mask is calculated as the classification loss. The distillation loss and the classification loss are weighted and summed to obtain the second total loss function.

[0037] The parameters of the student model are updated through backpropagation based on the second total loss function. During the training process, the average intersection-union ratio (IUR) on the validation set is calculated at a preset number of iterations. When the iteration training reaches a preset condition, the training stops and the student model weights when the average IUR on the validation set is optimal are saved to obtain the trained defect segmentation model. The preset condition is that the number of iterations reaches a preset number of rounds or the index does not improve within a number of consecutive iterations.

[0038] On the other hand, embodiments of the present invention provide an X-Ray image defect segmentation device based on enhanced guidance, comprising:

[0039] The first module is used to acquire the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder and an image decoder.

[0040] The second module is used to input the X-Ray image to be segmented into the defect segmentation model, extract features from the X-Ray image to be segmented by the image encoder to obtain an image embedding, generate a cue embedding by the cue encoder, and perform bidirectional interactive attention calculation on the image embedding and the cue embedding by the image decoder to obtain query features and image features, and obtain the unnormalized score and quality score of the mask based on the query features and image features.

[0041] The third module is used to generate a target defect mask based on the unnormalized score and the quality score of the mask, as the defect segmentation result of the X-Ray image to be segmented.

[0042] On the other hand, embodiments of the present invention provide an enhanced guided X-Ray image defect segmentation system, comprising:

[0043] At least one processor;

[0044] At least one memory for storing at least one program;

[0045] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0046] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described method.

[0047] The embodiments of the present invention have the following beneficial effects:

[0048] This invention achieves accurate defect segmentation of low-light, detail-blurred X-ray electrode images by designing an enhanced guided defect segmentation model that organically combines an image encoder, a cue encoder, an image decoder, and a dynamic masking device. The image encoder employs a ViTDeT architecture-based Transformer encoder, which effectively extracts multi-scale contextual information through periodic insertion of windowed attention and global attention mechanisms, overcoming the shortcomings of traditional convolutional networks in long-distance dependency modeling. The cue encoder innovatively integrates sparse and dense cues, encoding sparse information such as points and bounding boxes and dense information such as segmentation masks into a unified cue embedding, enhancing the model's ability to capture defect features. The image decoder promotes deep fusion of image embedding and cue embedding through bidirectional interactive attention computation, while the dynamic masking device generates high-quality defect masks based on query features and image features, significantly improving segmentation accuracy. Simultaneously, L1 channel pruning and knowledge distillation strategies are introduced during model training, effectively reducing model complexity while maintaining segmentation performance. This meets the real-time and resource constraints of industrial production lines, providing strong technical support for the efficient detection of lithium-ion battery electrode defects. In summary, this invention can improve the accuracy and generalization ability of X-Ray image defect segmentation, while taking into account the lightweight nature and real-time performance of the model. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the steps of an X-Ray image defect segmentation method based on enhanced guidance provided in an embodiment of the present invention.

[0051] Figure 2 This is a diagram of the defect segmentation model architecture provided in an embodiment of the present invention;

[0052] Figure 3 This is a binary image of defect segmentation provided in an embodiment of the present invention;

[0053] Figure 4 These are the original image and the binary overlay image provided in the embodiments of the present invention;

[0054] Figure 5 This is a structural block diagram of an X-Ray image defect segmentation device based on enhanced guidance provided in an embodiment of the present invention;

[0055] Figure 6This is a structural block diagram of an X-Ray image defect segmentation system based on enhanced guidance provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] It should be noted that although the device diagram shows a modular division and the flowchart illustrates a logical order, in some cases, the steps shown or described may be performed in a different order than the modular division in the device or the order shown in the flowchart. The terms "first," "second," etc., used in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0059] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0060] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0061] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0062] Defect segmentation methods in related technologies can be mainly classified into the following categories:

[0063] 1) Traditional image processing methods. Methods such as thresholding, edge detection, connected component analysis, morphological operations, filtering, and background correction can achieve certain results in scenes with regular backgrounds and stable lighting. However, in X-ray scenes, these methods are extremely sensitive to grayscale fluctuations and noise. The selection of thresholds and structuring elements relies heavily on experience and line-by-line parameter tuning, resulting in insufficient robustness. When the defect boundary is close to the background grayscale, or when there are periodic textures and roll embossing patterns, false negatives and missed detections are likely to occur, and the generalization ability is limited.

[0064] 2) Deep Learning Semantic / Instance Segmentation Methods. Networks such as FCN, U-Net, DeepLabv3+, PSPNet, and MaskR-CNN are relatively mature in natural scene segmentation. However, they often face some problems in industrial defect segmentation: ① High annotation cost. Large-scale, pixel-level high-quality masks are required, and it is difficult to collect a large number of qualified samples in a short time in industrial applications. ② Imbalanced scale between classes. Fine-tuning defects is rare and the deformation is large, making the network easily dominated by the background. ③ Domain shift and equipment differences. Differences in imaging styles between different production lines and batches cause the model's performance to degrade when crossing domains. ④ Resource constraints. Production lines have high real-time requirements, and the computing power and memory limitations at the deployment end are strict, making it difficult to directly deploy complex models online.

[0065] The emergence of basic visual models has brought new pathways to few-sample segmentation. Large-scale visual models are trained on tens of millions of images, covering a wide range of types and exhibiting strong generalization capabilities. However, directly applying the results of large-scale model pre-training to X-Ray battery electrode scenes still faces the following challenges: 1) Significant domain differences. X-Ray grayscale distribution differs greatly from texture statistics and natural images, potentially leading to missegmentation or incomplete segmentation at zero-sample mask boundaries. 2) Weak texture / low signal-to-noise ratio: Low contrast between micro-defects and the background, resulting in insufficient fine-grained reproduction by general-purpose decoders. 3) Engineering constraints. Full model fine-tuning requires a large number of parameters, high GPU memory and computing power, making iteration and deployment difficult on limited hardware.

[0066] Based on this, the present invention provides an enhancement-guided X-Ray image defect segmentation method and related equipment, which performs defect segmentation on the original X-Ray image by constructing an image segmentation network based on the Transformer architecture and training it based on the pre-trained weights of a large visual model.

[0067] like Figure 1 As shown, Figure 1 An enhanced guided X-ray image defect segmentation method is provided for embodiments of the present invention, the method comprising the following steps:

[0068] S100: Obtain the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder, and an image decoder.

[0069] S200, the X-Ray image to be segmented is input into the defect segmentation model, and the image encoder extracts features from the X-Ray image to be segmented to obtain an image embedding; the cue encoder generates a cue embedding; the image decoder performs bidirectional interactive attention calculation on the image embedding and the cue embedding to obtain query features and image features, and the unnormalized score and quality score of the mask are obtained based on the query features and image features;

[0070] S300, a target defect mask is generated based on the unnormalized score and the quality score of the mask, which serves as the defect segmentation result of the X-Ray image to be segmented.

[0071] This invention proposes an enhancement-guided X-ray image defect segmentation method and related equipment. By designing an enhancement-guided defect segmentation model, an image encoder, a cue encoder, an image decoder, and a dynamic masking device are organically combined to achieve accurate defect segmentation of low-light, detail-blurred X-ray polarimetric images. Specifically, the image encoder employs a ViTDeT architecture-based Transformer encoder, which effectively extracts multi-scale contextual information through periodic insertion of windowed attention and global attention mechanisms, overcoming the shortcomings of traditional convolutional networks in long-distance dependency modeling. The cue encoder innovatively integrates sparse and dense cues, encoding sparse information such as points and bounding boxes and dense information such as segmentation masks into a unified cue embedding, enhancing the model's ability to capture defect features. The image decoder promotes deep fusion of image embedding and cue embedding through bidirectional interactive attention computation, while the dynamic masking device generates high-quality defect masks based on query features and image features, significantly improving segmentation accuracy. Meanwhile, L1 channel pruning and knowledge distillation strategies were introduced during model training, which effectively reduced model complexity while ensuring segmentation performance. This met the real-time and resource constraints required by industrial production lines and provided strong technical support for the efficient detection of defects in lithium-ion battery electrode sheets.

[0072] The following section details an enhanced-guided X-Ray image defect segmentation method proposed in this invention, following the processing steps in engineering practice:

[0073] refer to Figure 2 First, a defect segmentation network (FOSNet) based on the Transformer architecture is constructed. The defect segmentation network consists of three parts: an image encoder, a cue encoder, and an image decoder.

[0074] In some embodiments, the step of extracting features from the X-Ray image to be segmented using the image encoder to obtain an image embedding includes:

[0075] S211, Obtain the X-Ray image to be segmented, and convert the X-Ray image to be segmented into a preprocessed image in tensor format; the X-Ray image to be segmented is a single-channel grayscale image;

[0076] S212 uses convolutional layers to reduce the dimensionality of the preprocessed image to obtain an initial feature map. The initial feature map is then input into a Transformer encoder based on the ViTDeT architecture. Deep features are extracted from the initial feature map through a windowed attention mechanism and a periodically inserted global attention mechanism, and a token map containing multi-scale contextual information is output.

[0077] S213, the token image is sequentially subjected to 1×1 convolution channel adjustment, LayerNorm2d activation function processing and 3×3 convolution feature optimization to obtain an image embedding containing batch dimension, channel dimension, height dimension and width dimension.

[0078] In this embodiment, the image encoder is based on the Transformer architecture. The original input image is transformed into a B×3×1024×1024 tensor after data preprocessing operations such as scaling and padding. A convolutional kernel of size 16 and stride 16 is used to convolve the image to obtain a 256×256 feature map, equivalent to 4096 patches, and the output feature map is mapped to ViT. ViT is based on the ViTDeT style, with most layers employing a windowed attention mechanism, and a few layers inserting a global attention mechanism to perform feature interaction across windows. The default window size (window_size) is 14, meaning the 64×64 token grid is divided into several 14×14 windows for attention. In the global attention mechanism, the attention in layers [2,5,8,11] is not windowed; it directly models the entire token map globally, ensuring long-distance dependencies. In terms of positional encoding, the attention mechanism incorporates relative positional encoding (RPE). The SAM model (SegmentAnything Model) uses decompositional relative position (decomposing along the height and width axes, borrowing from MViTV2), dynamically matching sizes during window or global computation. The ViT design allows most layers to compute attention within a small 14×14 window, effectively reducing computational complexity. Global feature information is captured through periodic insertion of a global attention mechanism. The B×64×64×768 token image output by ViT is processed with a 1×1 convolution to reduce the number of channels to 256, then further enhanced with a LayerNorm2d activation function, and finally further extracted using a 3×3 convolution. Simultaneously, the tensor is transformed from B×H×W×C to B×C×H×W, resulting in a B×256×64×64 image embedding, which also serves as the base image feature for the subsequent cue decoder.

[0079] In some embodiments, generating the cue embedding via the cue encoder includes:

[0080] S221, obtain sparse and dense cues, where sparse cues include point cues and bounding box cues, and dense cues are segmentation masks;

[0081] S222: Encode the sparse cue by mapping the 2D coordinates of the point cue to a position encoding vector of a preset dimension through random Fourier feature mapping. For the bounding box cue, embed the corresponding corner points by superimposing them separately to obtain the sparse cue embedding. Encode the dense cue by downsampling the segmentation mask to the same resolution as the image embedding through a convolutional layer, then adjusting the number of channels through convolutional mapping, and combining it with the dense position encoding generated by the random Fourier mechanism to obtain the dense cue embedding.

[0082] S223, sparse cue embedding and dense cue embedding are fused to generate cue embedding.

[0083] In this embodiment, the cue encoder encodes dense cues such as points and prior masks into a representation aligned with the image embedding for use by the decoder. Specifically, the cue encoder comprises two parts: a sparse branch and a dense branch.

[0084] The details of the sparse branch are as follows:

[0085] 1) Coordinate Position Encoding. 2D coordinates are mapped to 256 dimensions using random Fourier feature mapping to form a high-frequency sensitive position representation. The position calculation formula is shown below:

[0086] ;

[0087] Where p are normalized coordinates, and A is a Gaussian random matrix that is fixed during training. It is a 256-dimensional position encoding vector.

[0088] 2) Box type embedding. Boxes are treated as "two corner points", and two types of corner point embeddings of the bounding box are superimposed respectively, so as to prompt the encoder to know the semantics of the points.

[0089] The details of the dense branches are as follows:

[0090] 1) The details of the coordinate-dense location encoding are as follows: 1) Mask downsampling. The input mask of B×1×256×256 is downsampled to B×1×64×64 through two convolutional layers, and then the channels are mapped to 256 using 1×1 convolution. The resulting feature map has a shape of B×256×64×64.

[0091] 2) Dense position encoding.

[0092] Dense location codes (dense_pe) are generated for 64×64 grid locations using the same random Fourier mechanism: 1×256×64×64 (for the decoder to add location information to dense tokens).

[0093] Finally, dense prompt embeddings (B×256×64×64) with the same resolution as the image embedding are obtained, which are used to fuse with image feature elements in the decoder.

[0094] In some embodiments, the image decoder includes a bidirectional Transformer block and a dynamic masking block; the step of performing bidirectional interactive attention calculation on the image embedding and cue embedding through the image decoder to obtain query features and image features, and obtaining the unnormalized score and quality score of the mask based on the query features and image features, includes:

[0095] S231, input the image embedding and the prompt embedding into the bidirectional Transformer block, perform self-attention calculation on the query vector, establish the association between different point queries, box mask queries, IoU marker queries and mask marker queries, and obtain the query vector after self-attention mechanism processing;

[0096] S232, using the image markers with added image location encoding as keys and values, performs cross-attention calculation on the image markers through the query vector, so that the query vector focuses on the key regional features in the image markers, generates the cross-attention output of the queried image, and inputs the output into the feedforward neural network of the query vector after layer normalization to enhance the nonlinear expressive ability and update the query vector;

[0097] S233, using the updated query vector as the key and value, performs cross-attention calculation on the query vector through image labels, so that the image labels are reweighted according to the semantic information of the query vector, strengthens the expression of the region related to the prompt in the dense features, generates the cross-attention output from image to query, and inputs the output into the feedforward neural network of the image after layer normalization, and outputs the updated image labels.

[0098] S234, by stacking multiple bidirectional Transformer blocks, the query vector and image token are updated in multiple rounds to obtain the final query token and image token, which are used as the query feature and image feature input to the dynamic mask. The query token contains K mask tokens and 1 IoU token.

[0099] S235, the image token is reshaped into a feature map, and after being added element-wise to the dense cue embedding, it is upsampled to a preset resolution to obtain feature map F1;

[0100] S236, each mask token is processed by a multilayer perceptron to obtain feature map F2, and the inner product of the channel feature corresponding to F1 and F2 is performed at each pixel position to generate unnormalized mask score; and the IoU mark is processed by a multilayer perceptron to output the quality score of the mask.

[0101] In this embodiment, the image decoder is also based on the Transformer architecture. It has several core parameters: image embeddings (from the image encoder, with a shape of B×256×64×64), image position encoding (image_pe, used to add position information to dense tokens, with a shape of B×256×64×64), sparse prompt embeddings (point and box prompts), and dense prompt embeddings (dense prompt embeddings, projected dense features).

[0102] The image decoder includes a bidirectional Transformer block and a dynamic masking module. The bidirectional Transformer block includes a self-attention mechanism for the query vector, a cross-attention mechanism from the query to the image, a feedforward neural network for the query vector, a cross-attention mechanism from the image to the query, and a feedforward neural network for the image. The dynamic masking module includes a feature map processing module, a mask generation module, and a quality scoring module.

[0103] 1) Bidirectional Interactive Attention Stacking Layer: This layer allows sparse queries to interact with dense image tokens. The sparse query (hint token) focuses on the dense image token (cross-attention between the query and the image), while the dense image token looks back at the sparse query (cross-attention between the image and the query). This bidirectional interactive attention not only injects "hints" into the context of the feature map but also allows the image context to correct the scope of attention for the hints. Furthermore, self-attention is not applied to the 64×64 image patches to reduce computational complexity.

[0104] 2) Dynamic Masking: The dense features after passing through the Transformer are upsampled to 256×256. Then, each mask token is passed through an MLP network to generate a set of dynamic convolutional weights, which are used to linearly combine the upsampled features pixel by pixel, resulting in the output unnormalized score (logits) of the mask. On the other hand, there is also an IoU token (iou_tokens) that directly outputs the quality score of the mask through the MLP, which is used to select the best mask during inference.

[0105] The specific details of the bidirectional Transformer block are as follows: 1) Self-attention mechanism for query vectors (Queries). This connects different point / box mask queries with IoU tokens / mask token queries, unifying the prompt words. 2) Cross-attention to the query image. Image location encoding is used to add positional information to image tokens, which then serve as keys K / values ​​V, enabling full-image queries. 3) Feedforward neural network (FFN) for query vectors. This enhances the non-linearity of the network and improves the extraction of semantic information. 4) Cross-attention from image to query. This allows image tokens to be reweighted, enhancing the expressive power of dense features through the attention mechanism. 5) Feedforward neural network for images. This enriches the expressive power of features by adding the feature maps normalized by one layer and those processed by a multilayer perceptron (MLP).

[0106] The updated query token and image token are obtained by stacking multiple bidirectional Transformer blocks. The query token contains the most critical K mask tokens and 1 IoU token, and the image token is used to generate the final mask.

[0107] The specific details of the dynamic masking process are as follows: 1) The feature map after passing through the bidirectional Transformer block is reshaped to B×C×64×64, added element-wise with the dense cue embedding, and then the added feature map is upsampled to B×C×256×256 to obtain feature map F1. 2) For each mask token, a multilayer perceptron (MLP) is passed to obtain feature map F2. At each pixel position (i,j), F1[:,:,i,j] and F2 are multiplied together to obtain the unnormalized score (logits value) of that pixel, and finally a B×1×256×256 mask is output; F1[:,:,i,j] represents the vector obtained by slicing all channel values ​​at the i-th row and j-th column pixel position of feature map F1 in the channel dimension, with a dimension of C×1. The dimension of feature map F2 is also C×1. The channel dimension is compressed to 1 by the inner product operation (i.e., summing corresponding elements), thus obtaining the unnormalized score of a single pixel. 3) Output the IoU tag to the MLP layer and output a quality score with a scale of B×K. During inference, it can be used to select the optimal mask or return multiple candidates in parallel. For convenience, this invention directly returns the candidate with the highest score.

[0108] In some embodiments, the prompt encoder is trained in the following manner:

[0109] S201, expand the X-Ray image to be segmented into a 3-channel image and perform normalization processing, scale the normalized image, place the scaled image in the upper left corner and fill the lower right corner with pixel 0, then normalize the pixels of the filled image, and convert the pixel-normalized image into tensor format to obtain the preprocessed image.

[0110] S202, the preprocessed images are input into the prompt encoder one by one, and the coordinate information of the bounding boxes is stored in a dictionary, where the key of the dictionary is the name of the image after removing the suffix; wherein, for multiple masks in the preprocessed image, the corresponding multi-boxes are converted into single boxes and then sent to the prompt encoder, the loss of each single box is calculated and accumulated; the loss of the single box is obtained by jointly calculating the complete intersection-union loss and the binary cross-entropy loss.

[0111] S203: After training all single frames of a single image, backpropagation is performed to update the model parameters and save the optimal model parameters based on the average intersection-union ratio on the validation set.

[0112] S204, create a tensor identical to the predicted mask, assign 1 to the coordinate region within the box, find the intersection with the real mask to obtain the real mask region, and use the binary cross-entropy loss and Focal loss to jointly calculate the first total loss function.

[0113] S205, during training, the model parameters are adjusted to minimize the first total loss function, resulting in a fine-tuned cue encoder.

[0114] In this embodiment, the fine-tuning training process of the prompt encoder based on bounding box prompts is as follows: the original 16-bit single-channel grayscale image is expanded to 3 channels and normalized to [0, 255]. Then, the normalized image is scaled. Unlike center-filling, SAM places the image in the upper left corner and fills the lower right corner with pixels 0. Pixel normalization is then performed with a mean of [123.675, 116.28, 103.53] and a variance of [58.395, 57.12, 57.375]. This is then converted into a B×3×1024×1024 tensor and input into the network. During training, the bounding box coordinates are stored in a dictionary, where the key is the image name after removing its suffix. Images are fed into the network one by one. Since an image may contain multiple masks, to ensure the prompt encoder outputs the correct result, multiple boxes are converted into single boxes before being fed into the prompt encoder. The loss for each single box is calculated and accumulated. When calculating the loss function, a tensor identical to the predicted mask (pred_masks, the decoder's output prediction with its size restored to the original image size) is created, and the coordinates of the bounding boxes are assigned a value of 1. The intersection of this tensor with the ground truth mask is used to obtain the ground truth mask region. Then, the loss is calculated jointly using binary cross-entropy loss (BCELoss) and Focal loss (FocalLoss). After training all bounding boxes for a single image, backpropagation is performed to update the parameters. Simultaneously, the best model weights are saved based on the average intersection-union ratio (miou) calculated on the validation set for later testing.

[0115] The loss function is the core driving force of neural network training. It quantifies the difference between the model's prediction and the true label, providing a clear optimization objective for the optimization algorithm. Different vision tasks use different loss functions. For example, object detection tasks often use Complete Intersection over Union (CIoULoss) and Binary Cross-Entropy (BCELoss) as bounding box regression and classification losses. SAM is a cue-driven class-independent binary segmentation; therefore, during fine-tuning, the binary cross-entropy loss function and the FocalLoss loss function are jointly optimized. The first total loss function... The calculation formula is as follows:

[0116] ;

[0117] ;

[0118] ;

[0119] In the formula, For the first total loss function, The binary cross-entropy loss function is... FocalLoss loss function To control the proportional weight of the two loss functions, it is set to 0.5 during training. y is the true label, and p is the predicted label. The class balance coefficient is set to 0.75 during training. The modulation factor suppresses negative samples, amplifies positive samples, and increases the contribution of positive samples to the total loss. It is set to 5 during training. Predict probabilities for categories. The binary cross-entropy loss is used for the true label and the predicted label. FocalLoss loss is used to predict probabilities for each category.

[0120] In some embodiments, the defect segmentation model is trained in the following manner:

[0121] S110, Obtain the training dataset containing X-Ray images, segmentation masks, and bounding box cues;

[0122] S120, input the training dataset into the enhanced guided defect segmentation model, freeze the image encoder and cue encoder, train only the decoder part, and save the optimal model weights on the validation set according to the average intersection-over-union evaluation index to obtain the trained model;

[0123] S130, the trained model is pruned using the L1 channel pruning strategy, the pruned model is fine-tuned, and the optimal model weights are saved based on the average intersection-over-union ratio evaluation index on the validation set to obtain the pruned and fine-tuned model.

[0124] Specifically, S130 includes:

[0125] S131, calculate the L1 norm of the kernel weights corresponding to the output channels of each convolutional layer of the model after training, form the L1 norm set of each convolutional kernel, and mark the channels with L1 norm less than the pruning threshold as channels to be deleted.

[0126] S132, perform pruning operation, delete the marked channels to be deleted, and simultaneously delete the input channels corresponding to the pruned channels in the next layer to obtain the pruned model;

[0127] S133: Load the dataset and set the training parameters. Load the pruned model weights for fine-tuning training. Optimize the model parameters through forward propagation, loss function calculation, and backpropagation. Save the optimal model weights during the fine-tuning training process based on the average intersection-union ratio (IU) evaluation metric on the validation set to obtain the pruned and fine-tuned model.

[0128] S140 uses the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model for knowledge distillation. It supervises the differences between the student model output, the teacher model output, and the true label. Based on the indicators on the validation set, it saves the optimal student model weights to obtain the trained defect segmentation model.

[0129] In this embodiment, considering factors such as computing power and training efficiency, the image encoder and cue encoder are frozen, and only the decoder part is trained. During the model training process, the optimal model weights are saved on the validation set according to the mean intersection over union (mIoU) evaluation index to obtain the trained model. The trained model is then pruned using an L1 channel pruning strategy to reduce the model size, reduce computing resource consumption, accelerate model inference speed, and obtain the pruned model.

[0130] Pruning techniques, by removing redundant weights or neurons from the network, simplify the complex original model into a small and efficient equivalent version, achieving significant compression of model size, improved inference speed, and effective reduction of energy consumption with minimal loss of accuracy. Before pruning, the model is trained to obtain the optimal model weights. Then, a pruning strategy is used to prune the original weights, saving the pruned weights. Fine-tuning training is then performed based on these pruned weights to obtain the final pruned model weights. In this invention, L1 channel pruning is used to prune the original model, using the convolutional layer weight tensors... For example, the pruning process is as follows:

[0131] 1) Obtain the weights of the 0th convolutional kernel .

[0132] 2) Calculate the L1 norm , These are the height, width, number of input channels, and number of output channels of the convolutional kernel, respectively. These are the specific values ​​of the convolution kernel weights in the corresponding dimensions. It is an L1 norm. This represents the set of L1 norms for each convolution kernel. A smaller L1 norm indicates that the overall weight amplitude of the filter is smaller and its contribution to the output is weaker, so it will be preferentially pruned.

[0133] 3) Set a pruning threshold Delete The passage.

[0134] 4) Perform pruning and delete the corresponding input channel at the next level.

[0135] In some embodiments, the process of knowledge distillation using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model, monitoring the differences between the student model output and the teacher model output and the true labels, and saving the optimal student model weights based on the metrics on the validation set to obtain a trained defect segmentation model includes:

[0136] S141, the original defect segmentation model is used as the teacher model, the pruned and fine-tuned model is used as the student model, and the parameters of the teacher model are frozen so that it only provides forward output;

[0137] S142, input the training dataset into the student model, perform forward computation through the student model to obtain the first mask unnormalized score and the first mask quality score output by the student model; and input the same training data into the teacher model to obtain the second mask unnormalized score and the second mask quality score output by the teacher model, and use them as soft labels;

[0138] S143, calculate the KL divergence between the unnormalized score of the first mask and the unnormalized score of the second mask as the distillation loss, calculate the binary cross-entropy loss between the unnormalized score of the first mask and the true label mask as the classification loss, and weight the distillation loss and the classification loss to obtain the second total loss function.

[0139] S144, based on the second total loss function, the parameters of the student model are updated through backpropagation. During the training process, the average intersection-union ratio (IUR) on the validation set is calculated at each preset number of iterations. When the iteration training reaches the preset condition, the training is stopped, and the student model weights when the average IUR on the validation set is optimal are saved to obtain the trained defect segmentation model. The preset condition is that the number of iterations reaches the preset number of rounds or the index does not improve within multiple consecutive iterations.

[0140] In this embodiment, the pruned model is fine-tuned, and the optimal model weights during the fine-tuning process are saved according to the evaluation index to obtain the pruned and fine-tuned model.

[0141] The fine-tuning process is as follows: 1) Load the dataset and set the training parameters. 2) Load the pruned model weights and train the model, going through forward propagation, loss function calculation, backpropagation, etc. 3) Retain the optimal model weights based on the metrics on the validation set.

[0142] Next, the original defect segmentation network is used as the teacher model, and the pruned and fine-tuned model is used as the student model for distillation. The rich knowledge of the original teacher model is utilized to guide and restore the performance of the pruned student model, thus achieving efficient compression without sacrificing accuracy.

[0143] The necessity of distillation lies in the fact that fine-tuning after model pruning aims to restore accuracy, but conventional fine-tuning has limitations: the capacity of the pruned model has been reduced, like a "primary school student," and it is difficult to reach the performance ceiling of the original "university professor" level model simply by relearning training data. At this point, the necessity of knowledge distillation becomes apparent. Its core idea is to allow the pruned small model (student model) to no longer simply learn the original hard labels, but to directly imitate the output logic of the retained, unpruned large model (teacher model). The "soft labels" provided by the teacher model are rich in richer implicit knowledge, such as inter-class similarity, which can provide better guidance signals for the student model. Through this "mentor-apprentice" mechanism, the student model can learn the core decision logic of the teacher model more efficiently within its limited expressive capacity, thereby restoring accuracy and even achieving generalization ability that surpasses conventional fine-tuning, ultimately ensuring that the lightweight model is both fast and good. The distillation process is as follows: 1) Load the teacher model (original model) and the student model (pruned model). 2) Freeze the teacher model and obtain its forward output; obtain the student model's forward output, and perform normal gradient backpropagation. 3) Supervise the differences between the teacher's output, student's output, and the student's output and the true label. Specifically, use a second loss function for constraints during distillation. The calculation formula is:

[0144] ;

[0145] ;

[0146] in, This is the second total loss function. Distillation loss is used to measure the difference between student and teacher outputs. Softmax is the activation function, || denotes branch merging, and KL() represents KL divergence (Kullback-Leibler Divergence). The output provided to teachers was not normalized. The output for students is also not normalized; T represents the distillation temperature.

[0147] 4) Repeatedly iterate and save the optimal student model weights based on the indicators on the validation set.

[0148] In this embodiment, the optimal model weight during the distillation process is saved according to the evaluation index, and the image of the foreign object to be segmented is tested according to the obtained optimal model weight to obtain the segmentation result.

[0149] refer to Figure 3 and Figure 4The trained algorithm model is deployed into a hardware inspection device to achieve online real-time defect segmentation and detection of battery electrodes. The specific process is as follows: The workflow begins with the image acquisition module, which uses a high-precision X-ray imaging system to perform non-destructive scanning imaging of the battery electrodes on the production line, acquiring detailed, high-resolution X-ray images to provide a high-quality data foundation for subsequent analysis. The image data is then transmitted to the core of the system—the computing control unit. This unit incorporates a high-performance GPU computing card or an embedded device designed for edge computing (such as the NVIDIA Jetson AGX Orin), possessing powerful parallel computing capabilities. In this unit, the finely tuned and trained SAM model is deployed and optimized for operation. Accelerated by inference engines such as Tensor RT or ONNXRuntime, it efficiently performs a series of computationally intensive tasks, including image preprocessing, model forward inference, and result post-processing, achieving real-time defect identification and segmentation. To further ensure the accuracy and flexibility of the detection, the device is equipped with an interactive and prompting interface, integrating a high-sensitivity touchscreen display. When the automatic inspection results are uncertain or require manual verification, operators can directly review suspicious areas on the screen and provide point or box prompts (boundary box prompts) through intuitive touch controls, triggering the model to perform manual-assisted fine-tuning and segmentation, forming an efficient human-machine collaborative inspection closed loop. The final defect analysis results, including the precise location, morphological characteristics, and quantifiable size of the defects, are output and processed by the execution mechanism. This mechanism is deeply integrated with the existing PLC control system of the production line, not only uploading the inspection results to the Manufacturing Execution System (MES) in real time for quality traceability and analysis, but also driving sorting mechanisms (such as high-precision robotic arms or pneumatic push rods) to perform physical actions, realizing the automatic removal and sorting of defective electrode sheets, thus completing a fully automated process from "inspection-judgment-execution".

[0150] Furthermore, the system's network and storage units ensure high-speed and stable data transmission via an industrial gigabit Ethernet interface, and have built-in high-capacity solid-state drives (SSDs) reliably storing all inspection images, defect records, and model parameter files, providing solid data support for process optimization and quality big data analysis. The entire device seamlessly integrates algorithms, hardware, and industrial processes, ultimately achieving online, real-time, intelligent, and high-precision non-destructive testing and automated processing of battery electrode defects.

[0151] See Figure 5 This invention provides an X-Ray image defect segmentation device based on enhanced guidance, comprising:

[0152] The first module is used to acquire the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder and an image decoder.

[0153] The second module is used to input the X-Ray image to be segmented into the defect segmentation model, extract features from the X-Ray image to be segmented by the image encoder to obtain an image embedding, generate a cue embedding by the cue encoder, and perform bidirectional interactive attention calculation on the image embedding and the cue embedding by the image decoder to obtain query features and image features, and obtain the unnormalized score and quality score of the mask based on the query features and image features.

[0154] The third module is used to generate a target defect mask based on the unnormalized score and the quality score of the mask, as the defect segmentation result of the X-Ray image to be segmented.

[0155] It is evident that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented in this device embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0156] See Figure 6 This invention provides an enhanced guided X-Ray image defect segmentation system, comprising:

[0157] At least one processor;

[0158] At least one memory for storing at least one program;

[0159] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0160] It is evident that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0161] Furthermore, embodiments of the present invention also disclose a computer program product or computer program stored in a computer-readable storage medium. A processor of a computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the described method. Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0164] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0165] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0166] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A defect segmentation method for X-Ray images based on enhancement guidance, characterized in that, The method includes the following steps: Acquire the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder, and an image decoder. The X-Ray image to be segmented is input into the defect segmentation model. The image encoder extracts features from the X-Ray image to be segmented to obtain an image embedding. The cue encoder generates a cue embedding. The image decoder performs bidirectional interactive attention calculation on the image embedding and the cue embedding to obtain query features and image features. Based on the query features and image features, the unnormalized score and quality score of the mask are obtained. A target defect mask is generated based on the unnormalized score and quality score of the mask, and used as the defect segmentation result of the X-Ray image to be segmented. The image decoder includes a bidirectional Transformer block and a dynamic masking block; the bidirectional interactive attention calculation of the image embedding and the cue embedding through the image decoder to obtain query features and image features, and the unnormalized score and quality score of the mask based on the query features and image features, including: The image embedding and prompt embedding are input into a bidirectional Transformer block. After performing self-attention computation on the query vector to establish the association between different types of queries, the image tags with added image location encoding are used as keys. Cross-attention computation is performed on the image tags through the query vector, and the query vector is updated through layer normalization and a feedforward neural network of the query vector. The updated query vector is used as keys. Cross-attention computation is performed on the query vector through the image tags, and the image tags are updated through layer normalization and a feedforward neural network of the image. After stacking multiple bidirectional Transformer blocks, the output query token and image token are used as dynamic masking inputs for query features and image features. The query token contains K mask tokens and 1 IoU tag. The image token is reshaped into a feature map, and after being added element-wise to the dense cue embedding, it is upsampled to a preset resolution to obtain F1. Each mask token is processed by a multilayer perceptron to obtain F2. At the pixel position, the channel features corresponding to F1 are multiplied by F2 to generate the mask unnormalized score. At the same time, the IoU marker is processed by a multilayer perceptron to output the mask quality score. The defect segmentation model is trained in the following way: Obtain a training dataset containing X-Ray images, segmentation masks, and bounding box cues; The training dataset is input into the enhanced guided defect segmentation model, the image encoder and cue encoder are frozen, only the decoder part is trained, and the optimal model weights are saved on the validation set according to the average intersection-over-union ratio evaluation metric to obtain the trained model. The trained model is pruned using an L1 channel pruning strategy. The pruned model is then fine-tuned. The optimal model weights are saved based on the average intersection-over-union ratio (IoU) evaluation metric on the validation set, resulting in the pruned and fine-tuned model. Using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model, knowledge distillation is performed. The differences between the student model output, the teacher model output, and the true label are monitored. The optimal student model weights are saved based on the indicators on the validation set, resulting in a well-trained defect segmentation model.

2. The method according to claim 1, characterized in that, The step of extracting features from the X-Ray image to be segmented using the image encoder to obtain the image embedding includes: Obtain the X-Ray image to be segmented, and convert the X-Ray image to be segmented into a preprocessed image in tensor format; the X-Ray image to be segmented is a single-channel grayscale image; Convolutional layers are used to reduce the dimensionality of the preprocessed image to obtain an initial feature map. The initial feature map is then input into a Transformer encoder based on the ViTDeT architecture. Deep features are extracted from the initial feature map through a windowed attention mechanism and a periodically inserted global attention mechanism, and a token map containing multi-scale contextual information is output. The token image is sequentially subjected to 1×1 convolution channel adjustment, LayerNorm2d activation function processing, and 3×3 convolution feature optimization to obtain an image embedding containing batch dimension, channel dimension, height dimension, and width dimension.

3. The method according to claim 1, characterized in that, The step of generating the cue embedding through the cue encoder includes: Obtain sparse and dense tooltips, where sparse tooltips include point tooltips and bounding box tooltips, and dense tooltips are segmentation masks; The sparse cue is encoded by mapping the 2D coordinates of the point cue to a position encoding vector of a preset dimension through random Fourier feature mapping. The bounding box cue is embedded by superimposing the corresponding corner points on the two corner points respectively to obtain the sparse cue embedding. The dense cue is encoded by downsampling the segmentation mask to the same resolution as the image embedding through a convolutional layer, adjusting the number of channels through convolutional mapping, and then combining the dense position encoding generated by the random Fourier mechanism to obtain the dense cue embedding. The sparse cue embedding and the dense cue embedding are fused to generate a cue embedding.

4. The method according to claim 1, characterized in that, The prompt encoder was trained in the following way: The X-Ray image to be segmented is converted into a preprocessed image in tensor format and used as a training sample; The preprocessed images are input into the prompt encoder one by one, and the coordinate information of the bounding boxes is stored in a dictionary. The key of the dictionary is the name of the image after removing the suffix. For multiple masks in the preprocessed image, the corresponding multi-boxes are converted into single boxes and then sent to the prompt encoder. After training all single frames of a single image, backpropagation is performed. The loss of each single frame is calculated by combining the full intersection-union ratio loss and the binary cross-entropy loss and accumulated to obtain the loss value of the cue encoder. The model parameters are updated according to the loss value of the cue encoder and the optimal model parameters are saved according to the average intersection-union ratio on the validation set. Create a tensor identical to the predicted mask, assign 1 to the coordinate region within the box, and obtain the real mask region by finding the intersection with the real mask. The first total loss function is calculated by jointly using binary cross-entropy loss and Focal loss. During training, the model parameters are adjusted to minimize the first total loss function, resulting in a fine-tuned cue encoder.

5. The method according to claim 1, characterized in that, The process involves using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model for knowledge distillation. It monitors the differences between the student model output, the teacher model output, and the true labels. Based on the metrics on the validation set, it saves the optimal student model weights to obtain a trained defect segmentation model, including: The original defect segmentation model is used as the teacher model, and the pruned and fine-tuned model is used as the student model. The parameters of the teacher model are frozen so that it only provides forward output. The training dataset is input into the student model, and forward computation is performed through the student model to obtain the first mask unnormalized score and the first mask quality score output by the student model; the same training data is input into the teacher model to obtain the second mask unnormalized score and the second mask quality score output by the teacher model, which are used as soft labels. The KL divergence between the unnormalized score of the first mask and the unnormalized score of the second mask is calculated as the distillation loss, and the binary cross-entropy loss between the unnormalized score of the first mask and the true label mask is calculated as the classification loss. The distillation loss and the classification loss are weighted and summed to obtain the second total loss function. The parameters of the student model are updated through backpropagation based on the second total loss function. During the training process, the average intersection-union ratio (IUR) on the validation set is calculated at a preset number of iterations. When the iteration training reaches a preset condition, the training stops and the student model weights when the average IUR on the validation set is optimal are saved to obtain the trained defect segmentation model. The preset condition is that the number of iterations reaches a preset number of rounds or the index does not improve within a number of consecutive iterations.

6. A defect segmentation device for X-Ray images based on enhanced guidance, characterized in that, include: The first module is used to acquire the X-Ray image to be segmented and the trained defect segmentation model; The defect segmentation model includes an image encoder, a cue encoder, and an image decoder; The second module is used to input the X-Ray image to be segmented into the defect segmentation model, extract features from the X-Ray image to be segmented by the image encoder to obtain an image embedding, generate a cue embedding by the cue encoder, and perform bidirectional interactive attention calculation on the image embedding and the cue embedding by the image decoder to obtain query features and image features, and obtain the unnormalized score and quality score of the mask based on the query features and image features. The third module is used to generate a target defect mask based on the unnormalized score and the quality score of the mask, as the defect segmentation result of the X-Ray image to be segmented; The image decoder includes a bidirectional Transformer block and a dynamic masking block; the bidirectional interactive attention calculation of the image embedding and the cue embedding through the image decoder to obtain query features and image features, and the unnormalized score and quality score of the mask based on the query features and image features, including: The image embedding and prompt embedding are input into a bidirectional Transformer block. After performing self-attention computation on the query vector to establish the association between different types of queries, the image tags with added image location encoding are used as keys. Cross-attention computation is performed on the image tags through the query vector, and the query vector is updated through layer normalization and a feedforward neural network of the query vector. The updated query vector is used as keys. Cross-attention computation is performed on the query vector through the image tags, and the image tags are updated through layer normalization and a feedforward neural network of the image. After stacking multiple bidirectional Transformer blocks, the output query token and image token are used as dynamic masking inputs for query features and image features. The query token contains K mask tokens and 1 IoU tag. The image token is reshaped into a feature map, and after being added element-wise to the dense cue embedding, it is upsampled to a preset resolution to obtain F1. Each mask token is processed by a multilayer perceptron to obtain F2. At the pixel position, the channel features corresponding to F1 are multiplied by F2 to generate the mask unnormalized score. At the same time, the IoU marker is processed by a multilayer perceptron to output the mask quality score. The defect segmentation model is trained in the following way: Obtain a training dataset containing X-Ray images, segmentation masks, and bounding box cues; The training dataset is input into the enhanced guided defect segmentation model, the image encoder and cue encoder are frozen, only the decoder part is trained, and the optimal model weights are saved on the validation set according to the average intersection-over-union ratio evaluation metric to obtain the trained model. The trained model is pruned using an L1 channel pruning strategy. The pruned model is then fine-tuned. The optimal model weights are saved based on the average intersection-over-union ratio (IoU) evaluation metric on the validation set, resulting in the pruned and fine-tuned model. Using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model, knowledge distillation is performed. The differences between the student model output, the teacher model output, and the true label are monitored. The optimal student model weights are saved based on the indicators on the validation set, resulting in a well-trained defect segmentation model.

7. A defect segmentation system for X-Ray images based on enhancement guidance, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1 to 5.

Images