Enhanced guidance-based X-Ray image defect segmentation method and related equipment

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an enhanced guidance-based X-Ray image defect segmentation method and related equipment, and the method comprises the steps: inputting a to-be-segmented X-Ray image into a defect segmentation model, and carrying out the feature extraction of the to-be-segmented X-Ray image through an image encoder, and obtaining image embedding; generating a prompt embedding through the prompt encoder; performing bidirectional interactive attention calculation on the image embedding and the prompt embedding through the image decoder to obtain a query feature and an image feature, and obtaining a non-normalized score and a quality score of a mask based on the query feature and the image feature; generating a target defect mask based on the non-normalized score of the mask and the quality score of the mask, and taking the target defect mask as a defect segmentation result of the to-be-segmented X-Ray image; according to the method, the precision and generalization ability of X-Ray image defect segmentation can be improved, and the lightweight and real-time performance of the model are considered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to an X-Ray image defect segmentation method based on enhanced guidance and related equipment. BACKGROUND

[0002] With the rapid development of new energy vehicles and energy storage industries, the demand for lithium ion batteries has increased dramatically, and higher requirements have been placed on their manufacturing quality. As a core component of the battery, the surface defects (such as uneven coating, scratches, etc.) of the battery pole piece directly affect the electrochemical performance and safety performance of the battery. Currently, the industry generally uses X-Ray imaging technology for non-destructive testing of pole pieces to identify internal defects.

[0003] However, the defect segmentation method in the related art often has insufficient ability to capture complex defect features and limited segmentation accuracy when processing X-Ray image defect segmentation. In addition, there is a problem of slow segmentation speed, which cannot meet the requirements of industrial real-time detection scenarios. SUMMARY

[0004] Therefore, the embodiments of the present application provide an X-Ray image defect segmentation method based on enhanced guidance and related equipment to solve one or more technical problems in the prior art and provide at least one beneficial option or create conditions.

[0005] In one aspect, the embodiments of the present application provide an X-Ray image defect segmentation method based on enhanced guidance, which comprises the following steps: obtaining a to-be-segmented X-Ray image and a trained defect segmentation model; the defect segmentation model comprises an image encoder, a prompt encoder, and an image decoder; inputting the to-be-segmented X-Ray image into the defect segmentation model, extracting features of the to-be-segmented X-Ray image through the image encoder to obtain image embeddings; generating prompt embeddings through the prompt encoder; performing bidirectional interaction attention calculation on the image embeddings and the prompt embeddings through the image decoder to obtain query features and image features, and obtaining unnormalized scores and quality scores of masks based on the query features and the image features; generating a target defect mask based on the unnormalized scores of the masks and the quality scores of the masks as a defect segmentation result of the to-be-segmented X-Ray image.

[0006] Optionally, the feature extraction of the to-be-segmented X-Ray image through the image encoder to obtain the image embeddings comprises: An X-Ray image to be segmented is acquired, and the X-Ray image to be segmented is converted into a preprocessed image in a tensor format; the X-Ray image to be segmented is a single-channel grayscale image; A convolutional layer is used to perform feature dimension reduction on the preprocessed image to obtain an initial feature map, and the initial feature map is input into a Transformer encoder based on a ViTDeT architecture to perform deep feature extraction on the initial feature map through a windowed attention mechanism and a periodically inserted global attention mechanism, and output a token map containing multi-scale context information; The token map is sequentially subjected to 1x1 convolutional channel adjustment, LayerNorm2d activation function processing, and 3x3 convolutional feature optimization to obtain an image embedding containing batch dimension, channel dimension, height dimension, and width dimension.

[0007] Optionally, the generation of the prompt embedding by the prompt encoder comprises: Sparse prompts and dense prompts are acquired, wherein the sparse prompts comprise point prompts and bounding box prompts, and the dense prompts are segmentation masks; The sparse prompts are encoded, the 2D coordinates of the point prompts are mapped into position encoding vectors of a preset dimension through a random Fourier feature, and the bounding box prompts are respectively superimposed with corresponding corner embedding according to two corner points to obtain sparse prompt embedding; The dense prompts are encoded, the segmentation masks are down-sampled to the same resolution as the image embedding through a convolutional layer, and after adjusting the number of channels through convolutional mapping, the dense prompt embedding is obtained in combination with dense position encoding generated by a random Fourier mechanism; The sparse prompt embedding and the dense prompt embedding are fused to generate a prompt embedding.

[0008] Optionally, the image decoder comprises a bidirectional Transformer block and a dynamic mask head; the bidirectional interactive attention calculation of the image embedding and the prompt embedding by the image decoder obtains query features and image features, and the unnormalized scores and quality scores of the masks are obtained based on the query features and the image features, comprising: The image embedding and the prompt embedding are input into the bidirectional Transformer block, self-attention calculation is performed on the query vector to establish the association between different types of queries, the image marker with added image position encoding is taken as the key and value, cross-attention calculation is performed on the image marker by the query vector, and the query vector is updated through layer normalization and the feedforward neural network of the query vector; the updated query vector is taken as the key and value, cross-attention calculation is performed on the query vector by the image marker, and the image marker is updated through layer normalization and the feedforward neural network of the image; The stacked multiple bidirectional Transformer blocks output query tokens and image tokens as query features and image features to input a dynamic mask head, wherein the query tokens include K mask tokens and 1 IoU label; The image tokens are reshaped into a feature map, and after being element-wise added to the dense prompt embedding, are up-sampled to a preset resolution to obtain F1; each mask token is processed by a multi-layer perception to obtain F2; at a pixel position, F1 corresponding channel features and F2 are performed inner product to generate a mask non-normalized score, and the IoU label is processed by a multi-layer perception to output a mask quality score.

[0009] Optionally, the prompt encoder is trained by the following method: An X-Ray image to be segmented is converted into a preprocessed image in a tensor format, and is taken as a training sample; The preprocessed image is input into the prompt encoder, and coordinate information of the bounding box is stored in a dictionary, and a key value of the dictionary is a name of the image without a suffix; wherein, for multiple masks in the preprocessed image, the corresponding multiple boxes are converted into a single box and then input into the prompt encoder; After the training of all single boxes in a single image is completed, back propagation is performed, each single box loss is calculated by a complete intersection over union loss and a binary cross entropy loss, and is accumulated to obtain a loss value of the prompt encoder, model parameters are updated according to the loss value of the prompt encoder, and optimal model parameters are saved according to an average intersection over union on a validation set; A tensor same as a predicted mask is created, coordinates in a box are assigned as 1, and an intersection set of the predicted mask and a real mask is obtained to obtain a real mask region, and a first total loss function is calculated by a binary cross entropy loss and a focal loss; In the training process, the first total loss function is minimized by adjusting the model parameters to obtain a fine-tuned prompt encoder.

[0010] Optionally, the defect segmentation model is trained by the following method: A training data set containing an X-Ray image, a segmentation mask and a bounding box prompt is obtained; The training data set is input into the enhanced guided defect segmentation model, the image encoder and the prompt encoder are frozen, only the decoder part is trained, and optimal model weights are saved according to an average intersection over union evaluation index on a validation set to obtain a trained model; The trained model is pruned by an L1 channel pruning strategy, the pruned model is fine-tuned and trained, and optimal model weights are saved according to an average intersection over union evaluation index on a validation set to obtain a pruned and fine-tuned model; The original defect segmentation model is taken as a teacher model, a pruned and fine-tuned model is taken as a student model for knowledge distillation, differences between outputs of the student model and the teacher model and real labels are supervised, optimal student model weights are saved according to an index on a verification set, and a trained defect segmentation model is obtained.

[0011] Optionally, the original defect segmentation model is taken as a teacher model, a pruned and fine-tuned model is taken as a student model for knowledge distillation, differences between outputs of the student model and the teacher model and real labels are supervised, optimal student model weights are saved according to an index on a verification set, and a trained defect segmentation model is obtained, and the method comprises the following steps: The original defect segmentation model is taken as a teacher model, a pruned and fine-tuned model is taken as a student model for knowledge distillation, differences between outputs of the student model and the teacher model and real labels are supervised, optimal student model weights are saved according to an index on a verification set, and a trained defect segmentation model is obtained. The training data set is input into the student model, forward calculation is performed through the student model, first mask non-normalized scores and first mask quality scores output by the student model are obtained, and the same training data is input into the teacher model, second mask non-normalized scores and second mask quality scores output by the teacher model are obtained and used as soft labels. The KL divergence of the first mask non-normalized scores and the second mask non-normalized scores is calculated as a distillation loss, and the binary cross-entropy loss of the first mask non-normalized scores and real label masks is calculated as a classification loss, and the distillation loss and the classification loss are weighted and summed to obtain a second total loss function. The parameters of the student model are updated through back propagation based on the second total loss function, the average intersection over union index on the verification set is calculated every preset number of iterations during the training process, the training is stopped when a preset condition is met, and the student model weights at the time when the average intersection over union index on the verification set is optimal are saved, and a trained defect segmentation model is obtained.

[0012] In another aspect, an embodiment of the present application provides an X-Ray image defect segmentation device based on enhanced guidance, comprising: A first module is configured to obtain an X-Ray image to be segmented and a trained defect segmentation model; the defect segmentation model comprises an image encoder, a prompt encoder and an image decoder. A second module is configured to input the X-Ray image to be segmented into the defect segmentation model, perform feature extraction on the X-Ray image to be segmented through the image encoder to obtain image embedding, generate prompt embedding through the prompt encoder, and perform bidirectional interaction attention calculation on the image embedding and the prompt embedding through the image decoder to obtain query feature and image feature, and obtain non-normalized scores and quality scores of masks based on the query feature and the image feature. The third module is configured to generate a target defect mask based on the non-normalized score of the mask and the quality score of the mask, as a defect segmentation result of the X-Ray image to be segmented.

[0013] In another aspect, an embodiment of the present application provides an X-Ray image defect segmentation system based on enhanced guidance, comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0014] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores a processor executable program, and the processor executable program is used to execute the above method when executed by a processor.

[0015] Embodiments of the present application include the following beneficial effects: The present application designs an enhanced guidance defect segmentation model, organically combines an image encoder, a prompt encoder, an image decoder, and a dynamic mask head, and realizes accurate defect segmentation of low-illumination and detail-fuzzy X-Ray pole piece images. The image encoder adopts a Transformer encoder with a ViTDeT architecture, periodically inserts windowed attention and global attention mechanisms, can effectively extract multi-scale context information, and solves the deficiency of traditional convolutional networks in long-distance dependency modeling; the prompt encoder innovatively fuses sparse prompts and dense prompts, encodes sparse information such as points and boundary boxes and dense information such as segmentation masks into unified prompt embeddings, and enhances the model's ability to capture defect features; the image decoder promotes the deep fusion of image embeddings and prompt embeddings through bidirectional interactive attention calculation, and the dynamic mask head generates high-quality defect masks based on query features and image features, significantly improving the segmentation accuracy. At the same time, L1 channel pruning and knowledge distillation strategies are introduced in the model training process, effectively reducing the model complexity on the premise of ensuring the segmentation performance, meeting the real-time and resource constraints of industrial production lines, and providing strong technical support for efficient detection of lithium-ion battery pole piece defects. In summary, the present application can improve the precision and generalization ability of X-Ray image defect segmentation, and also considers the lightweight and real-time performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0017] Figure 1 is a step flow diagram of a defect segmentation method based on enhanced guidance of an X-Ray image provided by an embodiment of the present application; Figure 2 is an architecture diagram of a defect segmentation model provided by an embodiment of the present application; Figure 3 is a binary graph of defect segmentation provided by an embodiment of the present application; Figure 4 is an original graph and a binary superimposed graph provided by an embodiment of the present application; Figure 5 is a structural block diagram of a defect segmentation device based on enhanced guidance of an X-Ray image provided by an embodiment of the present application; Figure 6 is a structural block diagram of a defect segmentation system based on enhanced guidance of an X-Ray image provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0019] It should be noted that although the modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0021] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a sufficient understanding of the embodiments of the present application. However, one skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring the aspects of the present application.

[0022] The block diagrams shown in the drawings are merely functional entities and do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the drawings are merely exemplary illustrations and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0024] The defect segmentation methods in the related art mainly fall into the following categories: 1) Traditional image processing methods. Methods such as threshold segmentation, edge detection, connected component analysis, morphological operation, filtering and background correction can achieve certain results in regular background and stable lighting scenes. However, in X-Ray scenes, these methods are extremely sensitive to gray scale fluctuations and noise, and the selection of threshold and structure element depends on a large amount of experience and line-by-line parameter adjustment, which lacks robustness; when the defect boundary is close to the background gray scale, or there are periodic textures and roll marks, it is easy to miss detection and false detection, and the generalization ability is limited.

[0025] 2) Deep learning semantic / instance segmentation methods. Networks such as FCN, U-Net, DeepLabv3+, PSPNet, MaskR-CNN have been relatively mature in natural scene segmentation. However, in industrial defect segmentation, there are often some problems: ① high labeling cost. Large-scale, pixel-level high-quality masks are difficult to collect in a short time in industry ② class imbalance. Fine-tuning defects are small and have large deformations, and the network is easily dominated by the background. ③ domain shift and device difference. Different lines, different batches of imaging styles differ, causing the model to perform poorly when crossing domains. ④ resource constraints. The production line has high real-time requirements, and the deployment end has strict limitations on computing power and memory, making it difficult to directly deploy complex models.

[0026] With the emergence of basic vision models, a new path is opened up for small sample segmentation. Vision large models are trained on tens of millions of images, covering a wide range of categories and having strong generalization. However, directly using the pre-training results of large models in X-Ray battery tab scenarios still faces the following problems: 1) significant domain difference. The X-Ray gray scale distribution and texture statistics are very different from natural images, and the zero-sample mask boundary may be misclassified or not fully segmented. 2) weak texture / low signal-to-noise ratio: micro-defects have low contrast with the background, and the fine-grained restoration of general decoders is insufficient. 3) engineering constraints. Full model fine-tuning requires a large number of parameters, high memory and computing power, making it difficult to iterate and deploy on limited hardware.

[0027] Based on this, the application provides an X-Ray image defect segmentation method based on enhanced guidance and related equipment, which constructs an image segmentation network based on a Transformer architecture and trains it based on the pre-training weight of a visual large model to perform defect segmentation on an original X-Ray image.

[0028] As Figure 1 shown, Figure 1 An X-Ray image defect segmentation method based on enhanced guidance is provided for the embodiments of the application, and the method comprises the following steps: S100, obtaining an X-Ray image to be segmented and a trained defect segmentation model; the defect segmentation model comprises an image encoder, a prompt encoder and an image decoder; S200, inputting the X-Ray image to be segmented into the defect segmentation model, extracting features of the X-Ray image to be segmented by the image encoder to obtain image embedding; generating prompt embedding by the prompt encoder; performing bidirectional interactive attention calculation on the image embedding and the prompt embedding by the image decoder to obtain query features and image features, and obtaining unnormalized scores and quality scores of masks based on the query features and the image features; S300, generating a target defect mask based on the unnormalized scores of the masks and the quality scores of the masks as the defect segmentation result of the X-Ray image to be segmented.

[0029] The application provides an X-Ray image defect segmentation method based on enhanced guidance and related equipment. By designing an enhanced guidance defect segmentation model, the image encoder, the prompt encoder, the image decoder and the dynamic mask head are organically combined to realize accurate defect segmentation of X-Ray sheet image with low illumination and fuzzy details. The image encoder adopts a Transformer encoder with a ViTDeT architecture, which can effectively extract multi-scale context information through periodic insertion of windowed attention and global attention mechanisms, solving the shortcomings of traditional convolutional networks in long-distance dependency modeling. The prompt encoder innovatively combines sparse prompts and dense prompts, encoding sparse information such as points and bounding boxes and dense information such as segmentation masks into unified prompt embedding, enhancing the model's ability to capture defect features. The image decoder promotes the deep fusion of image embedding and prompt embedding through bidirectional interactive attention calculation, and the dynamic mask head generates high-quality defect masks based on query features and image features, significantly improving the segmentation accuracy. At the same time, L1 channel pruning and knowledge distillation strategies are introduced during model training, effectively reducing the model complexity while ensuring the segmentation performance, meeting the real-time and resource constraints of industrial production lines, and providing strong technical support for efficient detection of lithium-ion battery sheet defects.

[0030] The application will be described in detail below according to the processing steps in engineering practice: Reference Figure 2 First, a defect segmentation network (FOSNet) based on a Transformer architecture is constructed, and the defect segmentation network is composed of an image encoder, a prompt encoder and an image decoder; In some embodiments, the feature extraction of the X-Ray image to be segmented by the image encoder includes: S211, obtaining an X-Ray image to be segmented, and converting the X-Ray image to be segmented into a preprocessed image in tensor format; the X-Ray image to be segmented is a single-channel grayscale image; S212, performing feature dimension reduction on the preprocessed image by using a convolutional layer to obtain an initial feature map, inputting the initial feature map into a Transformer encoder based on a ViTDeT architecture, and performing deep feature extraction on the initial feature map by using a windowed attention mechanism and a periodically inserted global attention mechanism to output a token map containing multi-scale context information; S213, sequentially performing 1x1 convolutional channel adjustment, LayerNorm2d activation function processing and 3x3 convolutional feature optimization on the token map to obtain an image embedding containing batch dimension, channel dimension, height dimension and width dimension.

[0031] In this embodiment, the image encoder is based on the Transformer architecture. The original input image is converted into a tensor of Bx3x1024x1024 after data preprocessing operations such as scaling and padding. The image is convolved using a convolution kernel with a size of 16 and a step of 16 to obtain a feature map of 256x256, which is equivalent to 4096 patches. The output feature map is mapped into ViT. ViT is based on the ViTDeT style. Most layers use windowed attention, and individual layers insert global attention to interact features across windows. The window size (window_size) is set to 14 by default, which means that the 64x64 token grid is divided into several 14x14 windows for attention. In the global attention mechanism, the attention of layers [2, 5, 8, 11] is not divided into windows, but directly models the global token map to ensure long-distance dependence. In terms of position encoding, relative position encoding (RPE) is added to the attention mechanism. The SAM model (SegmentAnything Model) uses a decomposed relative position (which is decomposed in the high / width axis, inspired by MViTV2) to dynamically match the size during window or global calculation. The design of ViT allows most layers to calculate attention in a small window of 14x14, effectively reducing computational complexity. Global feature information is captured by periodically inserting global attention mechanisms. A 1x1 convolution is used to change the channel number of the Bx64x64x768 token map output by ViT to 256. The LayerNorm2d activation function is used to increase nonlinearity. Finally, a 3x3 convolution is used to further extract feature information, and the tensor is changed from BxHxWxC to BCxHxW, obtaining a Bx256x64x64 image embedding, which is also the bottom feature of the subsequent prompt decoder.

[0032] In some embodiments, the generating, by the prompt encoder, a prompt embedding comprises: S221, obtaining sparse prompts and dense prompts, wherein the sparse prompts include point prompts and bounding box prompts, and the dense prompts are segmentation masks; S222, encoding the sparse prompts to obtain sparse prompt embeddings, wherein the 2D coordinates of the point prompts are mapped to position encoding vectors of a preset dimension by using random Fourier features, and the bounding box prompts are respectively superimposed with corresponding corner embedding according to two corner points; and encoding the dense prompts to obtain dense prompt embeddings, wherein the segmentation masks are down-sampled to the same resolution as the image embedding by using a convolution layer, and then the channel number is adjusted by using a convolution mapping, and finally combined with the dense position encoding generated by the random Fourier mechanism. S223, fusing the sparse prompt embedding and the dense prompt embedding to generate a prompt embedding.

[0033] In this embodiment, the prompt encoder uniformly encodes the dense prompts such as points and prior masks into representations aligned with the image embedding for use by the decoder. Specifically, the prompt encoder includes two parts, a sparse branch and a dense branch.

[0034] The sparse branch details are as follows: 1) Coordinate position encoding. Map 2D coordinates to 256 dimensions through random Fourier features to form a high-frequency sensitive position representation. The position calculation formula is as follows: ; where p is the normalized coordinate, A is a Gaussian random matrix fixed in training, is a 256-dimensional position encoding vector.

[0035] 2) Box type embedding. The box is processed according to the "two corner points", and the two corner point embeddings of the bounding box are superimposed respectively, so that the prompt encoder knows the semantics of the point.

[0036] The dense branch details are as follows: 1) Coordinate dense position encoding details are as follows: 1) Mask down-sampling. The input mask of Bx1x256x256 is down-sampled to Bx1x64x64 through two layers of convolution, and then a 1x1 convolution is used to map the channel to 256. The feature map of Bx256x64x64 is obtained.

[0037] 2) Dense position encoding.

[0038] Generate dense position encoding (dense_pe) for 64x64 grid positions through the same random Fourier mechanism: 1x256x64x64 (for the decoder to add position information on the dense tokens).

[0039] Finally, the dense prompt embedding (dense_prompt_embeddings) with the same resolution as the image embedding is obtained: Bx256x64x64, which is used to fuse with the image feature elements in the decoder.

[0040] In some embodiments, the image decoder includes a bidirectional Transformer block and a dynamic mask head; the bidirectional interaction attention calculation of the image embedding and the prompt embedding through the image decoder obtains query features and image features, and the unnormalized scores and quality scores of the mask are obtained based on the query features and image features, including: S231, input the image embedding and the prompt embedding into the bidirectional Transformer block, perform self-attention calculation on the query vector, associate different point queries, frame mask queries, IoU label queries, and mask label queries, and obtain the query vector after self-attention mechanism processing; S232, take the image label after adding the image position encoding as the key and the value, perform cross-attention calculation on the image label by the query vector, make the query vector pay attention to the key region features in the image label, generate the cross-attention output from the query to the image, input the output after layer normalization into the feedforward neural network of the query vector, enhance the non-linear expression ability and update the query vector; S233, take the updated query vector as the key and the value, perform cross-attention calculation on the query vector by the image label, reweight the image label according to the semantic information of the query vector, strengthen the region expression related to the prompt in the dense feature, generate the cross-attention output from the image to the query, input the output after layer normalization into the feedforward neural network of the image, and output the updated image label; S234, update the query vector and the image label through stacking multiple bidirectional Transformer blocks, obtain the final query token and image token, and input the query token and the image token as the query feature and the image feature into the dynamic mask head, wherein the query token includes K mask tokens and 1 IoU label; S235, reshape the image token into a feature map, add the dense prompt embedding element by element, and upsample to a preset resolution to obtain a feature map F1; S236, process each mask token through a multi-layer perception machine to obtain a feature map F2, perform inner product of the corresponding channel features of F1 and F2 at each pixel position to generate an unnormalized mask score, and perform multi-layer perception machine processing on the IoU label to output a quality score of the mask.

[0041] In the embodiment, the image decoder is also based on the Transformer architecture. It has several core parameters: image embedding (image_embeddings, from the image encoder, shape Bx256x64x64), image position encoding (image_pe, used to add position information to the dense token, shape Bx256x64x64), sparse prompt embedding (sparse_prompt_embeddings, point and frame prompt), and dense prompt embedding (dense_prompt_embeddings, projected dense feature).

[0042] The image decoder comprises a bidirectional Transformer block and a dynamic mask head, the bidirectional Transformer block comprises a self-attention mechanism of a query vector, cross-attention from the query to an image, a feedforward neural network of the query vector, cross-attention from the image to the query, and a feedforward neural network of the image; and the dynamic mask head comprises a feature map processing module, a mask generation module and a quality score module.

[0043] 1) Bidirectional cross-attention stacked layers: let sparse queries interact with dense image tokens, sparse queries (prompt tokens) -> focus on dense image tokens (cross-attention from query to image), dense image tokens -> review sparse queries (cross-attention from image to query). Through bidirectional cross-attention, the "prompt" can be injected into the context of the feature map, and the image context can be used to correct the attention range of the prompt. In addition, no self-attention is performed on the 64x64 image patches to reduce the computational complexity.

[0044] 2) Dynamic mask head: upsample the dense features after the Transformer to 256x256, then let each mask token pass through an MLP network to generate a set of dynamic convolution weights, and perform pixel-by-pixel linear combination on the upsampled features -> output the unnormalized scores (logits) of the mask. On the other hand, there will also be an IoU token that directly outputs the quality score of the mask through an MLP, which is used to select the best mask during inference.

[0045] Among them, the specific details of the bidirectional Transformer block are as follows: 1) Self-attention mechanism of query vector (Queries). Let different point / frame mask queries and IoU token / mask token queries interact with each other to unify the prompt. 2) Cross-attention from query to image. Use image position encoding to add position information to image tokens as keys K / values V, so that full-image queries can be performed. 3) Feedforward neural network (FFN) of query vector. Enhance the nonlinearity of the network and enhance the extraction of semantic information. 4) Cross-attention from image to query. Let the image token be reweighted so that the attention mechanism enhances the expressive ability of dense features. 5) Feedforward neural network of image. By adding the feature map after layer normalization and the feature map after multi-layer perceptron (MLP), the expressive ability of the features is enriched.

[0046] The updated query token and image token are obtained by stacking multiple bidirectional Transformer blocks. The query token contains the most critical K mask tokens (mask_tokens) and 1 IoU token, and the image token is used to generate the final mask.

[0047] The dynamic mask head has the following specific details: 1) the feature map after the bidirectional Transformer block is reshaped to BxCx64x64, and is added element by element with the dense prompt embedding, and then the feature map after addition is up-sampled to BxCx256x256 to obtain a feature map F1; 2) each mask token is subjected to a multi-layer perception (MLP) to obtain a feature map F2. At each pixel position (i, j), the inner product of F1[:,:,i,j] and F2 is taken to obtain the unnormalized score (logits value) of the pixel, and finally a mask of Bx1x256x256 is output; F1[:,:,i,j] represents that all channel values of the pixel at the i-th row and the j-th column of the feature map F1 in the channel dimension are sliced to obtain a vector, and the dimension is Cx1; the dimension of the feature map F2 is also Cx1, and the channel dimension is compressed to 1 through the inner product operation (i.e., the sum of the corresponding elements after multiplication), so as to obtain the unnormalized score of a single pixel. 3) the IoU label output MLP layer outputs a quality score of BxK, which can be used to select the optimal mask or return multiple candidates during inference. For the sake of simplicity, the present application directly returns the candidate with the highest score.

[0048] In some embodiments, the prompt encoder is trained by the following method: S201, the X-Ray image to be segmented is expanded into a 3-channel image and normalized, the normalized image is scaled, the scaled image is placed in the upper left corner and filled with pixels 0 in the lower right corner, the filled image is then pixel normalized, and the pixel normalized image is converted into a tensor format to obtain a preprocessed image; S202, the preprocessed image is input into the prompt encoder, and the coordinate information of the bounding box is stored in a dictionary, and the key value of the dictionary is the name of the image without the suffix; wherein, for multiple masks in the preprocessed image, the corresponding multiple boxes are converted into single boxes and input into the prompt encoder, the loss of each single box is calculated and accumulated; the loss of the single box is calculated by combining the complete intersection over union loss and the binary cross entropy loss; S203, after the training of all single boxes of a single image is completed, the model parameters are updated and the optimal model parameters are saved according to the average intersection over union on the validation set; S204, a tensor identical to the predicted mask is created, the in-box coordinate region is assigned a value of 1, and the intersection with the real mask is obtained to obtain the real mask region, and the first total loss function is calculated by combining the binary cross entropy loss and the Focal loss; S205, in the training process, the first total loss function is minimized by adjusting the model parameters to obtain a fine-tuned prompt encoder.

[0049] In this embodiment, the fine-tuning training process of the prompt encoder based on the bounding box prompt is as follows: the original 16-bit single-channel grayscale image is expanded into 3 channels and normalized to [0, 255], then the normalized image is scaled, unlike center padding, SAM selects to place the image in the upper left corner and fill the right lower corner with pixel 0, and performs pixel normalization with a mean of [123.675, 116.28, 103.53] and a variance of [58.395, 57.12, 57.375], and then converts it into a Bx3x1024x1024 tensor input into the network. During training, the coordinate information of the bounding box is stored in a dictionary, and the key value of the dictionary is the name of the image without the suffix. The image is sent into the network one by one, and since there may be multiple masks in one image, in order to make the output of the prompt encoder correct, multiple boxes are converted into single boxes, and the loss of each single box is calculated and accumulated. When calculating the loss function, a tensor with the same size as the predicted mask (pred_masks, the predicted result output by the decoder and the original image size) is created, and the coordinates in the box are assigned with 1. The intersection of the real mask and the predicted mask is obtained, and then the binary cross entropy loss (BCELoss) and the focal loss (FocalLoss) are used to calculate the loss. After all the boxes in a single image are trained, back propagation is performed, the parameters are updated, and the best model weight is saved according to the average intersection over union (miou) calculated on the validation set, which is used for testing later.

[0050] wherein the loss function is the core driving force for neural network training, which quantifies the difference between model prediction and real label, and provides a clear optimization target for optimization algorithm. Different loss functions are used for different visual tasks, for example, complete intersection over union loss (CIoULoss) and binary cross entropy loss (BCELoss) are selected as the bounding box regression and classification loss for object detection task. SAM is a prompt-driven class-agnostic binary segmentation, based on this, binary cross entropy loss function and FocalLoss loss function are selected for joint optimization during fine-tuning. The calculation formula of the first total loss function is as follows: ; ; ; wherein, is the first total loss function, is the binary cross entropy loss function, is the FocalLoss loss function, is the proportion weight for controlling the two loss functions, which is set to 0.5 during training. y is the real label, and p is the predicted label. is a class balance coefficient, and is set to 0.75 during training, is a modulation factor that suppresses negative samples, amplifies positive samples, and increases the contribution of positive samples in the total loss, and is set to 5 during training, is a class prediction probability, is a binary cross-entropy loss of the real label and the predicted label, is a FocalLoss loss of the class prediction probability.

[0051] In some embodiments, the defect segmentation model is trained in the following manner: S110, obtaining a training data set containing an X-Ray image, a segmentation mask, and a bounding box prompt; S120, inputting the training data set into an enhanced guided defect segmentation model, freezing the image encoder and the prompt encoder, training only the decoder part, saving the optimal model weight on the validation set according to the average intersection over union evaluation index, and obtaining a trained model; S130, pruning the trained model using an L1 channel pruning strategy, fine-tuning the pruned model, saving the optimal model weight on the validation set according to the average intersection over union evaluation index, and obtaining a pruned and fine-tuned model; Specifically, S130 includes: S131, calculating the L1 norm of the convolution kernel weight corresponding to each convolution layer output channel of the trained model to form an L1 norm set of each convolution kernel, and marking the channels with an L1 norm less than a pruning threshold as channels to be deleted; S132, performing a pruning operation to delete the marked channels to be deleted, and synchronously deleting the input channels corresponding to the pruned channels in the next layer to obtain a pruned model; S133, loading the data set and setting the training parameters, loading the pruned model weight for fine-tuning, optimizing the model parameters through the processes of forward propagation, loss function calculation, and back propagation, saving the optimal model weight during the fine-tuning process according to the average intersection over union evaluation index on the validation set, and obtaining a pruned and fine-tuned model.

[0052] S140, using the original defect segmentation model as a teacher model and the pruned and fine-tuned model as a student model for knowledge distillation, supervising the difference between the output of the student model and the output of the teacher model and the real label, saving the optimal student model weight according to the index on the validation set, and obtaining a trained defect segmentation model.

[0053] In this embodiment, considering factors such as computing power and training efficiency, the image encoder and the prompt encoder are frozen, and only the decoder part is trained; during the model training process, the optimal model weight is saved on the validation set according to the mean intersection over union (mIoU) evaluation index, and the trained model is obtained; the trained model is pruned using the L1 channel pruning strategy to reduce the model size, reduce the consumption of computing resources, and speed up the inference speed of the model, and the pruned model is trained; The pruning technique simplifies the complex original model into a small and efficient equivalent version by removing redundant weights or neurons in the network, which realizes significant compression of the model size, improvement of the inference speed and effective reduction of the energy consumption with a small amount of accuracy loss. Before pruning, the model is trained to obtain the best model weight, and then a pruning strategy is used to prune the original weight, save the pruned weight, and fine-tune the pruned weight to obtain the final pruned model weight. In the present application, the L1 channel pruning is used to prune the original model, and the convolutional layer weight tensor For example, the pruning process is as follows: 1) Obtain the weight of the oth convolution kernel .

[0054] 2) Calculate the L1 norm , , the height, width, input channel number and output channel number of the convolution kernel, , the specific value of the convolution kernel weight in the corresponding dimension, , the L1 norm, , the L1 norm set of each convolution kernel. The smaller the L1 norm is, the smaller the overall weight amplitude of the filter is, and the weaker the contribution to the output is, so the filter is preferentially pruned.

[0055] 3) Set a pruning threshold , and delete the channel of .

[0056] 4) Perform pruning and delete the corresponding input channel in the next layer.

[0057] In some embodiments, the original defect segmentation model is used as a teacher model, the pruned and fine-tuned model is used as a student model, the difference between the output of the student model and the output of the teacher model and the real label is supervised, the optimal student model weight is saved according to the index on the validation set, and the trained defect segmentation model is obtained, including: S141, the original defect segmentation model is used as a teacher model, the pruned and fine-tuned model is used as a student model, the parameters of the teacher model are frozen to provide only forward output; S142, input the training data set to the student model, perform forward calculation through the student model to obtain the first mask non-normalized score and the first mask quality score output by the student model, and input the same training data to the teacher model to obtain the second mask non-normalized score and the second mask quality score output by the teacher model as a soft label; S143, calculate the KL divergence of the first mask non-normalized score and the second mask non-normalized score as a distillation loss, calculate the binary cross entropy loss of the first mask non-normalized score and the real label mask as a classification loss, and sum the distillation loss and the classification loss to obtain a second total loss function; S144, update the parameters of the student model based on the second total loss function through back propagation, calculate the average intersection over union index on the validation set every interval of a preset number of iterations during the training process, stop training when the preset condition is met, save the student model weight when the average intersection over union index on the validation set is optimal, and obtain a trained defect segmentation model; the preset condition is that the number of iterations reaches a preset number of rounds or the index does not improve for a plurality of continuous iterations.

[0058] In this embodiment, the pruned model is fine-tuned, the optimal model weight in the fine-tuning process is saved according to the evaluation index, and a pruned fine-tuned model is trained; The fine-tuning process is as follows: 1) load the data set and set the training parameters. 2) load the pruned model weight to perform model training, and perform processes such as forward propagation, loss function calculation, and back propagation. 3) retain the optimal model weight according to the index on the validation set.

[0059] Then, the original defect segmentation network is used as a teacher model, and the pruned fine-tuned model is used as a student model for distillation. The rich knowledge of the original teacher model is used to guide and restore the performance of the pruned student model, thereby realizing efficient compression without losing accuracy; The necessity of distillation is that the model pruning is followed by fine-tuning to restore the accuracy, but the conventional fine-tuning has limitations: the capacity of the pruned model has been reduced, like a "primary school student", and it is difficult to achieve the performance ceiling of the original "university professor" level original model by relearning the training data. At this time, the necessity of knowledge distillation is highlighted. The core idea is to let the small model (student model) after pruning no longer learn only the original hard label, but directly imitate the output logic of the retained, unpruned large model (teacher model). The "soft label" provided by the teacher model is rich in inter-class similarity and other more abundant implicit knowledge, which can provide better guidance signals for the student model. Through this "master-apprentice transmission" mechanism, the student model can more efficiently learn the core decision logic of the teacher model within its limited expression ability, thereby restoring the accuracy and even achieving better generalization ability than the conventional fine-tuning, and finally ensuring that the lightweight model is both fast and good. The process of distillation is as follows: 1) load the teacher model (original model) and student model (pruned model). 2) freeze the teacher model to get the teacher model forward output; get the student model forward output, and the gradient is normally back propagated. 3) supervise the difference between the teacher output and the student output and the real label. Specifically, the second loss function is used to constrain during distillation , the calculation formula is: ; ; wherein, is the second total loss function, is the distillation loss, which is used to measure the difference between the student output and the teacher output, Softmax is the activation function, || represents branch merging, KL() represents KL divergence (Kullback-Leibler Divergence), is the teacher output without normalization, is the student output without normalization, T is the distillation temperature, 4) repeat iteration according to the verification set index to save the optimal student model weight.

[0060] In this embodiment, the optimal model weight in the distillation process is saved according to the evaluation index, the foreign matter image to be segmented is tested according to the obtained optimal model weight, and a segmentation result is obtained; Reference Figure 3 and Figure 4The trained algorithm model is deployed into a hardware detection device to realize online real-time defect segmentation and detection of the battery pole piece. The specific process is as follows: the workflow starts from an image acquisition module, the module adopts a high-precision X-Ray imaging system to perform non-destructive scanning imaging on the battery pole piece on the production line, and can obtain a high-resolution X-Ray image with rich details, thereby providing a high-quality data basis for subsequent analysis. The image data is then transmitted to the core of the system, a computing control unit, which is built-in with a high-performance GPU computing card or an embedded device specially designed for edge computing (such as NVIDIA Jetson AGX Orin), and has strong parallel computing capability. In this unit, the SAM model trained and fine-tuned is deployed and optimized to run, and is accelerated by inference engines such as Tensor RT or ONNXRuntime, to efficiently perform a series of computationally intensive tasks such as image preprocessing, model forward inference and result post-processing, thereby realizing real-time recognition and segmentation of defects. In order to further ensure the accuracy and flexibility of detection, the device is equipped with an interactive and prompt interface, and a high-sensitivity touch display screen. When the automatic detection result is uncertain or needs to be manually verified, the operator can directly review the suspicious area on the screen and provide point or frame prompts (bounding box prompts) through intuitive touch control, to trigger the model to perform manual-assisted fine segmentation, thereby forming an efficient human-machine collaborative detection closed loop. The final defect analysis result, including the accurate position, morphological characteristics and quantitative size of the defect, is processed by a result output and execution mechanism. The mechanism is deeply linked with the existing PLC control system of the production line, not only uploads the detection result to the manufacturing execution system (MES) for quality traceability and analysis, but also drives the sorting mechanism (such as a high-precision mechanical arm or a pneumatic push rod) to perform physical actions, thereby realizing automatic offline and sorting of the defective pole piece, and completing the full-automatic process from “detection-judgment-execution”.

[0061] In addition, the network and storage unit of the system ensures high-speed and stable transmission of data through an industrial gigabit Ethernet interface, and stores all detection images, defect records and model parameter archives reliably in a large-capacity solid state drive (SSD), thereby providing a solid data support for process optimization and quality big data analysis. The entire device seamlessly integrates algorithms, hardware and industrial processes, and finally realizes online, real-time, intelligent and high-precision non-destructive detection and automatic processing of defects of the battery pole piece.

[0062] Referring to Figure 5 The embodiment of the present application provides a kind of X-Ray image defect segmentation device based on enhanced guidance, comprising: The first module is used to obtain the X-Ray image to be segmented and the trained defect segmentation model;The defect segmentation model includes image encoder, prompt encoder and image decoder; The second module is used to input the X-Ray image to be segmented into the defect segmentation model, extract features from the X-Ray image to be segmented by the image encoder to obtain an image embedding, generate a cue embedding by the cue encoder, and perform bidirectional interactive attention calculation on the image embedding and the cue embedding by the image decoder to obtain query features and image features, and obtain the unnormalized score and quality score of the mask based on the query features and image features. The third module is used to generate a target defect mask based on the unnormalized score and the quality score of the mask, as the defect segmentation result of the X-Ray image to be segmented.

[0063] It is evident that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented in the present device embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0064] See Figure 6 This invention provides an enhanced guided X-Ray image defect segmentation system, comprising: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0065] It is evident that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0066] Furthermore, embodiments of the present invention also disclose a computer program product or computer program stored in a computer-readable storage medium. A processor of a computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the described method. Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0069] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] It should be understood that in this invention, "at least one (item)" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0071] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present invention should be within the scope of the claims of the present invention.

Claims

1. A defect segmentation method for X-Ray images based on enhancement guidance, characterized in that, The method includes the following steps: Acquire the X-Ray image to be segmented and the trained defect segmentation model; the defect segmentation model includes an image encoder, a cue encoder, and an image decoder. The X-Ray image to be segmented is input into the defect segmentation model. The image encoder extracts features from the X-Ray image to be segmented to obtain an image embedding. The cue encoder generates a cue embedding. The image decoder performs bidirectional interactive attention calculation on the image embedding and the cue embedding to obtain query features and image features. Based on the query features and image features, the unnormalized score and quality score of the mask are obtained. A target defect mask is generated based on the unnormalized score and quality score of the mask, and is used as the defect segmentation result of the X-Ray image to be segmented.

2. The method according to claim 1, characterized in that, The step of extracting features from the X-Ray image to be segmented using the image encoder to obtain the image embedding includes: Obtain the X-Ray image to be segmented, and convert the X-Ray image to be segmented into a preprocessed image in tensor format; the X-Ray image to be segmented is a single-channel grayscale image; Convolutional layers are used to reduce the dimensionality of the preprocessed image to obtain an initial feature map. The initial feature map is then input into a Transformer encoder based on the ViTDeT architecture. Deep features are extracted from the initial feature map through a windowed attention mechanism and a periodically inserted global attention mechanism, and a token map containing multi-scale contextual information is output. The token image is sequentially subjected to 1×1 convolution channel adjustment, LayerNorm2d activation function processing, and 3×3 convolution feature optimization to obtain an image embedding containing batch dimension, channel dimension, height dimension, and width dimension.

3. The method according to claim 1, characterized in that, The step of generating the cue embedding through the cue encoder includes: Obtain sparse and dense tooltips, where sparse tooltips include point tooltips and bounding box tooltips, and dense tooltips are segmentation masks; The sparse cue is encoded by mapping the 2D coordinates of the point cue to a position encoding vector of a preset dimension through random Fourier feature mapping. The bounding box cue is embedded by superimposing the corresponding corner points on the two corner points respectively to obtain the sparse cue embedding. The dense cue is encoded by downsampling the segmentation mask to the same resolution as the image embedding through a convolutional layer, adjusting the number of channels through convolutional mapping, and then combining the dense position encoding generated by the random Fourier mechanism to obtain the dense cue embedding. The sparse cue embedding and the dense cue embedding are fused to generate a cue embedding.

4. The method according to claim 1, characterized in that, The image decoder includes a bidirectional Transformer block and a dynamic masking block; the bidirectional interactive attention calculation of the image embedding and the cue embedding through the image decoder to obtain query features and image features, and the unnormalized score and quality score of the mask based on the query features and image features, including: The image embedding and prompt embedding are input into a bidirectional Transformer block. After performing self-attention computation on the query vector to establish the association between different types of queries, the image tags with added image location encoding are used as keys. Cross-attention computation is performed on the image tags through the query vector, and the query vector is updated through layer normalization and a feedforward neural network of the query vector. The updated query vector is used as keys. Cross-attention computation is performed on the query vector through the image tags, and the image tags are updated through layer normalization and a feedforward neural network of the image. After stacking multiple bidirectional Transformer blocks, the output query token and image token are used as dynamic masking inputs for query features and image features. The query token contains K mask tokens and 1 IoU tag. The image token is reshaped into a feature map, and after being added element-wise to the dense cue embedding, it is upsampled to a preset resolution to obtain F1. Each mask token is processed by a multilayer perceptron to obtain F2. At the pixel position, the inner product of the channel features corresponding to F1 and F2 is generated to produce the mask unnormalized score. At the same time, the IoU marker is processed by a multilayer perceptron to output the mask quality score.

5. The method according to claim 1, characterized in that, The prompt encoder was trained in the following way: The X-Ray image to be segmented is converted into a preprocessed image in tensor format and used as a training sample; The preprocessed images are input into the prompt encoder one by one, and the coordinate information of the bounding boxes is stored in a dictionary. The key of the dictionary is the name of the image after removing the suffix. For multiple masks in the preprocessed image, the corresponding multi-boxes are converted into single boxes and then sent to the prompt encoder. After training all single frames of a single image, backpropagation is performed. The loss of each single frame is calculated by combining the full intersection-union ratio loss and the binary cross-entropy loss and accumulated to obtain the loss value of the cue encoder. The model parameters are updated according to the loss value of the cue encoder and the optimal model parameters are saved according to the average intersection-union ratio on the validation set. Create a tensor identical to the predicted mask, assign 1 to the coordinate region within the box, and obtain the real mask region by finding the intersection with the real mask. The first total loss function is calculated by jointly using binary cross-entropy loss and Focal loss. During training, the model parameters are adjusted to minimize the first total loss function, resulting in a fine-tuned cue encoder.

6. The method according to claim 1, characterized in that, The defect segmentation model is trained in the following way: Obtain a training dataset containing X-Ray images, segmentation masks, and bounding box cues; The training dataset is input into the enhanced guided defect segmentation model, the image encoder and cue encoder are frozen, only the decoder part is trained, and the optimal model weights are saved on the validation set according to the average intersection-over-union ratio evaluation metric to obtain the trained model. The trained model is pruned using an L1 channel pruning strategy. The pruned model is then fine-tuned. The optimal model weights are saved based on the average intersection-over-union ratio (IoU) evaluation metric on the validation set, resulting in the pruned and fine-tuned model. Using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model, knowledge distillation is performed. The differences between the student model output, the teacher model output, and the true label are monitored. The optimal student model weights are saved based on the indicators on the validation set, resulting in a well-trained defect segmentation model.

7. The method according to claim 6, characterized in that, The process involves using the original defect segmentation model as the teacher model and the pruned and fine-tuned model as the student model for knowledge distillation. It monitors the differences between the student model output, the teacher model output, and the true labels. Based on the metrics on the validation set, it saves the optimal student model weights to obtain a trained defect segmentation model, including: The original defect segmentation model is used as the teacher model, and the pruned and fine-tuned model is used as the student model. The parameters of the teacher model are frozen so that it only provides forward output. The training dataset is input into the student model, and forward computation is performed through the student model to obtain the first mask unnormalized score and the first mask quality score output by the student model; the same training data is input into the teacher model to obtain the second mask unnormalized score and the second mask quality score output by the teacher model, which are used as soft labels. The KL divergence between the unnormalized score of the first mask and the unnormalized score of the second mask is calculated as the distillation loss, and the binary cross-entropy loss between the unnormalized score of the first mask and the true label mask is calculated as the classification loss. The distillation loss and the classification loss are weighted and summed to obtain the second total loss function. The parameters of the student model are updated through backpropagation based on the second total loss function. During the training process, the average intersection-union ratio (IUR) on the validation set is calculated at a preset number of iterations. When the iteration training reaches a preset condition, the training stops and the student model weights when the average IUR on the validation set is optimal are saved to obtain the trained defect segmentation model. The preset condition is that the number of iterations reaches a preset number of rounds or the index does not improve within a number of consecutive iterations.

8. A defect segmentation device for X-Ray images based on enhanced guidance, characterized in that, include: The first module is used to acquire the X-Ray image to be segmented and the trained defect segmentation model; The defect segmentation model includes an image encoder, a cue encoder, and an image decoder; The second module is used to input the X-Ray image to be segmented into the defect segmentation model, extract features from the X-Ray image to be segmented by the image encoder to obtain an image embedding, generate a cue embedding by the cue encoder, and perform bidirectional interactive attention calculation on the image embedding and the cue embedding by the image decoder to obtain query features and image features, and obtain the unnormalized score and quality score of the mask based on the query features and image features. The third module is used to generate a target defect mask based on the unnormalized score and the quality score of the mask, as the defect segmentation result of the X-Ray image to be segmented.

9. A defect segmentation system for X-Ray images based on enhanced guidance, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1 to 7.

Images