Cdzn Te wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement

Abstract

The application belongs to the technical field of image segmentation methods, and provides a tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on group consistency enhancement. The technical scheme adopted is: step S1: a 12+1 TPO data set is constructed, including 12 tellurium-zinc-cadmium CZT crystal images of different angles and 1 label image; step S2: a group consistency enhancement semi-supervised semantic segmentation method is defined; step S3: a group consistency enhancement semi-supervised semantic segmentation framework ICAF is constructed; step S4: the ICAF is trained until convergence, and a trained model is used for semantic segmentation. The application uses the'many-to-one' characteristic in the semi-supervised process to improve the segmentation accuracy of the CZT crystal.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image segmentation methods, and particularly relates to a tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement. BACKGROUND

[0002] The tellurium-zinc-cadmium (CdZnTe, CZT) crystal is mainly composed of three elements of tellurium, zinc and cadmium, and is currently a semiconductor material with great engineering significance and strategic significance. Due to its excellent chemical properties and photoelectric performance, it is widely used in medical imaging, nuclear radiation detection, astrophysics, photovoltaic power generation and other fields. However, in the production process of the CZT crystal, the growth periods of the elements are different and the manufacturing process is imperfect, which causes the crystal lattice of the CZT crystal to be destroyed to produce useless inclusions, and the inclusions will affect the subsequent research and application of the CZT crystal, so the inclusions need to be cut off. At present, the segmentation of the CZT single crystal and the inclusions is mainly completed by manual work, but the efficiency of manual recognition and cutting is low, and long-time contact with the CZT crystal will cause harm to the human body. With the development of deep learning technology, researchers use the image segmentation method to realize the automatic cutting of the CZT crystal, but this method needs high-resolution images and accurate annotations, which means that a great amount of manual annotation cost is needed.

[0003] Semi-supervised semantic segmentation aims to alleviate the burden of pixel-level annotation by leveraging unlabeled data. This is particularly important in industrial and medical fields, where obtaining labels can be particularly difficult and costly. Research is increasingly focusing on how to effectively utilize a small amount of labeled data and a large amount of unlabeled data to achieve better segmentation results. In the early stages of semi-supervised segmentation research, GANs were often used to generate supervision signals for unlabeled data. However, the inherent instability of GANs during training prompted a shift towards more stable alternatives, such as consistency regularization and self-training. Inspired by FixMatch, Unimatch adopted a weak-to-strong consistency training paradigm, setting a new benchmark in semi-supervised semantic segmentation. Recently, Corrmatch further advanced the field by combining a label propagation strategy to enhance the utilization of unlabeled data. Despite these advances in semi-supervised segmentation algorithms, they are primarily designed for traditional "one-to-one" correspondence, where each image has a one-to-one correspondence with its ground truth. In multi-view scenarios, these algorithms struggle to fully leverage the correlation between images, resulting in suboptimal performance. Considering the reflective properties and surface variations of CZT crystals, appropriate application of "many-to-one" characteristics can provide an effective method to alleviate these issues. Therefore, how to effectively utilize the "many-to-one" characteristics in the semi-supervised process to improve the segmentation accuracy of CZT crystals remains an important area for further exploration. SUMMARY

[0004] The present application aims at the problems existing in the prior art, and proposes a tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement. The present application uses the "one-to-many" feature in the semi-supervised process to improve the segmentation accuracy of the CZT crystal.

[0005] In order to achieve the above-mentioned purpose, the present application is realized by the following technical scheme: a tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement, which is carried out according to the following steps:

[0006] S1, constructing a 12+1 TPO data set, which includes 12 different angle tellurium-zinc-cadmium CZT crystal images and 1 label image;

[0007] S2, defining an intra-group consistency enhanced semi-supervised semantic segmentation method, training the model by simultaneously minimizing the supervised loss and the unsupervised loss, and constructing a new baseline for the semi-supervised semantic segmentation task on the TPO data set, and randomly extracting three images from each image group for network training in each iteration;

[0008] S3, constructing a semi-supervised semantic segmentation framework ICAF based on intra-group consistency enhancement;

[0009] The pseudo-label correction network PCN in the semi-supervised semantic segmentation framework ICAF includes a boundary correction network BCN and a spatial correction network SCN; the BCN uses an encoder-decoder based adaptive weight generator AWG for guiding subsequent image correction, the generator integrates shallow and deep features to dynamically determine the image weight, the feature extraction is performed by the encoder, and the feature fusion is realized through the skip connection in the decoder;

[0010] S4, training the ICAF until convergence, and using the trained model to perform semantic segmentation.

[0011] Further, the 12 CZT crystal images in the TPO data set of step S1 share a common ground truth label.

[0012] Further, the specific process of the semi-supervised semantic segmentation method in step S2 is as follows:

[0013] Let Y i represents M groups of labeled images, wherein Y i,k is the corresponding ground truth segmentation image, represents a one-hot encoded label image, C represents the number of segmentation categories, and let Y

[0014] The semi-supervised semantic segmentation simultaneously minimizes the supervised loss and the unsupervised loss to train the model, total training loss is denoted as:

[0015]

[0016] wherein, denotes the supervised loss calculated from labeled data, denotes the unsupervised loss calculated from unlabeled data, λ is the weighting coefficient balancing the contribution of the two losses, and the standard cross-entropy loss l is used to train the model on labeled and unlabeled data respectively:

[0017]

[0018] wherein, denotes the batch size, p denotes the prediction of the model, Y denotes the true label, and index i corresponds to the i-th image in the batch, j denotes the j-th pixel in the i-th image; denotes the indicator function, which ensures that only the predictions with confidence scores exceeding the predefined threshold τ are included in the loss calculation.

[0019] Using the above training process, a new baseline network is constructed for the semi-supervised semantic segmentation task on the TPO dataset, and three images k = 3 are randomly selected from each image group for network training in each iteration.

[0020] Further, the specific operation of the boundary correction network BCN is:

[0021] Using the weight-guided method based on AWG, the features conducive to boundary recognition are adaptively extracted from the input image. Given O sampling images, the i-th group applies weak enhancement, denoted as:

[0022] w i,o = AWG(T w (X i,o )), for o = 1, …, O (6)

[0023] wherein, T w (·) denotes a common image operation, and finally, the weighted sum of these images obtains the boundary correction image X i .

[0024] Further, the specific operation of the spatial correction network SCN is:

[0025] Given P weak enhancement images and the correction image X i , the backbone network encoder is used to derive the latent feature map, denoted as:

[0026] F i,P = E n (Tw (X i,p )),for p=1,…,P (7)

[0027] F i =E n (X i ) (8)

[0028] where E n (·) denotes the backbone network;

[0029] Taking the p-th image as an example, a latent augmentation module LAM is used to further emphasize beneficial latent space cues while filtering out irrelevant interference, denoted as:

[0030] A i,P =σ(Embed(F i,P ,F i ))×F i,P (9)

[0031] where Embed(·) denotes a convolutional layer, and sigma denotes a sigmoid function, A i,P is the latent feature extracted from F i,P , and finally, a simple summation operation is performed on A i,P to summarize the corrected latent cues, where p=1,…,P.

[0032] Further, the step 4) trains the constructed ICAF framework using the training data set, calculates the loss value combined with the loss function of formula (1), and updates and optimizes the network parameters through back propagation until convergence; the constructed network is tested using the test data set, and the semantic segmentation performance of the network is evaluated.

[0033] Compared with the prior art, the beneficial effects of the present application are.

[0034] 1. In order to solve the confirmation bias problem caused by the light sensitivity of CZT semiconductor, the present application reconsiders the semi-supervised semantic segmentation process from the group-based perspective. To the best of the inventors' knowledge, this is the first attempt to utilize intra-group consistency regularization in this context.

[0035] 2. The present application proposes a simple and effective intra-group consistency augmentation framework (ICAF) for semi-supervised semantic segmentation of CZT semiconductor, which combines two novel pseudo-label correction strategies. Through a large number of experiments, it is proved that the method of the present application significantly improves the accuracy of pseudo-labels and effectively reduces the influence of confirmation bias in the training process.

[0036] 3. The ICAF proposed in this invention achieves state-of-the-art performance on the TPO dataset, especially excelling in extremely low label settings. More importantly, this is achieved without introducing any additional computational burden during inference. Attached Figure Description

[0037] Figure 1 A schematic diagram of ICAF, a semi-supervised semantic segmentation framework for enhancing intra-group consistency.

[0038] Figure 2 This is a schematic diagram of the Adaptive Weight Generator (AWG) in BCN.

[0039] Figure 3 This is a schematic diagram of the potential enhancement module LAM in SCN.

[0040] Figure 4 This is a comparison chart of the present invention and different solutions.

[0041] Figure 5 and Figure 6 This is a comparison chart of parameter performance in the intragroup consistency enhancement framework ICAF of this invention.

[0042] Figure 7 This is a diagram showing the ablation experimental data of each component of the ICAF (Intra-Group Consistency Enhancement Framework) of this invention.

[0043] Figure 8 This is a visualization of the false label correction process in the ablation experiment of this method.

[0044] Figure 9 This is a graph showing the qualitative comparison results between the method of this invention and the supervision baseline. Detailed Implementation

[0045] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples, so as to fully understand how the present invention uses technical means to solve technical problems and achieve technical effects and to implement it accordingly.

[0046] Based on multi-view Figure 1 A semi-supervised semantic segmentation method for cadmium zinc telluride wafer images with enhanced consistency includes the following steps:

[0047] Step S1: Construct a 12+1 TPO dataset, which includes 12 images of cadmium zinc telluride (CdZnTe, CZT) crystals from different angles and 1 label image (the 12 crystal images share a single ground truth label);

[0048] The TPO dataset is composed of 12 RGB images of CZT crystals taken from different illumination angles, and all images share a common ground truth label. To balance the training cost and model generalization ability, the dataset is augmented, and finally there are 410 groups of images for training and 894 groups of images for testing.

[0049] Step S2: define a semi-supervised semantic segmentation method with intra-group consistency enhancement, and the specific steps are as follows:

[0050] Let represent M groups of labeled images, where Y i is X i,k the corresponding ground truth segmentation map, represent the one-hot encoded label map, and C represents the number of segmentation categories. Similarly, let represent N groups of unlabeled images, and the spatial resolution of each image is HxW.

[0051] Semi-supervised semantic segmentation trains the model by minimizing the supervised loss and the unsupervised loss at the same time, and the total training loss is represented as:

[0052]

[0053] wherein, represents the supervised loss calculated according to the labeled data, represents the unsupervised loss calculated according to the unlabeled data, and λ is the weighting coefficient for balancing the contribution of the two losses. The standard cross-entropy loss l is used to train the labeled and unlabeled data respectively:

[0054]

[0055]

[0056] wherein, represents the batch size, p represents the model prediction, and Y represents the true label. The index i corresponds to the i-th image in the batch, and j represents the j-th pixel in the i-th image, represents the indicator function, which ensures that only the predictions with confidence scores exceeding the predefined threshold τ are included in the loss calculation.

[0057] The present application adopts the above training process to construct a new baseline for the semi-supervised semantic segmentation task on the TPO dataset, and randomly selects three images (k=3) from each image group for network training in each iteration.

[0058] Step S3: construct a semi-supervised semantic segmentation framework ICAF based on intra-group consistency enhancement.

[0059] In the constructed ICAF framework, the pseudo-label correction network (PCN) includes a boundary correction network (BCN) and a spatial correction network (SCN). The BCN uses an encoder-decoder-based adaptive weight generator (AWG) to guide subsequent image correction. This generator integrates shallow and deep features to dynamically determine image weights. Feature extraction is performed by the encoder, and feature fusion is achieved through skip connections in the decoder. The SCN processes the boundary-enhanced image obtained from the BCN, highlighting salient regions related to the boundary while mitigating the effects of noise and redundant information.

[0060] like Figure 1 As shown, the specific operation of the Boundary Correction Network (BCN) is as follows:

[0061] Adopting such Figure 2 The weighted guided method based on AWG (Advanced Graphical Expression) adaptively extracts features beneficial for boundary recognition from the input image. Given O sampled images, the i-th group is weakly enhanced, as shown below:

[0062] w i,o =AWG(T w (X i,o (6)

[0063] Among them, T w (·) indicates common image operations, such as resizing, cropping, and flipping. For simplicity, the subscripts l and u are omitted. Finally, the weighted sum of these images yields the boundary-corrected image X. i .

[0064] like Figure 1 As shown, the specific operation of the Spatial Correction Network (SCN) is as follows:

[0065] Given P weakly enhanced images and X corrected images i The encoder (backbone network) is used to derive the latent feature maps, represented as:

[0066] F i,P =E n (T w (X i,p )), for p=1,…,P (7)

[0067] F i =E n (X i (8)

[0068] Among them, E n (·) indicates the backbone network. For the sake of brevity, the subscripts l and u are omitted.

[0069] Taking the p-th image as an example, the potential enhancement module (LAM) shown in formula (8) is used to further emphasize the beneficial potential spatial cues while filtering out irrelevant interference, and is expressed as: Figure 3

[0070] A i,P i,P i i,P (9)

[0071] Where Embed(·) represents a convolution layer, sigma represents a Sigmoid function, and A i,P is a potential feature extracted from F i,P . Finally, a simple summation operation is performed on A i,P to summarize the corrected potential cues, where p = 1, …, P.

[0072] To emphasize the generalization ability of the evaluation model while minimizing the cost of artificial labeling, the present application uses a training data set to train the constructed ICAF framework, calculates the loss value combined with the loss function of formula (1), and performs back propagation to update and optimize the network parameters until convergence; the constructed network is tested using a test data set to evaluate the semantic segmentation performance of the network.

[0073] The following describes a simulation experiment of the tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement of the present application.

[0074] Experimental setup

[0075] DeepLabV3+ and ResNet-101 backbone network are used for pre-training on ImageNet, and the ResNet backbone is configured with an output step size of 16 in all experiments. Each training small batch contains 16 images, of which 8 are labeled images and 8 are unlabeled images. The initial learning rate is set to 0.001 using the SGD optimizer, and the model is trained 80 times using the multi-learning rate scheduler. The initial learning rate is set to 0.001, optimized using momentum SGD, and the model is trained 80 times using the multi-decay learning rate scheduler, with an input resolution fixed at 320x320 pixels.

[0076] The present application uses the mean intersection over union (mIoU) index widely used in semantic segmentation to evaluate, which is used to quantify the overlap between the predicted segmentation and the true segmentation.

[0077] Comparison with the most advanced method

[0078] As Figure 4 ​​​​As shown, the group-based ICAF of this invention consistently outperforms other methods. Compared to the supervised baseline, our method achieves improvements of +8.7%, +7.1%, +5%, +4.4%, and +3.1%. The performance advantage becomes even more pronounced when trained with fewer labeled images, evident in the significant gains in low-label scenarios. Excellent performance in 5‰ partitioning further validates the superior ability of our method under extreme labeling conditions. Furthermore, our method outperforms previous best results by 1.9%, 3.7%, 3.4%, 1.8%, and 1.7% on each dataset segmentation, clearly demonstrating the superior performance of our group-based solution.

[0079] ablation experiment

[0080] This invention evaluates the effectiveness of the proposed module under the 50% partitioning protocol.

[0081] ICAF parameter validity

[0082] Parameters O and P play a crucial role in determining the quality of pseudo-tags, such as Figure 5 As shown, the optimal values ​​of O and P were determined, along with the ideal number of reference images for boundary enhancement in BCN and spatial aggregation cues within the latent space in SCN. Clearly, using the same values ​​for O and P does not always lead to performance improvements. The guiding information, represented by O, should contain more diverse and discriminative features than the reference information, represented by P, to effectively extract useful cues. However, excessively increasing the number of guiding images (O) may introduce additional noise, potentially impacting overall segmentation performance. Figure 6 As shown, experiments were conducted to determine the optimal perturbation space Q. Therefore, O=6, P=3, and Q=2 were chosen as the optimal configuration for the method of this invention.

[0083] ICAF Module Effectiveness

[0084] The effectiveness of each module within the ICAF framework will be comprehensively evaluated based on the proposed group-based baselines. Traditional weak-to-strong baselines are referred to as "semi-baselines," while the newly proposed baselines detailed in Equation (3) are called "group baselines." Figure 7As shown, all proposed modules made significant contributions to improving the overall performance of ICAF. First, the mIoU increased from 81.1% to 82.5% after adopting the group baseline, verifying the effectiveness of the proposed intra-group consistency assumption. Second, the commonly used data augmentation techniques were adopted in the semi-supervised semantic segmentation: image augmentation by CutMix (denoted as IA) and feature augmentation using Dropout (FA). The results showed that the performance increased to 83.3% and 83.9%, respectively, confirming that the intra-group consistency regularization of the present invention effectively supplemented these augmentation methods. Next, the BCN proposed by the present invention was introduced to optimize the pseudo-labels, which greatly improved the performance to 85.2%, surpassing the state-of-the-art methods. Finally, the complete configuration of the present invention significantly improved the performance to 86.3%, verifying the effectiveness of the overall method of the present invention.

[0085] As Figure 8 shown, to help those skilled in the art better understand the correction process in the present invention, some results of the ablation study are displayed intuitively.

[0086] Qualitative study

[0087] 1) Optimization of pseudo-labels: To more intuitively illustrate the effectiveness of the pseudo-label correction strategy of the present invention during the training process, the visualization results corresponding to the 4th, 5th and 6th rows in Figure 7 are shown as Figure 8 shown. Specifically:

[0088] By comparing Figure 8 (c) and Figure 8 (d), it can be observed that the proposed multi-view data augmentation scheme implemented by BCN effectively emphasizes the boundary information from different perspectives, thereby significantly enhancing the previously ambiguous boundaries.

[0089] In addition, by comparing Figure 8 (c), Figure 8 (d) and Figure 8 (e), it can be clearly seen that both the number and connectivity of pixels classified as pseudo-labels have been significantly improved with the support of the pseudo-label correction strategy of the present invention. This improvement is particularly evident compared to scenarios without these strategies, indicating that the present invention effectively mitigates the impact of confirmation bias.

[0090] 2) Visualization examples of ICAF: In Figure 9 , the present invention visually compares the proposed semi-supervised semantic segmentation algorithm and the supervised baseline. It is clear that the baseline method tends to misclassify connected regions, while the method of the present invention effectively utilizes multiple input images to alleviate this problem by introducing the pseudo-label correction strategy, improving the accuracy and stability of the segmentation results.

[0091] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for CdZnTe wafer image semi-supervised semantic segmentation based on intra-group consistency enhancement, characterized in that: According to the following steps: S1, a 12+1 TPO dataset is constructed, which includes 12 images of cadmium zinc telluride (CZT) crystals at different angles and 1 label image; S2, a consistency-in-group-enhanced semi-supervised semantic segmentation method is defined, the model is trained by simultaneously minimizing supervised loss and unsupervised loss, a new baseline is constructed for the semi-supervised semantic segmentation task on the TPO dataset, and three images are randomly selected from each image group for network training in each iteration; S3, a semi-supervised semantic segmentation framework ICAF based on consistency-in-group-enhancement is constructed; The pseudo-label correction network PCN in the semi-supervised semantic segmentation framework ICAF includes a boundary correction network BCN and a spatial correction network SCN; the BCN uses an adaptive weight generator AWG based on an encoder-decoder to guide subsequent image correction, the generator integrates shallow and deep features to dynamically determine image weights, feature extraction is performed by an encoder, and feature fusion is achieved through skip connections in the decoder; S4, train ICAF until convergence, and perform semantic segmentation using the trained model.

2. The method of claim 1, wherein the method is based on intra-group consistency enhancement of a CdZnTe wafer image semi-supervised semantic segmentation. The 12 CZT crystal images in the TPO dataset of step S1 share a common ground truth label.

3. The method of claim 2, wherein the method is based on intra-group consistency enhancement of a CdZnTe wafer image semi-supervised semantic segmentation. The specific process of the semi-supervised semantic segmentation method in step S2 is as follows: Let denote M sets of labeled images, where Y i is the set of images with labels, and X i,k is the set of images without labels. denote one-hot encoded label maps, C denotes the number of segmentation classes, let denote N sets of unlabeled images, each image has spatial resolution of H x W; Semi-supervised semantic segmentation trains a model by simultaneously minimizing a supervised loss and an unsupervised loss The total training loss is represented as: wherein, represents a supervised loss computed from labeled data, represents an unsupervised loss computed from unlabeled data, λ is a weighting factor balancing the contribution of both losses, and the standard cross-entropy loss training on labeled and unlabeled data, respectively: wherein, denotes the batch size, p denotes the prediction of the model, Y denotes the true label, the index i corresponds to the i-th image in the batch, j denotes the j-th pixel in the i-th image, denotes an indicator function that ensures that only predictions with a confidence score above a predefined threshold τ are taken into account for the loss computation; Using the above training process, a new baseline network is constructed for the semi-supervised semantic segmentation task on the TPO dataset, and three images k=3 are randomly selected from each image group for network training in each iteration.

4. The method of claim 3, wherein the method is based on intra-group consistency enhancement of a CdZnTe wafer image semi-supervised semantic segmentation. The specific operation of the boundary correction network BCN is as follows: Using the weight guidance method based on AWG, features conducive to boundary recognition are adaptively extracted from the input image, given O sampling images, the i-th group applies weak enhancement, denoted as: w i,o = AWG(T w (X i,o )), for o = 1,..., O (6) where T w (·) denotes usual image operations, and finally, the weighted sum of these images gives the boundary corrected image X i .

5. The method of claim 4, wherein the method is based on intra-group consistency enhancement of a CdZnTe wafer image semi-supervised semantic segmentation. The specific operation of the spatial correction network SCN is as follows: Given P weakly augmented images and a corrected image X i The backbone network encoder is used to derive a latent feature map, denoted as: F i,P = E n (T w (X i,p )), for p = 1,..., P (7) F i = E n (X i )(8) wherein E n (·) represents the backbone network; Taking the p-th image as an example, the latent enhancement module LAM is used to further emphasize beneficial latent spatial clues while filtering out irrelevant interference, denoted as: A i,P = σ (Embed(F i,P , F i )) x F i,P (9) where Embed(·) denotes a convolutional layer, σ denotes a Sigmoid function, A i,P is the latent feature extracted from F i,P and finally, a simple summation operation is performed on A i,P to aggregate the corrected latent cues, where p = 1,..., P.

6. The method of claim 5, wherein the method is based on intra-group consistency enhancement of a CdZnTe wafer image semi-supervised semantic segmentation. In step S4, the training data set is used to train the constructed ICAF framework, the loss value is calculated by combining the loss function of formula (1), and the network parameters are updated and optimized by back propagation until convergence; the constructed network is tested using the test data set to evaluate the semantic segmentation performance of the network.

Images