Cadmium zinc telluride wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement

Abstract

The invention belongs to the technical field of image segmentation methods, and provides a tellurium-zinc-cadmium wafer image semi-supervised semantic segmentation method based on intra-group consistency enhancement. According to the technical scheme, the method comprises the steps that S1, a 12 + 1 TPO data set is constructed, wherein the 12 + 1 TPO data set comprises 12 CZT crystal images of different angles and a label image; s2, defining an intra-group consistency enhanced semi-supervised semantic segmentation method; s3, constructing a semi-supervised semantic segmentation framework ICAF based on intra-group consistency enhancement; and S4, training the ICAF until convergence, and performing semantic segmentation by using the trained model. According to the method, the segmentation precision of the CZT crystal is improved by utilizing the many-to-one characteristic in the semi-supervised process.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation methods, and in particular relates to a semi-supervised semantic segmentation method of cadmium zinc telluride wafer images based on intra-group consistency enhancement. Background Art

[0002] Cadmium zinc telluride (CdZnTe, CZT) crystals are mainly composed of three elements: tellurium, zinc, and cadmium. They are currently a semiconductor material of great engineering and strategic significance. Due to its excellent chemical properties and optoelectronic properties, it is widely used in medical imaging, nuclear radiation detection, astrophysics, photovoltaic power generation and other fields. However, in the production process of CZT crystals, the different growth cycles of various elements and the imperfect production process cause the lattice of CZT crystals to be destroyed and produce useless impurity crystals. Impurity crystals will affect the subsequent research and application of CZT crystals, so it is necessary to remove the impurity crystals. At present, the segmentation of CZT single crystals and impurity crystals is mainly done manually, but the efficiency of manual identification and cutting is low, and long-term contact with CZT crystals will cause harm to the human body. With the development of deep learning technology, researchers use image segmentation methods to realize automatic cutting of CZT crystals, but this method requires high-resolution images and precise annotations, which means that a huge cost of manual annotation is required.

[0003] Semi-supervised semantic segmentation aims to alleviate the burden of pixel-level annotation by leveraging unlabeled data. This is particularly important in fields such as industrial manufacturing and medicine, where obtaining labels can be particularly difficult and costly. Research has increasingly focused on how to effectively leverage a small amount of labeled data and a large amount of unlabeled data to achieve better segmentation results. In the early stages of semi-supervised segmentation research, GANs were often used to generate supervisory signals for unlabeled data. However, the inherent instability of GANs during training has prompted a shift toward more stable alternatives such as consistency regularization and self-training. Inspired by FixMatch, Unimatch adopted a weak-to-strong consistency training paradigm and established a new leading position in semi-supervised semantic segmentation. Recently, Corrmatch has further advanced the field by incorporating a label propagation strategy to enhance the utilization of unlabeled data. Despite the progress made by these semi-supervised segmentation algorithms, they are mainly designed for the traditional "one-to-one" correspondence, where each image has a one-to-one correspondence with its ground truth. In multi-view scenarios, these algorithms have difficulty fully leveraging the correlation between images, resulting in poor performance. Considering the reflective properties and surface variations of CZT crystals, the proper application of the “many-to-one” feature can provide an effective way to alleviate these problems. Therefore, how to effectively utilize the “many-to-one” feature in the semi-supervised process to improve the segmentation accuracy of CZT crystals remains an important area for further exploration. Summary of the invention

[0004] The present invention aims at the problems existing in the prior art and proposes a semi-supervised semantic segmentation method for cadmium zinc telluride wafer images based on enhanced intra-group consistency. The present invention utilizes the "many-to-one" characteristic in the semi-supervised process to improve the segmentation accuracy of CZT crystals.

[0005] In order to achieve the above object, the present invention is implemented by the following technical scheme: a semi-supervised semantic segmentation method of cadmium zinc telluride wafer images based on enhanced intra-group consistency is performed according to the following steps:

[0006] S1. Build a 12+1 TPO dataset, which includes 12 images of cadmium zinc telluride (CZT) crystals at different angles and 1 label image.

[0007] S2. Define a semi-supervised semantic segmentation method with enhanced intra-group consistency. Train the model by minimizing both supervised and unsupervised losses. At the same time, build a new baseline for the semi-supervised semantic segmentation task on the TPO dataset. In each iteration, three images are randomly selected from each image group for network training.

[0008] S3, construct ICAF, a semi-supervised semantic segmentation framework based on intra-group consistency enhancement;

[0009] The pseudo-label correction network PCN in the semi-supervised semantic segmentation framework ICAF includes a boundary correction network BCN and a spatial correction network SCN; BCN uses an encoder-decoder-based adaptive weight generator AWG to guide subsequent image correction. The generator integrates shallow and deep features to dynamically determine image weights. Feature extraction is performed by the encoder, and feature fusion is achieved through skip connections in the decoder.

[0010] S4. Train ICAF until convergence and use the trained model for semantic segmentation.

[0011] Furthermore, the 12 CZT crystal images in the TPO dataset of step S1 share a common true value label.

[0012] Furthermore, the specific process of the semi-supervised semantic segmentation method in step S2 is as follows:

[0013] make represents M sets of labeled images, where Y i Yes X i,k The corresponding true value segmentation map, represents the one-hot encoded label map, C represents the number of segmentation categories, and represents N sets of unlabeled images, each with a spatial resolution of H×W;

[0014] Semi-supervised semantic segmentation by simultaneously minimizing the supervised loss and unsupervised loss To train the model, the total training loss It is expressed as:

[0015]

[0016] in, represents the supervision loss calculated based on the labeled data, represents the unsupervised loss calculated based on the unlabeled data, λ is the weighting coefficient that balances the contribution of the two losses, and the standard cross entropy loss l is used to train the labeled and unlabeled data separately:

[0017]

[0018] in, represents the batch size, p represents the model's prediction, Y represents the true label, index i corresponds to the i-th image in the batch, and j represents the j-th pixel in the i-th image; represents the indicator function, ensuring that only predictions with a confidence score exceeding a predefined threshold τ are included in the loss calculation.

[0019] Using the above training process, a new baseline network was constructed for the semi-supervised semantic segmentation task on the TPO dataset. In each training iteration, three images k = 3 were randomly selected from each image group for network training.

[0020] Furthermore, the specific operation of the boundary correction network BCN is:

[0021] The weight-guided method based on AWG is used to adaptively extract features that are conducive to boundary recognition from the input image. Given O sampled images, the i-th group applies weak enhancement, which is expressed as:

[0022] w i,o =AWG(T w (X i,o )),for o=1,…,O (6)

[0023] Among them, T w (·) represents common image operations. Finally, the weighted sum of these images obtains the boundary correction image X i .

[0024] Furthermore, the specific operation of the spatial correction network SCN is:

[0025] Given P weakly enhanced images and a rectified image X i , the backbone network encoder is used to derive the latent feature map, expressed as:

[0026] F i,P =E n (Tw (X i,p )), for p=1,…,P (7)

[0027] F i =E n (X i ) (8)

[0028] Among them, E n (·) indicates backbone network;

[0029] Taking the pth image as an example, the potential enhancement module LAM is used to further emphasize the beneficial latent space clues while filtering out irrelevant interference, which is expressed as:

[0030] A i,P =σ(Embed(F i,P ,F i ))×F i,P (9)

[0031] Among them, Embed(·) represents the convolution layer, σ represents the Sigmoid function, and A i,P It is from F i,P Finally, the latent features extracted from A i,P A simple summation operation is performed to aggregate the corrected potential cues, where p=1,…,P.

[0032] Furthermore, in step 4), the constructed ICAF framework is trained using a training data set, the loss value is calculated in combination with the loss function of formula (1), and back propagation is performed to update and optimize the network parameters until convergence; the constructed network is tested using a test data set to evaluate the semantic segmentation performance of the network.

[0033] Compared with the prior art, the invention has the following beneficial effects.

[0034] 1. In order to solve the confirmation bias problem caused by the light sensitivity of CZT semiconductors, the present invention re-examines the semi-supervised semantic segmentation process from a group-based perspective. As far as the inventors know, this is the first attempt in the art to use intra-group consistency regularization in this context.

[0035] 2. This paper proposes a simple and effective intra-group consistency enhancement framework (ICAF) for semi-supervised semantic segmentation of CZT semiconductors, combining two novel pseudo-label correction strategies. Through a large number of experiments, it is proved that the method of the present invention significantly improves the pseudo-label accuracy and effectively mitigates the impact of confirmation bias during training.

[0036] 3. The proposed ICAF achieves state-of-the-art performance on the TPO dataset, especially in the extremely low-label setting. More importantly, this is achieved without introducing any additional computational burden during the inference process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of ICAF, a semi-supervised semantic segmentation framework with enhanced intra-group consistency.

[0038] Figure 2 Schematic diagram of the adaptive weight generator AWG in BCN.

[0039] Figure 3 Schematic diagram of the potential enhancement module LAM in SCN.

[0040] Figure 4 It is a comparative data diagram of the present invention and different solutions.

[0041] Figure 5 and Figure 6 This is a parameter performance comparison chart in the intra-group consistency enhancement framework ICAF of the present invention.

[0042] Figure 7 This is a graph of ablation experiment data of each component of the intra-group consistency enhancement framework ICAF of the present invention.

[0043] Figure 8 This is a visualization of the pseudo-label correction process in this method’s ablation experiment.

[0044] Fig. 9 This is a qualitative comparison result diagram of the method of the present invention and the supervised baseline. DETAILED DESCRIPTION

[0045] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and examples, so as to fully understand and implement the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects.

[0046] Based on multi-view Figure 1 A consistency-enhanced semi-supervised semantic segmentation method for cadmium zinc telluride wafer images comprises the following steps:

[0047] Step S1: construct a 12+1 TPO dataset, which includes 12 cadmium zinc telluride (CdZnTe, CZT) crystal images at different angles and 1 label image (the 12 crystal images share one true value label);

[0048] Each set of TPO datasets consists of 12 RGB images of CZT crystals taken from different lighting angles, and all images share a common true value label. In order to balance the training cost and model generalization ability, the dataset was expanded, and finally there were 410 sets of images for training and 894 sets of images for testing.

[0049] Step S2: Define a semi-supervised semantic segmentation method with enhanced intra-group consistency. The specific steps are as follows:

[0050] make represents M sets of labeled images, where Y i Yes X i,k The corresponding true value segmentation map, represents the one-hot encoded label map, and C represents the number of segmentation categories. Similarly, let Represents N sets of unlabeled images, each with a spatial resolution of H×W.

[0051] Semi-supervised semantic segmentation by simultaneously minimizing the supervised loss and unsupervised loss To train the model, the total training loss It is expressed as:

[0052]

[0053] in, represents the supervision loss calculated based on the labeled data, represents the unsupervised loss calculated based on the unlabeled data, and λ is the weighting coefficient that balances the contribution of the two losses. The standard cross entropy loss l is used to train the labeled and unlabeled data separately:

[0054]

[0055]

[0056] in, represents the batch size, p represents the model's prediction, and Y represents the true label. Index i corresponds to the i-th image in the batch, j represents the j-th pixel in the i-th image, represents the indicator function, ensuring that only predictions with a confidence score exceeding a predefined threshold τ are included in the loss calculation.

[0057] The present invention adopts the above training process to construct a new baseline for the semi-supervised semantic segmentation task on the TPO dataset. In each iterative training, three images (k=3) are randomly selected from each image group for network training.

[0058] Step S3: construct a semi-supervised semantic segmentation framework ICAF based on intra-group consistency enhancement;

[0059] In the constructed ICAF framework, the pseudo-label correction network PCN includes a boundary correction network BCN and a spatial correction network SCN. BCN uses an encoder-decoder-based adaptive weight generator (AWG) to guide subsequent image correction. The generator integrates shallow and deep features to dynamically determine image weights. Feature extraction is performed by the encoder, and feature fusion is achieved through skip connections in the decoder; SCN processes the boundary-enhanced image obtained by BCN to highlight the salient areas related to the boundary while reducing the impact of noise and redundant information.

[0060] like Figure 1 As shown, the specific operation of the boundary correction network BCN is:

[0061] Use Figure 2 The weight-guided method based on AWG shown in Figure 1 adaptively extracts features that are beneficial to boundary recognition from the input image. Given O sampled images, the i-th group applies weak enhancement, which is expressed as:

[0062] w i,o =AWG(T w (X i,o )),for o=1,…,O (6)

[0063] Among them, T w (·) represents common image operations such as resizing, cropping, and flipping. For brevity, the subscripts l and u are omitted. Finally, the weighted sum of these images yields the boundary-corrected image X i .

[0064] like Figure 1 As shown, the specific operation of the spatial correction network SCN is:

[0065] Given P weakly enhanced images and a rectified image X i , the encoder (backbone network) is used to derive the latent feature map, expressed as:

[0066] F i,P =E n (T w (X i,p )), for p=1,…,P (7)

[0067] F i =E n (X i ) (8)

[0068] Among them, E n (·) denotes the backbone network, and the subscripts l and u are omitted for brevity.

[0069] Taking the pth image as an example, we use Figure 3 The latent enhancement module (LAM) shown in this paper further emphasizes beneficial latent space cues while filtering out irrelevant distractions, which is expressed as:

[0070] A i,P =σ(Embed(F i,P ,F i ))×F i,P (9)

[0071] Among them, Embed(·) represents the convolution layer, σ represents the Sigmoid function, and A i,P It is from F i,P Finally, for A i,P A simple summation operation is performed to aggregate the corrected potential cues, where p=1,…,P.

[0072] In order to emphasize the generalization ability of the evaluation model and minimize the cost of manual annotation, the present invention uses a training data set to train the constructed ICAF framework, calculates the loss value in combination with the loss function of formula (1), and performs back propagation to update the optimized network parameters until convergence; the constructed network is tested using a test data set to evaluate the semantic segmentation performance of the network.

[0073] The following is a simulation experiment on the semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency of the present invention.

[0074] Experimental setup

[0075] DeepLabV3+ and ResNet-101 backbone network are used, pre-trained on ImageNet. The ResNet backbone is configured with an output stride of 16 in all experiments. Each training mini-batch contains 16 images, 8 labeled images and 8 unlabeled images. The initial learning rate is set to 0.001 using the SGD optimizer, and the model is trained 80 times using a multi-learning rate scheduler. The initial learning rate is set to 0.001, optimized using momentum SGD, and the model is trained 80 times using a multi-decay learning rate scheduler. The input resolution is fixed to 320×320 pixels.

[0076] The present invention adopts the mean intersection over union (mIoU) metric widely used in semantic segmentation for evaluation, which is used to quantify the overlap between the predicted segmentation and the true segmentation.

[0077] Comparison with state-of-the-art methods

[0078] like Figure 4As shown, the group-based ICAF of the present invention consistently outperforms other methods. Compared with the supervised baseline, our method achieves improvements of +8.7%, +7.1%, +5%, +4.4% and +3.1%. The performance advantage becomes more obvious when training with fewer labeled images, as can be seen from the significant gains in low-label scenarios. The outstanding performance under 5‰ partitioning further verifies the superior ability of the present method under extreme labeling conditions. In addition, the present method is 1.9%, 3.7%, 3.4%, 1.8% and 1.7% higher than the previous best results in each dataset segmentation, clearly demonstrating the excellent performance of the group-based solution of the present invention.

[0079] Ablation experiment

[0080] The present invention evaluates the effectiveness of the proposed module under a 50% partitioning protocol.

[0081] ICAF parameter validity

[0082] The parameters O and P play a crucial role in determining the quality of pseudo-labels, such as Figure 5 As shown in Figure 3, the optimal values ​​of O and P are determined, which determine the ideal number of reference images for boundary enhancement in BCN and the spatial aggregation cues within the latent space in SCN. Obviously, using the same value for O and P does not always lead to performance improvement. The guidance information represented by O should contain more diverse and discriminative features than the reference information represented by P in order to effectively extract useful cues. However, excessively increasing the number of guidance images (O) may introduce additional noise, which may affect the overall segmentation performance. Figure 6 As shown, the present invention conducts experiments to determine the optimal perturbation space Q. Therefore, O=6, P=3 and Q=2 are selected as the optimal configuration of the method of the present invention.

[0083] ICAF module validity

[0084] Based on the proposed group-based baseline, the effectiveness of each module in the ICAF framework will be comprehensively evaluated. The traditional weak to strong baseline is called "semi-baseline", while the newly proposed baseline detailed in formula (3) is called "group baseline". Figure 7As shown, all the proposed modules have made significant contributions to improving the overall performance of ICAF. First, after adopting the group baseline, the mIoU increased from 81.1% to 82.5%, verifying the effectiveness of the intra-group consistency hypothesis proposed in the present invention. Secondly, the present invention adopts commonly used data enhancement techniques in semi-supervised semantic segmentation: image enhancement through CutMix (denoted as IA) and feature enhancement using Dropout (FA). The results show that the performance is improved to 83.3% and 83.9%, respectively, confirming that the intra-group consistency regularization of the present invention effectively supplements these enhancement methods. Next, the BCN proposed by the present invention is introduced to optimize the pseudo-labels, which greatly improves the performance to 85.2%, surpassing the most advanced methods. Finally, the complete configuration of the present invention significantly improves the performance to 86.3%, verifying the effectiveness of the overall method of the present invention.

[0085] like Figure 8 As shown, in order to help those skilled in the art better understand the correction process in the present invention, some results of the ablation study are intuitively displayed.

[0086] Qualitative Research

[0087] 1) Optimization of pseudo labels: In order to more intuitively illustrate the effectiveness of the pseudo label correction strategy of the present invention in the training process, the following example is presented. Figure 7 The visualization results corresponding to rows 4, 5, and 6 are as follows: Figure 8 Specifically:

[0088] By comparison Figure 8 (c) and Figure 8 (d) It can be observed that the proposed multi-view data augmentation scheme implemented by BCN effectively emphasizes the boundary information from different viewpoints, thus significantly enhancing the previously blurred boundaries.

[0089] In addition, by comparing Figure 8 (c) Figure 8 (d) and Figure 8 (e), it can be clearly seen that with the support of the pseudo-label correction strategy of the present invention, the number and connectivity of pixels classified as pseudo-labels are significantly improved. This improvement is particularly obvious compared to the scene without these strategies, indicating that the present invention effectively mitigates the impact of confirmation bias.

[0090] 2) Visualization example of ICAF: Fig. 9 In this paper, the proposed semi-supervised semantic segmentation algorithm and the supervised baseline are visually compared. It is obvious that the baseline method tends to misclassify connected regions, while the proposed method effectively utilizes multiple input images to alleviate this problem by introducing a pseudo-label correction strategy, thereby improving the accuracy and stability of the segmentation results.

[0091] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A semi-supervised semantic segmentation method for CdZnTe wafer images based on enhanced intra-group consistency, characterized by: Follow these steps: S1. Build a 12+1 TPO dataset, which includes 12 images of cadmium zinc telluride (CZT) crystals at different angles and 1 label image. S2. Define a semi-supervised semantic segmentation method with enhanced intra-group consistency. Train the model by minimizing both supervised and unsupervised losses. At the same time, build a new baseline for the semi-supervised semantic segmentation task on the TPO dataset. In each iteration, three images are randomly selected from each image group for network training. S3, construct ICAF, a semi-supervised semantic segmentation framework based on intra-group consistency enhancement; The pseudo-label correction network PCN in the semi-supervised semantic segmentation framework ICAF includes a boundary correction network BCN and a spatial correction network SCN; BCN uses an encoder-decoder-based adaptive weight generator AWG to guide subsequent image correction. The generator integrates shallow and deep features to dynamically determine image weights. Feature extraction is performed by the encoder, and feature fusion is achieved through skip connections in the decoder. S4. Train ICAF until convergence and use the trained model for semantic segmentation.

2. The semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency according to claim 1, characterized in that: The 12 CZT crystal images in the TPO dataset of step S1 share a common true value label.

3. The semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency according to claim 2, characterized in that: The specific process of the semi-supervised semantic segmentation method in step S2 is as follows: make represents M sets of labeled images, where Y i Yes X i,k The corresponding true value segmentation map, represents the one-hot encoded label map, C represents the number of segmentation categories, and represents N sets of unlabeled images, each with a spatial resolution of H×W; Semi-supervised semantic segmentation by simultaneously minimizing the supervised loss and unsupervised loss To train the model, the total training loss It is expressed as: in, represents the supervision loss calculated based on the labeled data, represents the unsupervised loss calculated based on unlabeled data, λ is the weighting coefficient that balances the contribution of the two losses, and the standard cross entropy loss is used. Train on labeled and unlabeled data separately: in, represents the batch size, p represents the model's prediction, Y represents the true label, index i corresponds to the i-th image in the batch, j represents the j-th pixel in the i-th image, represents the indicator function, ensuring that only predictions with confidence scores exceeding a predefined threshold τ are included in the loss calculation; Using the above training process, a new baseline network was constructed for the semi-supervised semantic segmentation task on the TPO dataset. In each training iteration, three images k = 3 were randomly selected from each image group for network training.

4. The semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency according to claim 3, characterized in that: The specific operation of the boundary correction network BCN is: The weight-guided method based on AWG is used to adaptively extract features that are conducive to boundary recognition from the input image. Given O sampled images, the i-th group applies weak enhancement, which is expressed as: w i,o =AWG(T w (X i,o )),for o=1,…,O (6) Among them, T w (·) represents common image operations. Finally, the weighted sum of these images obtains the boundary correction image X i .

5. The semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency according to claim 4, characterized in that: The specific operation of the spatial correction network SCN is: Given P weakly enhanced images and a rectified image X i , the backbone network encoder is used to derive the latent feature map, expressed as: F i,P =E n (T w (X i,p )),for p=1,…,P (7) F i =E n (X i )(8) Among them, E n (·) indicates backbone network; Taking the pth image as an example, the potential enhancement module LAM is used to further emphasize the beneficial latent space clues while filtering out irrelevant interference, which is expressed as: A i,P =σ(Embed(F i,P ,F i ))×F i,P (9) Among them, Embed(·) represents the convolutional layer, σ represents the Sigmoid function, and A i,P It is from F i,P Finally, the latent features extracted from A i,P A simple summation operation is performed to aggregate the corrected potential cues, where p=1,…,P.

6. The semi-supervised semantic segmentation method of CdZnTe wafer images based on enhanced intra-group consistency according to claim 5, characterized in that: In step S4, the constructed ICAF framework is trained using the training data set, the loss value is calculated in combination with the loss function of formula (1), and back propagation is performed to update the optimized network parameters until convergence; the constructed network is tested using the test data set to evaluate the semantic segmentation performance of the network.

Images