Method for predicting air cavity dissemination from lung cancer histopathologic image

Abstract

The invention belongs to the technical field of medical image analysis and artificial intelligence, and relates to a method for predicting air cavity dissemination from lung cancer histopathologic images, which comprises the following steps: S1, a pathologist performs air cavity dissemination diagnosis on a lung cancer histopathologic section, performs WSI digitization, and then performs image preprocessing to obtain an air cavity dissemination result of the lung cancer histopathologic section; dividing the obtained lung cancer tissue WSI data set into a training set and a verification set; s2, a multi-mode attention perception multi-instance learning model STAMP is constructed; s3, on the basis of the STAMP, similarity regularization constraint is further introduced, and the training set is adopted to train the STAMP to predict whether air cavity dissemination exists in the lung cancer tissue WSI or not; verifying the trained STAMP by using a verification set; s4, using the trained STAMP to predict whether the lung cancer tissue WSI has air cavity dissemination or not. A reliable auxiliary tool can be provided for clinical diagnosis of lung cancer air cavity dissemination, manual misjudgment is reduced, and the diagnosis efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of medical image analysis and artificial intelligence technology, specifically relating to a method for predicting airway dissemination from lung cancer histopathological images. Background Technology

[0002] Spread through air spaces (STAS) is a novel invasive pattern in lung cancer that has been identified in recent years. In 2013, Onozato et al. first discovered STAS using three-dimensional reconstruction. STAS is characterized by tumor cells scattered within alveolar spaces, often separated from the primary tumor by several alveoli. The World Health Organization subsequently defined STAS as the presence of tumor cells, such as micropapillary clusters, solid masses, or single tumor cells, within air spaces outside the periphery of the primary tumor, and included it in the pathological classification system of lung adenocarcinoma (LUAD). STAS is very common in LUAD, with reported incidence rates exceeding 30% in both domestic and international studies. It is significantly associated with various aggressive clinicopathological features, such as pleural invasion, vascular invasion, large tumor size, and higher pathological stages. Furthermore, STAS is considered a strong predictor of occult lymph node metastasis in stage IA LUAD. Due to the close association between STAS and poor prognosis and a high risk of recurrence after local resection, its diagnosis has become an extremely important clinical task.

[0003] Histopathological images are considered the gold standard for diagnosing STAS in the lungs. STAS diagnosis heavily relies on pathologists' detailed microscopic examination of numerous slides. This process is not only highly subjective but also often unreliable in hospitals with limited medical resources. Furthermore, STAS diagnosis is extremely labor-intensive, prone to misdiagnosis and missed diagnosis. One major cause of STAS misdiagnosis is the non-standardized procedures during tissue sampling and slide preparation, which may lead to the artificial shedding of tumor cells. Intraoperative STAS detection provides crucial information for surgical planning, such as deciding between lobectomy and sublobar resection. Postoperative STAS diagnosis also plays a key role in helping oncologists develop individualized treatment plans. However, studies have reported that the AUC of manual STAS diagnosis ranges from 0.63 to 0.8, with an overall accuracy of approximately 0.7. Its sensitivity is generally low, indicating a high risk of missed diagnosis. Therefore, there is an urgent need for an objective, accurate, and efficient auxiliary method for STAS diagnosis.

[0004] With the development of deep learning, artificial intelligence (AI) has gradually been integrated into pathological image analysis, achieving significant progress in cancer diagnosis and histological grading. Given the limitations of traditional pathological methods, AI-based STAS histopathological diagnosis is becoming an important direction. STAS mainly manifests in three histological morphologies: ① micropapillary clusters (lacking a fibrovascular core, sometimes forming a ring structure within the airway); ② solid nests (composed of tightly packed tumor cells); ③ single tumor cells (isolated distribution). The morphological heterogeneity and subtle features of STAS pose significant challenges to AI models, especially in identifying irregular or sparse lesions. STAS diagnosis relies on high-quality whole-slice images (WSIs) and expert annotation, which are expensive, labor-intensive, and subjective, resulting in limited and unevenly distributed training data. Furthermore, differences in staining protocols among different hospitals further hinder the model's generalization ability. Although deep learning models perform well in routine image classification tasks, their accuracy in STAS detection is still insufficient to support clinical applications. These challenges highlight the urgent need to develop more robust diagnostic strategies to improve the consistency, reliability, and efficiency of STAS identification. Summary of the Invention

[0005] The purpose of this invention is to provide a method for predicting airway dissemination from lung cancer histopathological images, in order to address the problem that the accuracy of current deep learning models in STAS detection is insufficient to support clinical applications.

[0006] To achieve the above objectives, the present invention provides a method for predicting airway dissemination from lung cancer tissue pathological sections, comprising the following steps:

[0007] S1. Pathologists diagnose airway dissemination in lung cancer tissue pathological sections selected according to inclusion and exclusion rules, and perform WSI digitization on the selected lung cancer tissue pathological sections. Then, image preprocessing is performed to obtain the lung cancer tissue WSI dataset, and the lung cancer tissue WSI dataset is divided into training set and validation set.

[0008] S2. Construct a multi-modal attention-aware multi-instance learning model, named STAMP; STAMP includes a dual-branch embedding module, a Transformer-based instance encoding module, and a multi-modal attention aggregation module;

[0009] S3. Based on STAMP, a similarity regularization constraint is further introduced. The training set is used to train STAMP to predict whether there is air space dissemination in WSI of lung cancer tissue; and the training set is used to validate the trained STAMP.

[0010] S4. Use the trained STAMP to predict the presence of airway dissemination in digitized lung cancer tissue pathological sections.

[0011] In one specific implementation, the image preprocessing includes selecting the ROI region of lung cancer tissue and segmenting the patch;

[0012] Image preprocessing specifically includes:

[0013] The WSI was segmented into tissue, background, and blurred regions using the Otsu algorithm's automatic thresholding method. Then, a sliding window strategy was used to divide the 20X-based WSI into 256X256 patches, and the spatial coordinates of the patches were obtained. All patches of the WSI were then enhanced with color using a structure-preserving generative adversarial network trained on the cancer genome atlas. Finally, all patches were used for image feature extraction using Gigapath.

[0014] In one specific implementation, the dual-branch embedding module in the STAMP introduces two types of learnable tokens during input: a head token and a tail token. The head token is biased towards capturing macroscopic or morphological features, while the tail token is biased towards capturing local context or microscopic differences.

[0015] Each token consists of a trainable parameter vector, which is appended to the beginning of the instance sequence from the input sequence.

[0016] In one specific implementation, the Transformer-based instance encoding module in the STAMP is:

[0017] After the two-branch embedding, the input sequence of each branch is fed into a lightweight Transformer encoder; this encoder employs the Nyström Attention technique; specifically, each branch applies the following layers:

[0018] H h =TransLayer(X h ),H t =TransLayer(X t );

[0019] X h and X t These represent input data with different features, respectively; TransLayer represents a neural network layer; H... h and H t They represent X respectively h and X t The output after processing by TransLayer;

[0020] Each branch extracts features independently, and the two branches are then concatenated to obtain a global feature representation that contains richer pathological semantics.

[0021] In one specific implementation, the multi-modal attention aggregation module in the STAMP integrates two mechanisms: gating attention and content-aware attention. The gating attention mechanism filters out areas unrelated to lesions through gating units, while the content-aware attention mechanism dynamically adjusts weights based on the contextual relationships between instances. Furthermore, through dual-path attention fusion, the results of the two attention mechanisms are weighted and integrated to generate the final global weight distribution.

[0022] In one specific implementation, the STAMP employs a classification and similarity regularization strategy;

[0023] After multimodal attention aggregation, STAMP generates a global representation vector M, which is input into the classifier to complete bag-level prediction, thereby determining whether the slice is positive for airway dissemination.

[0024] To avoid redundancy in features learned from different branches or patterns, STAMP further introduces a similarity regularization constraint: cosine similarity is used to measure the correlation between different feature representations, and a penalty is applied to the loss function when the similarity is too high, encouraging the model to learn complementary rather than repetitive feature patterns; ultimately, the overall loss function L... total Due to cross-entropy loss L CE And similarity constraint loss L sim Together they form a whole, and their calculation form is: L total =λL CE +(1-λ)L sim , where λ is set to 0.9.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] This invention can dynamically screen tissue regions strongly associated with STAS, suppress irrelevant noise, and improve global feature representation capabilities. By introducing similarity regularization constraints, feature redundancy is effectively avoided and diagnostic accuracy is improved. Experimental results show that the method of this invention achieves superior diagnostic performance (AUC exceeding 0.79) on multiple independent datasets, demonstrating good robustness and generalizability. This invention can provide a reliable auxiliary tool for the clinical diagnosis of STAS in lung cancer, reduce human error, improve diagnostic efficiency, and has broad application prospects.

[0027] The dual-branch representation of this invention overcomes the limitations of single-branch models, guiding the network to learn diverse lesion patterns from different semantic spaces (such as morphological features and contextual features), thereby enhancing feature diversity. Combining Transformer-based instance encoding and a multi-modal attention aggregation module (MPAA), this invention can dynamically select key regions related to lesions, avoiding noise interference and improving the discriminative power of the global representation. Simultaneously, a similarity regularization constraint learning strategy is employed to prevent pattern redundancy, ensuring that the learned lesion patterns possess high discriminativeness and complementarity.

[0028] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The present invention will now be described in further detail. Attached Figure Description

[0029] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0030] Figure 1 This is a flowchart of an embodiment of the present invention for analyzing lung cancer histopathological images; wherein, a is the annotation (cross-validation) and digitization of WSI of lung cancer patient histopathological images; b is the preprocessing of WSI, including segmentation, surface subdivision and repair; c is the preprocessing of WSI image features, dual token embedding, feature extraction (including Transformer-based instance encoding and MPAA module) and STAS diagnosis (classification using regularized similarity loss).

[0031] Figure 2 STAS is a common pathological feature in lung cancer histopathological images, which is distributed outside the main tumor in the form of micropapillary clusters.

[0032] Figure 3 STAS is a common pathological feature in lung cancer histopathological images, where it appears as a single cancer cell distributed outside the main tumor.

[0033] Figure 4 STAS is a common pathological feature in lung cancer histopathological images, which is distributed outside the main tumor in the form of solid cell nests. Detailed Implementation

[0034] The embodiments of the present invention will be described in detail below. The specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0035] Example 1

[0036] This invention provides a method for predicting airway dissemination from lung cancer tissue pathological sections, comprising the following steps:

[0037] S1. This embodiment first collected pathological slides from patients with air cavity dissemination (STAS) from Xiangya No. 2 Hospital and Xiangya No. 3 Hospital of Central South University, as well as the TCGA LUAD dataset. Pathologists diagnosed STAS in lung cancer tissue pathological slides selected according to inclusion and exclusion rules, thereby constructing three datasets: STAS-SXY, STAS-TXY, and STAS-TCGA.

[0038] Each slice was labeled with the presence or absence of STAS and its dissemination type. The selected lung cancer tissue pathological slices were digitized using WSI, and then the images were preprocessed to obtain the lung cancer tissue WSI dataset. The lung cancer tissue WSI dataset was divided into training set and validation set.

[0039] S2. Construct a multi-modal attention-aware multi-instance learning model, named STAMP; STAMP includes a dual-branch embedding module, a Transformer-based instance encoding module, and a multi-modal attention aggregation module;

[0040] S3. Based on STAMP, a similarity regularization constraint is further introduced. The training set is used to train STAMP to predict whether there is air space dissemination in WSI of lung cancer tissue; and the training set is used to validate the trained STAMP.

[0041] The STAMP model is trained using a five-fold cross-validation method. The STAMP model is considered complete when the loss during training no longer changes.

[0042] S4. Use the trained STAMP to predict the presence of airway dissemination in digitized lung cancer tissue pathological sections.

[0043] The image preprocessing includes ROI region selection and patch segmentation of lung cancer tissue; the specific image preprocessing is as follows:

[0044] The WSI (Waste Skin Indicators) were segmented into tissue, background, and blurred regions using the Otsu algorithm's automatic thresholding method, and effective pathological information was obtained through image enhancement. A sliding window strategy was then used to divide the 20X WSI into 256x256 patches, and the spatial coordinates of each patch were obtained. A structure-preserving generative adversarial network (GAN) trained on the Cancer Genome Atlas (TCGA) was used to perform staining enhancement on all WSI patches. Finally, Gigapath was used to extract image features from all patches. This approach reduced the impact of different scanning devices and slide quality on the model's generalization performance.

[0045] In diagnostic tasks using whole-slide images (WSI), a high-resolution pathological slide is typically divided into numerous patches, which are then grouped together as "instances" into a "bag." Each bag corresponds to a slide, and while the overall label of the slide (i.e., STAS positive or negative) is known, the label of each individual instance is usually unknown. This weakly supervised scenario is well-suited for using multi-instance learning methods.

[0046] Formalized, a slice WSI can be represented as a bag: X = {(x1,y1),(x2,y2),…,(x...} n ,y n )};

[0047] Where, x i Represents the i-th instance (patch), y i Let Y be the corresponding label (unknown). The entire slice corresponds to a bag-level label Y∈{0,1}. If none of the patches contain STAS, then Y=0; if at least one patch contains STAS, then Y=1. Therefore, the goal of the model is to infer whether STAS exists in the entire slice by learning the feature distribution of instances within the bag.

[0048] The dual-branch embedding module in STAMP is also known as the dual-token embedding module. In order for the model to extract complementary features from different semantic spaces, STAMP introduces two types of learnable tokens at input: head tokens and tail tokens. Head tokens are biased towards capturing macroscopic or morphological features, while tail tokens are biased towards capturing local context or microscopic differences.

[0049] Each group contains np vectors, which are initialized as parameters: T h ,T t ∈R np×d ;

[0050] Each token consists of a trainable parameter vector obtained from training a neural network, which is appended to the beginning of the instance sequence from the input sequence.

[0051] In this way, the network is guided to extract pathological features from different representation spaces, thereby increasing feature diversity.

[0052] X h =W h ([T h ;X]),X t =W t ([T t;X]);

[0053] X h and X t They represent different pathological features; W h and W t These represent different trainable weight matrices, which are used in neural networks to perform linear transformations on the input data; [T] i [;X] represents some form of indexing or slicing operation, that is, retrieving from X the value that is related to T. i The relevant parts.

[0054] The Transformer-based instance encoding module in STAMP is as follows:

[0055] After the two-branch embedding, the input sequence of each branch is fed into a lightweight Transformer encoder; this encoder employs the Nyström Attention technique.

[0056] Specifically, each branch applies the following layers:

[0057] H h =TransLayer(X h ),H t =TransLayer(X t );

[0058] X h and X t These represent input data with different features, respectively; TransLayer represents a neural network layer; H... h and H t They represent X respectively h and X t The output after processing by TransLayer;

[0059] Each branch extracts features independently, and the two branches are then concatenated to obtain a global feature representation containing richer pathological semantics. This two-branch design avoids the limitations of single-branch models, enabling the network to simultaneously capture macroscopic tissue structure and microscopic cell morphology, as shown below:

[0060] H=[H (1:np) h ;H (1:np) t ]∈R 2np×L ;

[0061] H is the final matrix obtained by concatenating the two branches, that is, the global feature representation containing richer pathological semantics after concatenation; H (1:np) h and H(1:np) t These represent the feature representations of different branches.

[0062] These aggregated tags are expected to encode region-related semantic patterns, which are useful for the final classification.

[0063] The Multimodal Attention Aggregation Module (MPAA) in the Multimodal Attention-Aware Multi-Instance Learning Model (STAMP) integrates two mechanisms: gating attention and content-aware attention.

[0064] Gated attention mechanism filters out areas unrelated to lesions through gating units, while content-aware attention mechanism dynamically adjusts weights based on the contextual relationship between instances, thereby focusing more accurately on lesion features; and further, through dual-path attention fusion, the results of the two types of attention mechanisms are weighted and integrated to generate the final global weight distribution.

[0065] With this design, MPAA can significantly amplify the features of STAS-related regions, effectively improving diagnostic performance at the whole-slice level.

[0066] AV=tanh(W v H ),AU=σ(W u H ), A=Softmax(W a (AV⊙AU));

[0067] Where ⊙ represents element-wise multiplication, W v W u ∈R L×D W a ∈R D×1 It is a trainable matrix. The attention weights A ∈ R. 1 ×2np It is used to aggregate labeled features into a global representation M∈R 1×L The formula is: M = A·H.

[0068] The multimodal attention-aware multi-instance learning model (STAMP) employs classification and similarity regularization strategies.

[0069] After multimodal attention aggregation, the multimodal attention-aware multi-instance learning model (STAMP) generates a global representation vector M, which is input into the classifier to complete bag-level prediction, thereby determining whether the slice is STAS positive.

[0070] To avoid redundancy in features learned from different branches or modes, the multi-modal attention-aware multi-instance learning model further introduces similarity regularization constraints:

[0071] Cosine similarity is used to measure the correlation between different feature representations. When the similarity is too high, a penalty is applied to the loss function to encourage the model to learn complementary rather than repetitive feature patterns. Finally, the overall loss function L... total Due to cross-entropy loss L CE And similarity constraint loss L sim Together they form a whole, and their calculation form is: L total =λL CE +(1-λ)L sim , where λ is set to 0.9.

[0072] Experimental results:

[0073] To evaluate the performance of different multiple instance learning (MIL) methods in STAS diagnosis, this invention benchmarked 10 representative MIL methods. Evaluation metrics included accuracy (ACC), AUC (area under the curve), precision, recall, and F1 score. The following is a comparison of the model's performance on three STAS datasets:

[0074] Table 1. STAS-SXY dataset:

[0075] The STAMP method performs best on the STAS-SXY dataset, with an AUC of 0.8058, an F1 score of 0.7376, and an accuracy of 0.7449, outperforming all benchmark methods.

[0076] In contrast, MaxPooling and MeanPooling performed poorly, indicating that they failed to capture key pathological clues.

[0077] Table 2, STAS-TXY dataset:

[0078] On the STAS-TXY dataset, STAMP also achieved the best performance with an accuracy of 0.8124, an F1 score of 0.7623, and an AUC of 0.8017. Although the AUC is slightly lower than CLAM-SB (0.8130), the results show that it has similar discriminative ability.

[0079] Table 3. STAS-TCGA dataset:

[0080] On the more challenging STAS-TCGA dataset, STAMP continues to perform strongly, with an AUC of 0.7928 and an F1 score of 0.6962, outperforming other models. While CLAM-MB achieves the best accuracy (0.7464), its lower F1 score (0.5337) reveals a trade-off between sensitivity and accuracy.

[0081] Ablation experiment:

[0082] Impact of the number of patterns: By evaluating different numbers of patterns (np=1 to 5), this invention found that the model performance is optimal when np=3, indicating that the optimal balance is achieved between representational ability and generalization ability.

[0083] Comparison of single-branch and two-branch architectures: The two-branch architecture (H,T) is consistently superior to the single-branch architecture (H), showing significant improvements across all metrics, especially in accuracy and F1 score.

[0084] Impact of label embedding strategy: Post-embedding consistently outperforms pre-embedding, indicating that post-embedding enhances contextual relevance and improves model performance when instance features interact with region-aware labels.

[0085] Impact of Multimodal Attention Aggregation Strategies: This invention compares feature-level aggregation (FA) with prediction-level aggregation (PA). The results show that feature-level aggregation (FA) consistently outperforms prediction-level aggregation (PA), especially when the number of multimodal aggregations is 3, achieving an accuracy of 0.7449, exceeding PA's 0.7301.

[0086] This embodiment first collected pathological slides of STAS patients from Xiangya Second Hospital and Xiangya Third Hospital of Central South University, as well as the TCGA LUAD dataset. Three senior pathologists were invited to perform cross-annotation, thus constructing three datasets: STAS-SXY, STAS-TXY, and STAS-TCGA. Each slide was labeled with the presence or absence of STAS and its propagation type. To address the complex features of STAS in whole-slice images, this invention designs a multimodal attention-aware multi-instance learning (MIL) model called STAMP. Its bi-branch representation overcomes the limitations of single-branch models, guiding the network to learn diverse lesion patterns from different semantic spaces (such as morphological features and contextual features), thereby improving feature diversity. Combining Transformer-based instance encoding and the multimodal attention aggregation module (MPAA), this invention can dynamically select key regions related to lesions, avoiding noise interference and improving the discriminative power of the global representation. Simultaneously, a similarity regularization constraint learning strategy is adopted to prevent pattern redundancy, ensuring that the learned lesion patterns have high discriminativeness and complementarity.

[0087] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions and substitutions can be made without departing from the inventive concept, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method of predicting air-space dissemination from lung cancer histopathological sections, characterized in that, It comprises the following steps: S1, a pathologist makes a diagnosis of air cavity dissemination on lung cancer tissue pathological sections screened according to inclusion and exclusion rules, and then digitizes the screened lung cancer tissue pathological sections, and then performs image preprocessing to obtain a lung cancer tissue WSI dataset, and divides the lung cancer tissue WSI dataset into a training set and a validation set; S2, a multi-mode attention perception multi-instance learning model named STAMP is constructed; the STAMP includes a double-branch embedding module, a Transformer-based instance encoding module and a multi-mode attention aggregation module; S3, based on the STAMP, further introduce similarity regularization constraints, and use the training set to train the STAMP to predict whether there is air cavity dissemination in the lung cancer tissue WSI; and use the validation set to verify the trained STAMP; S4, use the trained STAMP to predict whether there is air cavity dissemination in the digitized lung cancer tissue pathological sections.

2. The method of predicting air cavity dissemination from lung cancer histopathology images of claim 1, wherein, The image preprocessing includes ROI region selection and patch segmentation of lung cancer tissue; The image preprocessing is specifically: The WSI is segmented into tissue, background and fuzzy regions using the automatic threshold calculation method of Otsu algorithm; then the WSI based on 20X is divided into 256X256 patches using the sliding window strategy, and the spatial coordinates of the patches are obtained; the structural preservation generative adversarial network trained on the Cancer Genome Atlas is used to enhance the staining of all patches of the WSI; then, all patches use Gigapath for image feature extraction.

3. The method of predicting air cavity dissemination from lung cancer histopathology images of claim 1, wherein, The double-branch embedding module in the STAMP, that is, the STAMP introduces two types of learnable tokens when inputting, namely head tokens and tail tokens, the head tokens are biased to capture macro or morphological features; the tail tokens are biased to capture local context or microscopic differences; Each token is composed of a trainable parameter vector and is spliced into an instance sequence at the front end of the input sequence.

4. The method of predicting air cavity dissemination from lung cancer histopathology images of claim 3, wherein, The Transformer-based instance encoding module in the STAMP is: After double-branch embedding, the input sequence of each branch enters a lightweight Transformer encoder respectively; the encoder adopts Nyström Attention technology; specifically, the following layers are applied to each branch: H h = TransLayer(X h ), H t = TransLayer(X t ); X h and X t represent input data of different characteristics, respectively, TransLayer represents a neural network layer, H h and H t represent X h and X t output after processing by TransLayer; Each branch independently extracts features, and finally splices between the two branches to obtain a global feature representation containing richer pathological semantics.

5. The method of predicting air cavity dissemination from lung cancer histopathology images of claim 1, wherein, The multi-mode attention aggregation module in the STAMP comprehensively uses the gating attention and content-aware attention mechanisms; the gating attention mechanism filters out the regions irrelevant to the lesion through the gating unit, and the content-aware attention mechanism dynamically adjusts the weight based on the context relationship between instances; and further through double-path attention fusion, the results of the two types of attention mechanisms are weighted and integrated to generate the final global weight distribution.

6. The method of predicting air cavity dissemination from lung cancer histopathology images of claim 5, wherein, In the STAMP, classification and similarity regularization strategies are adopted; After multi-modal attention aggregation, STAMP generates a global representation vector M, which is input into the classifier to complete the bag-level prediction to determine whether the slice is positive for cavity dissemination; To avoid redundancy of features learned by different branches or different modes, STAMP further introduces a similarity regularization constraint: by measuring the correlation between different feature representations through cosine similarity, a penalty is imposed in the loss function when the similarity is too high to encourage the model to learn complementary rather than repetitive feature patterns; finally, the overall loss function L total is composed of the cross-entropy loss L CE and the similarity constraint loss L sim , whose calculation form is: L total = λL CE + (1-λ)L sim , where λ is set to 0.9.

Images