Medical image classification method and system based on class prototype enhancement

Abstract

The application discloses a medical image classification method and system based on prototype-like enhancement, and the method comprises the following steps: acquiring medical images with known category labels and image category description text features; determining a prototype of each image category through the medical images with known category labels; combining the prototype of each image category with the image category description text features to obtain an enhanced prototype of each image category; classifying the medical images according to the category labels and the enhanced prototype of each medical image to obtain clean samples and distribution-in-noise label samples; training a constructed medical image classification model through the clean samples and the distribution-in-noise label samples, obtaining a trained medical image classification model after the training; and classifying medical images by using the trained medical image classification model. The application realizes accurate identification of medical images and improves the robustness of the medical image classification model.

Description

Technical Field

[0001] This invention relates to the field of model training technology, and in particular to a medical image classification method and system based on class prototype enhancement. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Current research primarily utilizes deep neural networks to construct medical image classification models, training these models with labeled medical images, and then using the trained models for medical image classification. In the widespread application of deep neural networks, handling noisy labels, especially out-of-distribution noisy labeled samples, remains a key challenge. Noisy labels refer to incorrect classification labels present in the training data, which may cause the model to learn inaccurate features, thus degrading performance. On the other hand, out-of-distribution noisy labeled samples refer to samples of new categories that do not belong to the training data distribution. This usually involves data distribution shifts, which may be due to limitations in data collection methods, data sources, or data volume. These samples may interfere with the model's generalization ability, causing it to perform poorly on unknown categories. The model needs to maintain good generalization ability when facing these distributional differences.

[0004] However, medical image annotation is highly dependent on expert experience, resulting in high annotation costs and susceptibility to subjective factors, which may introduce noisy labels, such as mislabeling rare lesions as common lesions. In addition, actual clinical data may contain out-of-distribution noise samples, such as non-lesion areas, unknown lesion types, or heterogeneous data collected by different imaging devices (such as MRI, CT, X-ray, etc.). These samples may be misclassified into known categories. That is, the samples currently used to train medical image classification models not only include clean samples and in-distribution noise samples, but may also include out-of-distribution noise samples.

[0005] Existing methods for training medical image classification models only target noisy labeled samples within the training distribution, enhancing robustness by introducing regularization terms or improving network structure. However, these methods have limited ability to handle out-of-distribution noise samples, failing to effectively identify and process noise samples not belonging to the training distribution, thus limiting the model's robustness. Furthermore, the class prototype discrimination methods used in existing medical image classification models primarily rely on class prototypes for identification and discrimination. Class prototypes are constructed based on the features of each class in the training data, effectively representing the core features of each class. When a sample is input into the model, its similarity to each class prototype is calculated to determine whether it belongs to a known class. While this alleviates overfitting to some extent, the quality of class prototypes is low due to sample noise contamination, resulting in unsatisfactory sample purification. Moreover, existing methods typically focus only on intra-class features, ignoring the differences between inter-class features. This leads to insufficient discriminative power of class features, further impacting model performance and ultimately causing inaccurate medical image classification. Therefore, current medical image models suffer from inaccurate medical image classification and limited robustness. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a medical image classification method and system based on class prototype enhancement, which achieves accurate identification of medical images and improves the robustness of the medical image classification model.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a medical image classification method based on prototype enhancement is proposed, including: Obtain the features of medical images and image category description text with multiple known category labels; Determine the class prototype for each image category using medical images with known category labels; The class prototype of each image category is combined with the textual features describing the image category to obtain the enhanced class prototype of each image category; Medical images are classified based on their category label and enhanced class prototype to obtain clean samples and in-distribution noise-labeled samples. The constructed medical image classification model is trained using clean samples and in-distribution noisy labeled samples, and the trained medical image classification model is obtained after training is completed. Medical images are classified using a pre-trained medical image classification model.

[0008] Furthermore, the medical image classification model includes two parallel image classification networks; each image classification network includes a feature extractor and a classifier; the feature extractor extracts image features of the medical image; the classifier classifies the image features extracted by the feature extractor; the classification results output by the two classifiers are averaged to obtain the final medical image classification result.

[0009] Furthermore, when training the constructed medical image classification model, the large language model is used to extract the key regions of interest from clean samples and in-distribution noisy labeled samples; the key regions of interest extracted by the large language model are used as supervision signals to train the medical image classification model; after training is completed, a trained medical image classification model is obtained.

[0010] Furthermore, image features are extracted from each medical image; Based on the image features and labels of each medical image, a class prototype for each image category is calculated and determined.

[0011] Furthermore, for each medical image with a known category label, the predicted category of the medical image is calculated using two image classification networks. When both predicted categories match the category label of the medical image, the medical image sample is considered a clean sample. When only one of the two predicted categories matches the category label of the medical image, the sample is considered an in-distribution noise sample. When neither of the two predicted categories matches the category label of the medical image, the medical image sample is considered an out-of-distribution noise sample.

[0012] Furthermore, obtain the descriptive text for each medical image category; By using a text feature extraction model, features of the descriptive text for each medical image category are extracted to obtain the image category descriptive text features.

[0013] Secondly, a medical image classification system based on prototype enhancement is proposed, including: The data acquisition unit is used to acquire medical images and image category description text features with multiple known category labels; The class prototype determination unit is used to determine the class prototype of each image category based on medical images with known category labels. The class prototype enhancement unit is used to combine the class prototype of the image category with the image category description text features to obtain the enhanced class prototype of each image category. The sample classification unit is used to classify medical images based on the category label and the enhanced class prototype of each medical image, and obtain clean samples and in-distribution noise-labeled samples. The model training unit is used to train the constructed medical image classification model using clean samples and in-distribution noisy labeled samples. Once training is complete, a trained medical image classification model is obtained. The medical image classification unit is used to classify medical images using a trained medical image classification model.

[0014] Thirdly, a computer device is proposed, the device comprising: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the medical image classification method based on class prototype enhancement proposed in the first aspect.

[0015] Fourthly, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed by the prototype-based medical image classification method proposed in the first aspect.

[0016] Fifthly, a computer program product is proposed, which includes a computer program that, when executed by a processor, implements the medical image classification method based on class prototype enhancement proposed in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a medical image classification method and system based on class prototype enhancement. Before training the medical image classification model, the method first uses the textual features describing medical image categories to enhance the class prototype of each determined image category, obtaining an enhanced class prototype for each image category. By incorporating rich semantic knowledge into the class prototype, the enhanced class prototype becomes more comprehensive and accurate, enhancing its stability and reducing the impact of sample feature bias. Then, the enhanced class prototype is used to accurately classify medical images with known category labels, discarding noisy label samples between distributions and extracting only clean samples and noisy label samples within the distribution. This is used to train the constructed medical image classification model, thereby reducing the impact of noise on model training, avoiding overconfident predictions of out-of-distribution noisy samples, improving the training effect of the medical image classification model, and ultimately improving the accuracy of medical image classification.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0020] Figure 1 The flowchart of the medical image classification method based on class prototype enhancement disclosed in the embodiment is shown below. Figure 2 This is the overall framework of the existing medical image classification model disclosed in the embodiments; Figure 3 This is the overall framework of the medical image classification method based on class prototype enhancement disclosed in the embodiments; Figure 4 This is a detailed flowchart of the overall model training in the medical image classification method based on class prototype enhancement disclosed in the embodiment. Figure 5 This is a simplified diagram of the overall model training process in the medical image classification method based on class prototype enhancement disclosed in the embodiment. Figure 6 The sample screening flowchart disclosed in the embodiment; Figure 7 The flowchart for region-sensitive distillation regularization disclosed in the embodiment is shown. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0024] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0025] It should be noted that all data acquisition is conducted in accordance with laws and regulations and with user consent, and the data is used legally.

[0026] First, the terms used in the embodiments of this application will be explained, including: Clean sample: The label is correct and it belongs to the predefined category; Noise-labeled samples within the distribution: The labels are incorrect, but the true category belongs to the predefined category; Out-of-distribution noise labeled samples: mislabeled samples where the true class does not belong to a predefined class. Class prototype: A class representation vector obtained by averaging or weighting the features of samples of the same class, used to describe the core features of the class.

[0027] Large Language Models (LLMS): General-purpose language models (such as GPT-4 and CLIP) pre-trained on massive amounts of data, possess cross-modal semantic understanding and generation capabilities, and can provide rich prior knowledge.

[0028] Multimodal: By processing and integrating information of multiple data types (such as text, images, audio, and video), computers can more comprehensively understand and cope with complex information environments.

[0029] Teacher-student model: This is a knowledge distillation framework that improves the performance and generalization ability of student models by transferring knowledge from pre-trained teacher models (such as large language models) to student models (such as target networks).

[0030] Attention region: refers to the part of the input data that the model focuses on and extracts most relevant to the current task through the attention mechanism, so as to improve the relevance of feature extraction and the performance of the model.

[0031] Upsampling: A technique that increases the number of data points by interpolation or other methods to improve the resolution of a signal or image.

[0032] First, the application scenarios of the medical image classification method based on class prototype enhancement proposed in the embodiments of this application are described.

[0033] The medical image classification method based on class prototype enhancement proposed in this application is applied to medical image classification scenarios. The medical images can include dermoscopy images, pathological slide images, and ultrasound images, etc.

[0034] Current methods utilize pre-trained medical image classification models to classify medical images. However, the training samples used in training these models include not only clean samples and in-distribution noise samples, but also out-of-distribution noise samples. Furthermore, current medical image classification models only target in-distribution noise labeled samples during training, enhancing the model's robustness by introducing regularization terms or improving the network structure. This approach fails to effectively identify and handle out-of-distribution noise samples that do not belong to the training distribution, thus limiting the model's robustness. Furthermore, existing medical image classification models primarily use class prototype discrimination methods to identify and distinguish categories. By calculating the similarity between a sample and each class prototype, it can be determined whether the sample belongs to a known category. Class prototypes are constructed based on training samples of known categories. Therefore, if the training data itself is noisy or imbalanced (e.g., too few or poor-quality samples in some categories), the class prototype may not accurately represent the true characteristics of the category. This can lead to inaccurate model identification of out-of-distribution (OOD) samples. Class prototypes are usually calculated using the average features of samples within a category or some aggregation method, which may not fully capture the diversity of samples within a category. Especially for highly variable categories, the class prototype may not be able to represent the overall characteristics of the category well, resulting in unsatisfactory detection performance for OOD samples.

[0035] In this application, the prototype of the medical image category is enhanced by the image category description text features, making the enhanced prototype more comprehensive and accurate. This enhances the stability of the prototype, reduces the impact of sample feature bias, and uses the enhanced prototype to classify and screen medical images. Out-of-distribution noise label samples are discarded, and only clean samples and in-distribution noise label samples are used to train the constructed medical image classification model. This ensures the training effect of the medical image classification model, improves the accuracy of medical image classification, and enhances the robustness and adaptability of the medical image classification model in practical applications.

[0036] like Figure 2As shown, methods for screening out-of-distribution (OOD) labeled samples have evolved from traditional statistical methods to deep learning and class prototype methods. Early methods were primarily based on density estimation and probabilistic models, such as Gaussian Mixture Models (GMMs) and One-Class SVMs. These methods learned the feature distribution of known classes and detected samples deviating from this distribution during testing. However, these methods had limited effectiveness on high-dimensional data. With the development of deep learning, confidence detection based on the Softmax distribution became mainstream. This method uses the maximum Softmax probability of the classifier as the confidence that a sample belongs to a known class; samples below a threshold are classified as OOD samples. However, Softmax is prone to overconfidence, leading to misclassification. To address this, subsequent methods introduced temperature scaling and perturbation techniques to enhance the model's discriminative power. Furthermore, the Mahalanobis distance method utilizes a multidimensional Gaussian distribution in the feature space to calculate the distance between a sample and the class center, thereby detecting OOD samples. Recently, class prototype-based OOD detection methods have become a new trend. These methods construct a class prototype (i.e., the feature center of the class) for each class and calculate the similarity between the test sample and the class prototype during the inference phase. If a sample has low similarity to all class prototypes, it is classified as Out-of-Depth (OOD). Compared to traditional methods, the class prototype approach can more stably describe the commonalities and differences between categories, avoiding false positives caused by relying solely on Softmax confidence. Furthermore, the latest methods combine multimodal information (such as class descriptions generated by language models) to enhance the semantic expressive power of class prototypes, making OOD detection more accurate and robust in complex scenarios.

[0037] The medical image classification method based on class prototype enhancement disclosed in the embodiments of this application will be described in detail below.

[0038] like Figure 1 and Figures 3-7 As shown in the embodiments of this application, the medical image classification method based on class prototype enhancement includes: Obtain the features of medical images and image category description text with multiple known category labels; Determine the class prototype for each image category using medical images with known category labels; The class prototype of each image category is combined with the textual features describing the image category to obtain the enhanced class prototype of each image category; Medical images are classified based on their category label and enhanced class prototype to obtain clean samples and in-distribution noise-labeled samples. The constructed medical image classification model is trained using clean samples and in-distribution noisy labeled samples, and the trained medical image classification model is obtained after training is completed. Medical images are classified using a pre-trained medical image classification model.

[0039] The medical image classification model constructed in this application includes two parallel image classification networks; each image classification network includes a feature extractor and a classifier; the feature extractor extracts image features of the medical image; the classifier classifies the image features extracted by the feature extractor; the classification results output by the two classifiers are averaged to obtain the final medical image classification result.

[0040] Preferably, the cosine similarity between the image feature and the enhanced prototype of each image category is calculated, and the category corresponding to the enhanced prototype with the largest cosine similarity is selected as the image category of the image feature and the predicted category of the medical image to which the image feature belongs.

[0041] Taking dermoscopic images as an example in medical imaging, the corresponding image categories include: normal, melanoma, mole, etc.

[0042] When training the constructed medical image classification model, large language models (LLMs) are used to extract the key regions of interest from clean samples and in-distribution noisy labeled samples. The key regions of interest extracted by the large language models are used as supervision signals to train the medical image classification model. After training, a trained medical image classification model is obtained. The key regions of interest of the samples refer to the regions in the image samples that have the greatest impact on the final classification result.

[0043] This application leverages the powerful feature extraction capabilities of a large language model to enhance the predictive performance of a medical image classification model. To avoid the problem of the model overemphasizing global features and neglecting local details due to direct feature supervision, this application introduces region-sensitive distillation regularization during the training of the medical image classification model. By aligning the deep semantic knowledge of the large language model with the early, inaccurate classification network, the network is guided to focus on key regions of the samples, thereby improving performance. Specifically, the medical image classification model is used as the student model, the large language model as the teacher model, and the attention region of the last layer of the large language model as the supervision signal. The size of the attention region is unified through upsampling, and knowledge transfer is performed using distillation learning. This method effectively improves the network's ability to learn important regions, thereby optimizing the predictive accuracy of the medical image classification model.

[0044] Preferably, the feature extractor can be constructed using a convolutional neural network.

[0045] The classifier used is the Softmax classifier.

[0046] The large language model uses GPT-4o to generate descriptive text, and the CLIP model is used to extract features from the descriptive text.

[0047] In the sample screening stage, embodiments of this application extract image features for each medical image; Based on the image features and labels of each medical image, a class prototype for each image category is calculated and determined, and the core image features of each image category are represented by the class prototype.

[0048] The process of classifying medical images includes: for each medical image with a known category label, calculating the predicted category of the medical image using two image classification networks; when both predicted categories match the category label of the medical image, the medical image sample is considered a clean sample; when only one of the two predicted categories matches the category label of the medical image, the sample is considered an in-distribution noise sample; when neither of the two predicted categories matches the category label of the medical image, the medical image sample is considered an out-of-distribution noise sample.

[0049] The image classification network extracts image features from medical images, calculates the cosine similarity between the image features and each enhancement class prototype, and selects the enhancement class prototype with the highest cosine similarity as the predicted category of the medical image.

[0050] This embodiment obtains the descriptive text for each medical image category; By using a text feature extraction model, features of the descriptive text for each medical image category are extracted to obtain the image category descriptive text features.

[0051] Preferably, the image category description text features are extracted through the multi-view description module.

[0052] The multimodal semantic synchronization module synchronizes the image category description text features extracted by the multi-view description module to the class prototype of the corresponding image category, thereby enhancing the class prototype and obtaining an enhanced class prototype. Then, based on the enhanced class prototype, medical image samples are screened.

[0053] The medical image classification model is trained using the selected samples through the region-sensitive distillation regularization module.

[0054] The multi-view description module utilizes large language models to generate descriptive text for each medical image category, including descriptions of intra-class commonalities and inter-class differences, to improve the accuracy of class prototype computation on datasets with noisy labels and out-of-distribution samples. By designing cue templates, LLMs generate text describing category commonalities from multiple perspectives, as well as descriptions of differences emphasizing inter-category distinguishing features. Subsequently, a text encoder and adapter are used to extract features from these descriptions, which are then used as the corresponding image category description text features to optimize class prototype computation.

[0055] The descriptive text for tumor images includes common descriptions within the same category and differences between categories. Common descriptions within the same category include: In CT images, tumors typically appear as abnormal density shadows, presenting as round, oval, or irregular masses, with densities higher or lower than the surrounding normal tissue, such as low-density liver cancer or high-density calcifications. Benign tumors often have clear borders and a complete capsule, while malignant tumors often have blurred, spiculated edges, accompanied by signs of infiltration into surrounding tissues. During contrast-enhanced CT scans, malignant tumors, due to their rich blood supply, often show heterogeneous enhancement with no enhancement in the central necrotic area, while benign tumors show more uniform enhancement or no significant enhancement. Tumors may compress or invade adjacent structures, such as vascular displacement, organ deformation, or luminal obstruction, and some are accompanied by secondary changes such as surrounding edema and effusion. Some tumors show characteristic features, such as the "bull's-eye sign" (metastases), "fat density" (lipomas), or "popcorn calcifications" (pulmonary hamartomas), providing clues for differential diagnosis.

[0056] Inter-class differences are described as follows: tumors often form localized space-occupying lesions, with a round / irregular shape, compressing or infiltrating surrounding structures (such as vascular displacement, organ deformation), and the surrounding edema zone (such as white matter edema around brain tumors) may gradually worsen; The multimodal semantic synchronization module employs two parallel feature extractors from the medical image classification model to extract image features from medical images. Based on these image features, a class prototype for each image category is determined. Then, two integrators synchronize the image category description text features to the corresponding class prototypes. Each class prototype is then enhanced using these image category description text features to obtain an enhanced class prototype. This enhanced class prototype contains information on intra-class commonalities and inter-class differences, thus reducing errors caused by noisy samples and achieving accurate sample classification. Next, a sample cleansing strategy is employed. For each medical image with a known category label, the cosine similarity between the image features and each enhanced class prototype is calculated using two image classification networks. Based on the calculated cosine similarity, two predicted categories are determined for the medical image. When both predicted categories match the medical image's category label, the medical image sample is considered a clean sample. When only one of the two predicted categories matches the medical image's category label, the sample is considered an in-distribution noise sample. When neither predicted category matches the medical image's category label, the medical image sample is considered an out-of-distribution noise sample.

[0057] Specifically, when determining the predicted category of a medical image based on cosine similarity, the image category corresponding to the highest cosine similarity is selected as the predicted category of the medical image.

[0058] Subsequently, the medical image classification model was trained using only clean samples and noisy labeled samples within the segment, thus purifying the training data of the medical image classification model and improving its robustness and accuracy.

[0059] Preferably, the integrator in this embodiment adopts the classic attention mechanism to fuse the image category description text features and class prototypes, and integrates the semantic features from the text with the class prototype information to achieve the purpose of class prototype enhancement and obtain the corresponding enhanced class prototype.

[0060] The Region Sensitive Distillation Regularization module uses region sensitive distillation regularization, utilizing clean samples and noisy labeled samples within a region, taking the large language model as the teacher model and the medical image classification model as the student model, to train the medical image classification model. After training is complete, a trained medical image classification model is obtained.

[0061] When training the medical image classification model, clean samples and in-distribution noisy labeled samples are input into a large language model to extract the key regions of interest in the intermediate layer. These are then used as teacher models to guide student models, from which region-sensitive distillation regularization loss is calculated. Different strategies are applied to clean samples and in-distribution noisy labeled samples to calculate the classification loss. All losses are linearly summed to calculate the total loss of the medical image classification model, and backpropagation is performed based on the total loss to optimize the medical image classification model. When the number of optimization iterations of the medical image classification model reaches a set value, or when the optimization effect of the medical image classification model reaches its optimal value, it indicates that the training of the medical image classification model is complete, and a well-trained medical image classification model is obtained.

[0062] The medical image classification method based on class prototype enhancement proposed in this application generates descriptive text for each medical image category by using a large language model to describe intra-class commonalities and inter-class differences. It simultaneously optimizes class prototypes using multimodal semantics, utilizes prediction consistency to cleanse samples, filters out noisy samples, and removes out-of-distribution (OOD) noisy labels, and further improves the performance of the medical image classification model through region-sensitive knowledge distillation. This series of steps effectively improves the model's robustness and prediction accuracy in the case of noisy labels and OOD samples.

[0063] Compared to existing methods, this application adds two processes: image category description text feature acquisition and class prototype enhancement. First, two parallel networks are used to calculate the class prototype for each image category based on the sample labels. Then, a large language model is used to generate diverse and more discriminative category descriptions for each category, i.e., description text for medical image categories, and a text extractor is used to extract the text features of this description, i.e., image category description text features. These image category description text features and the calculated class prototypes are then input into a pre-designed semantic compressor. The semantic compressor can fuse multimodal information from images and text to generate semantically more detailed class prototypes, serving as enhanced class prototypes. This solves the problem of low class prototype quality and lack of information caused by mislabeled and worthless out-of-distribution noise samples. Subsequently, the enhanced class prototypes are used to classify each sample, removing worthless out-of-distribution noise-labeled samples and retaining clean samples and in-distribution noise-labeled samples as valuable samples.

[0064] For the selected valuable samples, this application proposes a knowledge distillation method based on a teacher-student framework. The large language model contains pre-trained knowledge, thus possessing powerful feature extraction capabilities. To avoid the problem of the network over-focusing on global features and neglecting local details due to direct feature supervision, this application introduces region-sensitive distillation regularization. The large language model extracts key attention regions of the samples as supervision, guiding the student network to focus on the key regions of the samples, thereby further improving performance. Specifically, the attention region of the last layer of the large language model is selected as the supervision signal. After upsampling to unify the size of the attention region, knowledge transfer is performed through distillation. This improves the network's ability to learn key information and optimizes the model's robustness.

[0065] The framework of this application demonstrates significant advantages in multimodal fusion and class prototype optimization. First, by generating multi-view descriptions, the model can gain a more comprehensive and detailed understanding of the features of each category. These multi-view descriptions not only include visual features at the image level but also incorporate rich semantic information, thus providing a deeper and more comprehensive category representation. Second, by introducing a multimodal semantic synchronization mechanism, text descriptions and image features can be seamlessly integrated, making the model's prediction results more stable and accurate. This synchronization mechanism ensures the effective integration of information from different modalities, making the class prototype not only more representative but also enhancing the model's adaptability to complex scenes. Furthermore, the framework of this application further improves the model's performance through region-sensitive distillation regularization. This regularization term guides the model to not only focus on global features during the learning process but also effectively capture local details in the image. In this way, the model can more accurately determine key regions in the image, thereby improving the model's prediction accuracy and robustness, especially in the face of complex backgrounds or blurred images, where the model exhibits stronger generalization ability. Under diverse datasets and complex environments, the model effectively handles different image features and variations, demonstrating stronger robustness and generalization ability than traditional methods. Overall, the embodiments in this application effectively improve the quality of class prototype generation in image classification tasks and enhance the model's deep semantic learning capabilities by cleverly combining multi-view descriptions generated by LLMs, multimodal semantic synchronization mechanisms, and region-sensitive distillation regularization. This method provides a novel approach and methodology for solving image classification problems with out-of-distribution noisy labeled samples, possessing significant application value and research significance.

[0066] This application also provides a medical image classification system based on class prototype enhancement, including: The data acquisition unit is used to acquire medical images and image category description text features with multiple known category labels; The class prototype determination unit is used to determine the class prototype of each image category based on medical images with known category labels. The class prototype enhancement unit is used to combine the class prototype of the image category with the image category description text features to obtain the enhanced class prototype of each image category. The sample classification unit is used to classify medical images based on the category label and the enhanced class prototype of each medical image, and obtain clean samples and in-distribution noise-labeled samples. The model training unit is used to train the constructed medical image classification model using clean samples and in-distribution noisy labeled samples. Once training is complete, a trained medical image classification model is obtained. The medical image classification unit is used to classify medical images using a trained medical image classification model.

[0067] The present invention also discloses a computer device, the device comprising: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the prototype-based medical image classification method disclosed in the embodiments of this application.

[0068] The present invention also discloses a computer-readable storage medium storing a computer program adapted for loading by a processor and executing the prototype-based medical image classification method disclosed in the embodiments of this application.

[0069] The present invention also discloses a computer program product, which includes a computer program that, when executed by a processor, implements the medical image classification method based on class prototype enhancement disclosed in the embodiments of this application.

[0070] The method disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0071] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0072] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A medical image classification method based on prototype-based enhancement, characterized in that, include: Obtain the features of medical images and image category description text with multiple known category labels; Determine the class prototype for each image category using medical images with known category labels; The class prototype of each image category is combined with the textual features describing the image category to obtain the enhanced class prototype of each image category; Medical images are classified based on their category label and enhancement class prototype to obtain clean samples and in-distribution noise label samples; the in-distribution noise label samples are specimens with incorrect labels but whose true category belongs to a predefined category. The constructed medical image classification model is trained using clean samples and in-distribution noisy labeled samples. The trained medical image classification model is obtained by: using a large language model to extract the key regions of interest from the clean samples and in-distribution noisy labeled samples; and using the key regions of interest extracted by the large language model as supervision signals to train the medical image classification model. Training is complete, and a well-trained medical image classification model is obtained. The key attention region of the sample refers to the region in the image sample that has the greatest impact on the final classification result. The medical image classification model is used as the student model, the large language model is used as the teacher model, and the attention region of the last layer of the large language model is used as the supervision signal. The size of the attention region is unified by upsampling, and knowledge transfer is performed by distillation learning. Medical images are classified using a pre-trained medical image classification model.

2. The medical image classification method based on class prototype enhancement as described in claim 1, characterized in that, The medical image classification model consists of two parallel image classification networks. Each image classification network includes a feature extractor and a classifier. The feature extractor extracts image features from the medical image. The classifier classifies the image features extracted by the feature extractor. The classification results output by the two classifiers are averaged to obtain the final medical image classification result.

3. The medical image classification method based on class prototype enhancement as described in claim 2, characterized in that, For each medical image with a known category label, the predicted category of the medical image is calculated using two image classification networks. When both predicted categories match the category label of the medical image, the medical image is considered a clean sample. When only one of the two predicted categories matches the category label of the medical image, the medical image is considered an in-distribution noise sample. When neither of the two predicted categories matches the category label of the medical image, the medical image is considered an out-of-distribution noise sample.

4. The medical image classification method based on class prototype enhancement as described in claim 1, characterized in that, Extract the image features of each medical image; Based on the image features and labels of each medical image, a class prototype for each image category is calculated and determined.

5. The medical image classification method based on class prototype enhancement as described in claim 1, characterized in that, Retrieve the descriptive text for each medical image category; By using a text feature extraction model, features of the descriptive text for each medical image category are extracted to obtain the image category descriptive text features.

6. A medical image classification system based on prototype-based enhancement, characterized in that, include: The data acquisition unit is used to acquire medical images and image category description text features with multiple known category labels; The class prototype determination unit is used to determine the class prototype of each image category based on medical images with known category labels. The class prototype enhancement unit is used to combine the class prototype of the image category with the image category description text features to obtain the enhanced class prototype of each image category. The sample classification unit is used to classify medical images according to the category label and the enhanced class prototype of each medical image to obtain clean samples and in-distribution noise labeled samples; the in-distribution noise labeled samples are specimens with incorrect labels but whose true category belongs to a predefined category; The model training unit is used to train the constructed medical image classification model using clean samples and in-distribution noisy labeled samples. After training, a trained medical image classification model is obtained. Specifically, the unit uses a large language model to extract the key regions of interest from the clean samples and in-distribution noisy labeled samples. The key regions of interest extracted by the large language model are used as supervision signals to train the medical image classification model. Training is complete, and a well-trained medical image classification model is obtained. The key attention region of the sample refers to the region in the image sample that has the greatest impact on the final classification result. The medical image classification model is used as the student model, the large language model is used as the teacher model, and the attention region of the last layer of the large language model is used as the supervision signal. The size of the attention region is unified by upsampling, and knowledge transfer is performed by distillation learning. The medical image classification unit is used to classify medical images using a trained medical image classification model.

7. An electronic device, characterized in that, The device includes: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the medical image classification method based on class prototype enhancement as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed by the prototype-based medical image classification method according to any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the medical image classification method based on class prototype enhancement as described in any one of claims 1-5.

Images