Method and device for carrying out focus detection on pancreatic ultrasonic endoscope image

Abstract

The invention discloses a method and a device for performing focus detection on a pancreatic ultrasonic endoscope image, and belongs to the field of computer-aided diagnosis and image data processing. According to the method, a focus detection model capable of processing two tasks of disease classification and image focus segmentation is constructed, the focus detection model is trained by constructing a pancreas ultrasonic endoscope image data set with a real label, and focus detection is performed on an input pancreas ultrasonic endoscope image by using the trained focus detection model. The disease type and the lesion area in the segmented image can be accurately predicted, the lesion detection model can be optimized through fine tuning in clinical application, and the prediction accuracy and applicability are improved.

Description

Technical Field

[0001] The invention belongs to the field of computer-aided diagnosis and image data processing, and in particular relates to a method and a device for detecting lesions in pancreatic ultrasonic endoscopic images. Background Art

[0002] Pancreatic cancer is highly malignant, progresses rapidly, and is insensitive to treatments such as radiotherapy and chemotherapy. Its onset is insidious, and early symptoms are atypical, so the incidence and mortality rates are clearly increasing worldwide. Endoscopic ultrasound (EUS), as a method for close-up examination of the pancreas, is currently the most effective examination for detecting solid pancreatic lesions. Unlike ordinary endoscopes, ultrasound images are presented in different grayscales, and it is difficult for doctors to identify texture information above the second order, which is very useful for cluster analysis of ultrasound images. This is a huge challenge for doctors.

[0003] Medical images contain a wealth of patient information, as well as a lot of irrelevant information. In order to provide doctors with more intuitive and clear auxiliary diagnostic information, it is very important to identify and detect the type and area of ​​lesions in medical images. The classification of pancreatic lesions can reveal the status of pancreatic lesions, and the segmentation of pancreatic lesions can provide detailed feature analysis of the size and shape of pancreatic lesions. The combination of the two can better assist clinicians in diagnosing lesions.

[0004] Multitask Learning (MTL) is a machine learning method that aims to improve the generalization ability of the model by using the correlation information contained in the task training process. The commonly used method is to train tasks through shared representations. This method improves the generalization ability of the model by learning the correlation knowledge between different tasks during the training process. However, most multitask frameworks are based on multiple models and do not consider the advantages of multitask shared networks. In addition, the complex network structure of such methods leads to poor real-time performance and cannot meet clinical diagnosis needs, which also makes many models unable to be applied in clinical practice.

[0005] Interactive machine learning is a method that combines human-computer interaction ideas and technologies with machine learning. It provides more valuable new data for model training and optimization through the data generated during the interaction between people and models, solves the shortcomings of insufficient generalization of machine learning in actual performance, and enhances the interpretability and user trust of the model to a certain extent. In addition, interactive machine learning can provide a new way to implement user personalization enhancement and reduce the requirements of machine learning for user expertise.

[0006] Medical imaging technology has become an important auxiliary method for clinical diagnosis and is increasingly used in clinical work. Doctors have an increasing demand for automated or semi-automated data analysis. Ultrasound endoscopic images contain rich patient information, but also a lot of irrelevant information, which brings more burden to doctors' diagnostic work. Since its introduction, deep learning has shown great advantages in the field of image processing. Currently widely used models, such as ResUNet and Resnet, are usually single-task oriented. The classification of pancreatic lesions can reveal the state of pancreatic lesions, and the segmentation of pancreatic lesions can comprehensively analyze their size and morphology, which is crucial for effective diagnosis and treatment planning. Although there are some frameworks based on multi-task methods that can solve this problem, most multi-task frameworks are based on multi-models and do not consider the advantages of multi-task shared networks. In addition, the complex network structure of such methods leads to poor real-time performance and cannot meet clinical diagnosis needs, which also leads to the inability of many models to be applied in clinical practice.

[0007] In addition, most of the current auxiliary diagnosis models are trained on specific data sets, which limits the universality of the models. Once deployed in clinical settings, the enhancement and optimization of these models will be limited. In addition, the lack of integration of clinician expertise in the continued application of these models will further affect the application of the models. Summary of the invention

[0008] The purpose of the present invention is to propose a method and system for lesion detection in pancreatic endoscopic ultrasound images, which can utilize the advantages of algorithm model parameter sharing to simultaneously classify and segment endoscopic ultrasound images in a shorter time and accurately identify lesions displayed in pancreatic endoscopic ultrasound images.

[0009] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0010] A method for detecting lesions in pancreatic endoscopic ultrasound images comprises the following steps:

[0011] 1) Construct a pancreatic ultrasound endoscopic image dataset with real labels of lesion areas and types and use it as a training dataset;

[0012] 2) Construct a lesion detection model, which includes a residual neural network ResNet, a feature pyramid network FPN, a pyramid pooling module PPM, a fusion module Fuse, a classification head module and a segmentation head model;

[0013] The ResNet network is used to extract multi-scale features of pancreatic ultrasound endoscopic images;

[0014] The FPN network is used to encode multi-scale features and generate multi-scale coded features;

[0015] The PPM module is used to process the multi-scale coding features and generate a feature map containing multi-scale information;

[0016] The Fuse module is used to fuse the feature map generated by the PPM module with the multi-scale coding features generated by the FPN network to generate a fused feature map;

[0017] The classification head module is used to classify diseases based on the feature maps generated by the PPM module;

[0018] The segmentation head module is used to segment the lesion area in the pancreatic ultrasound endoscopic image according to the fusion feature map;

[0019] 3) Use the training data set to train the lesion detection model, construct a multi-task loss function consisting of a classification loss function and a segmentation loss function, and optimize the model parameters with the minimum multi-task loss as the objective function;

[0020] 4) Use the trained lesion detection model to process pancreatic ultrasound endoscopic images to generate disease classification results and lesion area segmentation results in the image.

[0021] Furthermore, the processing steps of the PPM module include:

[0022] Perform parallel pooling on the input multi-scale features to generate multi-scale deep features;

[0023] Perform convolution on the deep features generated by each pooling layer to generate multiple abstract features;

[0024] Interpolate the abstract features generated by each convolution to the same size as the input multi-scale features;

[0025] The interpolated multiple features are concatenated with the input multi-scale features in the depth direction to generate a feature map containing multi-scale information.

[0026] Furthermore, a bilinear method is used to interpolate the feature map generated by the PPM module into a feature of the same size as the multi-scale coding feature generated by the FPN network, and the feature generated by the difference is connected with the coding feature to generate a new feature map, which is input into the Fuse module to be fused with the multi-scale coding feature generated by the FPN network.

[0027] Furthermore, both the classification loss function and the segmentation loss function use the BCE loss function

[0028]

[0029] Among them, y i is the true label of the i-th sample in the training dataset, p iis the classification probability value predicted by the classification head module or the segmentation probability value predicted by the segmentation head module, N is the total number of samples, and the log base is not limited.

[0030] Furthermore, when the trained lesion detection model is used in a clinical environment, the generated disease classification results and image lesion area segmentation results are modified and re-input into the model as fine-tuning data for fine-tuning training to optimize the parameters of the classification head module and the segmentation head module.

[0031] A device for detecting lesions on pancreatic ultrasonic endoscopic images comprises: a trained lesion detection model, the model comprising a residual neural network ResNet, a feature pyramid network FPN, a pyramid pooling module PPM, a fusion module Fuse, a classification head module and a segmentation head model; wherein the ResNet network is used to extract multi-scale features of the pancreatic ultrasonic endoscopic images; the FPN network is used to encode the multi-scale features to generate multi-scale encoding features; the PPM module is used to process the multi-scale encoding features to generate a feature map containing multi-scale information; the Fuse module is used to fuse the feature map generated by the PPM module with the multi-scale encoding features generated by the FPN network to generate a fused feature map; the classification head module is used to classify diseases according to the feature map generated by the PPM module; and the segmentation head module is used to segment the lesion area in the pancreatic ultrasonic endoscopic images according to the fused feature map.

[0032] Furthermore, the PPM module includes:

[0033] Multiple parallel pooling layers: used to process multi-scale features of the input and generate multi-scale deep features;

[0034] Convolutional layer: Each pooling layer is followed by a convolutional layer to process deep features and extract abstract features;

[0035] Interpolation layer: used to adjust the extracted abstract features to the same size as the input multi-scale features;

[0036] Splicing layer: It is used to splice the multiple features adjusted by the interpolation layer with the input multi-scale features in the depth direction to generate a feature map containing multi-scale information.

[0037] Furthermore, the Fuse module includes multiple convolution blocks, each of which is used to process features of different scales.

[0038] Furthermore, the classification head module includes a convolutional layer, an adaptive global average pooling layer, and a convolutional layer for disease classification.

[0039] Furthermore, the segmentation head module includes a convolutional layer for segmenting the lesion area in the image.

[0040] The present invention has the advantages that:

[0041] 1. The lesion detection model constructed by the present invention is a multi-task model of classification and segmentation. The model applies the UpperNet framework and selects ResNet as the basic model. It shares the features from the extracted backbone, and then uses different decoders to classify the disease and segment the lesions separately. It can take advantage of parameter sharing to classify and segment ultrasound endoscopic images simultaneously in a shorter time.

[0042] 2. The lesion detection model constructed by the present invention can be optimized during clinical application. The network parameters of the original pre-trained lesion detection model can be fine-tuned by using the data from the interaction between the doctor and the model. Only the last layer of parameters of the model are trained using new interaction data, thereby achieving continuous optimization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a structural diagram of the lesion detection model in an embodiment of the present invention.

[0044] Figure 2 4 is a structural diagram of a pyramid pooling module in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the various technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0046] The embodiment of the present invention specifically proposes a method and system for lesion detection on pancreatic ultrasonic endoscopic images, which mainly includes four parts: data collection, lesion detection model construction and detection, lesion detection model training and lesion detection model optimization.

[0047] 1. Data Collection

[0048] Construct a pancreatic ultrasound endoscopic image dataset with true labels of lesion areas and types. For example, a pancreatic ultrasound image dataset constructed by a team of digestive endoscopy experts from Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, was used. The dataset collected ultrasound endoscopic images, endoscopic records, and pathology reports from the endoscopy center from January 2021 to December 2023, and selected pancreatic solid mass cases with clear pathological diagnosis, including pancreatic cancer and autoimmune pancreatitis. Ultrasound endoscopic images of other diseases are used as normal pancreatic images. The labels of this dataset are annotated by four experienced endoscopic experts on the pancreatic lesion areas in ultrasound endoscopic images. During the annotation process, the existence of the lesions will be confirmed again, the specific classification of pancreatic related lesions will be predicted, and the boundaries of the solid mass lesions will be marked based on pathological characteristics. To ensure reliability, each image is labeled and confirmed by at least two different endoscopic experts. The entire dataset is then randomly partitioned: 75% of the cases in each diagnostic category are assigned to the training dataset, and the remaining 25% are assigned to the test dataset. It should be mentioned that the pancreatic images recorded for the same patient appear only in one group. The training set and test set consisted of 16771 and 4194 pancreatic images, respectively. The median age of the test dataset was 56 years old, ranging from 20 to 88 years old, and the overall male-female ratio was 3:1.

[0049] 2. Construction and detection of lesion detection model

[0050] A lesion detection model was constructed, which has the ability to quickly and accurately detect pancreatic lesion areas and lesion types in pancreatic ultrasound endoscopic images. The lesion detection model uses the UpperNet framework, which is a multi-task learning (MTL) framework that focuses on image lesion segmentation and disease classification tasks. The UpperNet framework shares features from the extracted backbone, uses different decoders to classify diseases, and segments lesion areas separately. The specific architecture of the lesion detection model is as follows Figure 1 As shown, it specifically includes the following modules:

[0051] 1. Residual Neural Network (ResNet): As a basic model, it can capture multi-scale feature maps for segmentation or object detection.

[0052] 2. Feature Pyramid Network (FPN): Receives multi-scale feature maps, fuses information between these feature maps, and outputs a set of multi-scale feature maps.

[0053] 3. Pyramid pooling module (PPM, PPM Head): used to process multi-scale feature maps and fuse multi-scale information. Its output is then fused with the feature maps of different scales output by FPN. Figure 2The figure shows the specific structure of the pyramid pooling module, which consists of multiple parallel convolutional layers with different pooling expansion steps, followed by a global average pooling layer and a final convolutional layer. The module specifically includes:

[0054] a. Multiple parallel pooling layers (POOL), each pooling layer adopts different pooling expansion steps to process feature maps at multiple scales. In this embodiment, there are four parallel pooling layers to process feature maps of four scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32.

[0055] b. Convolution Layers (CONV): Each pooling layer is followed by a convolution layer to extract more abstract features.

[0056] c. Interpolation Layer (INTERPOLATE): The convolutional feature map is adjusted to the same size as the input feature map through the interpolation layer.

[0057] d. Concatenation Layer (CONCAT): All interpolated feature maps are concatenated with the input feature map to form a feature map containing multi-scale features.

[0058] The processing process of the pyramid pooling module is as follows:

[0059] 1) Input the multi-scale feature map extracted by ResNet into PPM.

[0060] 2) PPM processes the feature map through four pooling layers of different scales to generate four deep features of different sizes.

[0061] 3) Each pooled feature map passes through a convolutional layer to further extract features.

[0062] 4) Each convolutional feature map is interpolated to the same size as the input feature map.

[0063] 5) The four interpolated feature maps are concatenated with the input feature map in the depth direction to form a feature map containing multi-scale information.

[0064] When the pyramid pooling model is trained, the number of training rounds determines the degree to which the input feature map is downsampled. The larger the pooling scale, the larger the range on the feature map, and a single feature map can be downsampled to a scale; the smaller the pooling scale, the smaller the range, and the more downsampling results. These extracted features of different scales are then concatenated together and put into the classification head.

[0065] 4. Fusion module (Fuse): used to fuse feature maps of different scales after PPM processing to generate a unified fused feature map. The fused feature map will be used for subsequent lesion area segmentation tasks.

[0066] 5. Classification Head Module: It consists of a convolutional layer (Conv 3x3), an adaptive global average pooling layer (Global Avg.Pooling), and a convolutional layer (Classifier) ​​that maps all summary features to three diseases. This module can effectively capture objects and useful image context at multiple scales, while extracting global views and fine-grained details to classify disease types. At the same time, whether in the PPM module or the classification head, the adaptive pooling method reduces computational and memory costs compared to the fixed pooling method.

[0067] 6. Segmentation Head: In the process of segmenting the lesion area, the multi-scale features generated by the PPM module need to be used. Unlike the traditional UNet structure that uses the upsampling method, the segmentation head module only uses the multi-scale features generated by the PPM module and the multi-scale feature maps from different encoder stages of the FPN module to calculate the segmentation results, that is, the fused feature map output by the Fuse is segmented through the convolution layer (Conv 3x3) to obtain the segmentation result of the lesion area. Long-term experience has shown that directly using the feature map of the encoder can obtain better results and computational efficiency than the upsampling method.

[0068] The processing of the lesion detection model is as follows:

[0069] 1) The input pancreatic ultrasound endoscopic image is subjected to feature extraction through the ResNet model to obtain feature maps of multiple scales (such as 1 / 4, 1 / 8, 1 / 16, and 1 / 32).

[0070] 2) The above multi-scale feature maps are input into FPN, and the multi-scale encoder of FPN encodes the features to generate a set of new encoded features while maintaining different resolutions (such as 1 / 4, 1 / 8, 1 / 16, 1 / 32).

[0071] 3) PPM processes the encoded features generated by FPN to generate a feature map I containing multi-scale information.

[0072] 4) Use a bilinear method to interpolate the feature map I output by PPM into a feature map of the same size as the different encoder feature maps of FPN, and connect these two multi-scale feature maps together as a series of new feature maps I′.

[0073] 5) The above series of new feature maps I′ are input into the Fuse module and placed into a convolutional block respectively. Features of different depths are extracted into the same size, and the two features output by FPN and PPM are combined. These new features of the same channel size are interpolated into the same size (such as 1 / 4) and connected again to fuse into the final fused feature map (Fused Feature Map).

[0074] 6) The feature map I generated by PPM is processed by the convolution layer of the classification head module, and global average pooling is performed (Global Avg. Pooling), and then input into the classifier for disease classification, and the disease classification result is output.

[0075] 7) Taking the fused feature map as input, it is processed by the convolutional layer of the segmentation head to obtain the lesion area segmentation result.

[0076] 3. Lesion Detection Model Training

[0077] The lesion detection model is trained using the training data set. Since the model needs to perform classification and segmentation tasks, the constructed multi-task loss function consists of a classification loss function and a segmentation loss function. Both the classification loss function and the segmentation loss function use the following Binary Cross-Entropy (BCE) loss function: Since the lesion detection model is based on UpperNet, the loss function weight ratio of the segmentation task is higher, so the multi-task loss function of the present invention is It is the cross entropy loss function of the classification task and the segmentation task and They are composed according to their weights, with a weight ratio of 1:100, that is,

[0078]

[0079] Among them, y i is the true label of the i-th sample, p i is the classification probability value predicted by the classification head module / segmentation probability value predicted by the segmentation head module (usually between [0,1]), N is the total number of samples, and the log base is not limited.

[0080] In the model training of this embodiment, the input size of the pancreatic ultrasound endoscopic image is set to 320*576, and the number of randomly selected images in each batch is set to 32. The training epochs is 100, and the network model with the best performance during the training process is retained. The Adam optimizer is used to update the network weights. The learning rate is controlled by CosineAnnealingLR and gradually decreases with the increase of training sessions. The initial learning rate is set to 0.00001 and the minimum learning rate is set to 0.00000001.

[0081] IV. Lesion Detection Model Optimization

[0082] The trained model is deployed in the clinical environment, and its actual performance may drop significantly, even for a highly accurate deep learning model. Because in the training process, the training data set cannot cover all instances in the real world and the diversity of disease cases. Therefore, the model should be able to adapt to the clinical environment and is self-adaptive without the need for computer experts. For this reason, the lesion detection model of the present invention has a fine-tuning optimization function that adapts to clinical applications. When the lesion detection model output result is inaccurate, the clinician can modify the output result according to his own experience and knowledge, including modifying the classification result and modifying the lesion segmentation area to improve the accuracy of diagnosis. This process is the interaction between the clinician's knowledge and the knowledge of the model, and the data generated in this interaction process is lacking in the model training process. Therefore, the lesion detection model can be iteratively trained to optimize the model through the interaction data between the clinician and the model, so that the model can adapt to complex clinical environments according to the clinician's knowledge and obtain better application effects. Specifically, the last layer of parameters of the lesion detection model is trained using new interaction data, i.e., the parameters of the classification head module and the segmentation head module are modified.

[0083] In addition, in order to better incorporate the expertise of clinicians into the improvement of the model, the present invention chooses a data-driven approach. Specifically, the present invention creates an interface for clinicians to efficiently review model outputs and provide correct labels, and can easily collect new annotated data under the guidance of expert knowledge. Once the amount of data annotated by clinicians reaches one-third of the size of the original training data, the model is retrained using two data sets, where the weight of the clinician-annotated data is twice that of the original data.

[0084] Experimental test:

[0085] The lesion detection model of the present invention is compared with the existing models ResUNet, ResuNet++, ResUNet (Cross-Stic), ResUNet (Cross-Stic), and ResUNet++ (Cross-Stich) on the above-mentioned test data set. The test indicators include accuracy (Micro-AC), average accuracy (Macro-ACC), average precision (Macro-PRE), average intersection over union (MIoU) and delay (Delay). The test results are shown in Table 1.

[0086] Table 1 Model performance test results

[0087] Model Micro-AC Macro-ACC Macro-PRE MIoU Delay(ms) ResUNet 0.8170 0.7425 0.9659 0.6454 16.66 ResuNet++ 0.7857 0.7385 0.9702 0.6349 19.42 ResUNet(Cross-Stic) 0.8259 0.8494 0.9748 0.6843 24.41 ResUNet++(Cross-Stich) 0.8393 0.8638 0.9846 0.6728 20.35 The present invention 0.9286 0.8951 0.9931 0.7671 15.62

[0088] It can be seen from the test results in Table 1 that the lesion detection model of the present invention is superior to these existing models in all indicators and has the best performance.

[0089] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. A method for detecting lesions in pancreatic endoscopic ultrasound images, characterized in that: The following steps are involved: 1) Construct a pancreatic ultrasound endoscopic image dataset with real labels of lesion areas and types and use it as a training dataset; 2) Construct a lesion detection model, which includes a residual neural network ResNet, a feature pyramid network FPN, a pyramid pooling module PPM, a fusion module Fuse, a classification head module and a segmentation head model; The ResNet network is used to extract multi-scale features of pancreatic ultrasound endoscopic images; The FPN network is used to encode multi-scale features and generate multi-scale coded features; The PPM module is used to process the multi-scale coding features and generate a feature map containing multi-scale information; The Fuse module is used to fuse the feature map generated by the PPM module with the multi-scale coding features generated by the FPN network to generate a fused feature map; The classification head module is used to classify diseases based on the feature maps generated by the PPM module; The segmentation head module is used to segment the lesion area in the pancreatic ultrasound endoscopic image according to the fusion feature map; 3) Use the training data set to train the lesion detection model, construct a multi-task loss function consisting of a classification loss function and a segmentation loss function, and optimize the model parameters with the minimum multi-task loss as the objective function; 4) Use the trained lesion detection model to process pancreatic ultrasound endoscopic images to generate disease classification results and lesion area segmentation results in the image.

2. The method according to claim 1, characterized in that The processing steps of the PPM module include: Perform parallel pooling on the input multi-scale features to generate multi-scale deep features; Perform convolution on the deep features generated by each pooling layer to generate multiple abstract features; Interpolate the abstract features generated by each convolution to the same size as the input multi-scale features; The interpolated multiple features are concatenated with the input multi-scale features in the depth direction to generate a feature map containing multi-scale information.

3. The method according to claim 1, characterized in that The feature map generated by the PPM module is interpolated into a feature of the same size as the multi-scale coding feature generated by the FPN network using a bilinear method, and the feature generated by the difference is connected with the coding feature to generate a new feature map, which is input into the Fuse module to be fused with the multi-scale coding feature generated by the FPN network.

4. The method according to claim 1, characterized in that Both the classification loss function and the segmentation loss function use the BCE loss function Among them, y i is the true label of the i-th sample in the training dataset, p i is the classification probability value predicted by the classification head module or the segmentation probability value predicted by the segmentation head module, N is the total number of samples, and the log base is not limited.

5. The method according to claim 1, characterized in that When the trained lesion detection model is used in a clinical environment, the generated disease classification results and image lesion area segmentation results are modified and re-input into the model as fine-tuning data for fine-tuning training to optimize the parameters of the classification head module and the segmentation head module.

6. A device for detecting lesions in pancreatic endoscopic ultrasound images, implementing the method according to any one of claims 1 to 5, characterized in that: include: A trained lesion detection model includes a residual neural network ResNet, a feature pyramid network FPN, a pyramid pooling module PPM, a fusion module Fuse, a classification head module and a segmentation head model; wherein the ResNet network is used to extract multi-scale features of pancreatic ultrasound endoscopic images; the FPN network is used to encode multi-scale features and generate multi-scale coding features; the PPM module is used to process multi-scale coding features and generate a feature map containing multi-scale information; the Fuse module is used to fuse the feature map generated by the PPM module with the multi-scale coding features generated by the FPN network to generate a fused feature map; the classification head module is used to classify diseases according to the feature map generated by the PPM module; and the segmentation head module is used to segment the lesion area in the pancreatic ultrasound endoscopic image according to the fused feature map.

7. The system according to claim 6, characterized in that PPM modules include: Multiple parallel pooling layers: used to process multi-scale features of the input and generate multi-scale deep features; Convolutional layer: Each pooling layer is followed by a convolutional layer to process deep features and extract abstract features; Interpolation layer: used to adjust the extracted abstract features to the same size as the input multi-scale features; Splicing layer: It is used to splice the multiple features adjusted by the interpolation layer with the input multi-scale features in the depth direction to generate a feature map containing multi-scale information.

8. The system according to claim 6, characterized in that The Fuse module includes multiple convolution blocks, each of which is used to process features of different scales.

9. The system according to claim 6, characterized in that The classification head module consists of a convolutional layer, an adaptive global average pooling layer, and a convolutional layer for disease classification.

10. The system according to claim 6, characterized in that The segmentation head module includes a convolutional layer for segmenting the lesion region in the image.

Images