A pathological myopia analysis method and system based on multi-task learning

Abstract

The application relates to the technical field of medical image segmentation, and particularly discloses a pathological myopia analysis method and system based on multi-task learning, which comprises the following steps: inputting a pathological myopia fundus image into a shared feature encoder to extract features and obtain an image embedding vector; the shared feature encoder is based on a Swin Transformer architecture and dynamically adjusts the rank value of LoRA parameters through information entropy; the image embedding vector is subjected to feature enhancement through a layered progressive head attention module to obtain an enhanced image embedding vector; and the enhanced image embedding vector is input into a multi-task decoding head to obtain an analysis result. The application can significantly improve the diagnosis accuracy and calculation efficiency, provides strong technical support for the automatic diagnosis and personalized treatment of ophthalmic diseases, and promotes the wide application of intelligent medical technology in clinical practice.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image segmentation, in particular to a pathological myopia analysis method and system based on multi-task learning. BACKGROUND

[0002] In the field of medical image analysis, pathological myopia is one of the main causes of irreversible visual loss worldwide. Its accurate multi-task analysis is crucial for early diagnosis, treatment decision-making, and prognosis evaluation. In the development of deep learning technology, convolutional neural networks such as UNet and its improved versions UNet++, Attr-UNet have made significant achievements in medical image segmentation. With the rise of visual Transformer (ViT), researchers have begun to apply Transformer architecture to medical image analysis. In recent years, multi-task learning has shown significant efficiency and generalization advantages in medical image analysis by sharing feature representations to handle different tasks simultaneously. Existing multi-task medical models such as DeepUNet and ACSNet can extract local lesion features, and Transformer models can enhance feature representation capabilities in complex scenarios by modeling global dependencies. However, these models often have large parameter quantities and high computational requirements, limiting their deployment capabilities in resource-constrained clinical environments.

[0003] Existing multi-task medical image analysis methods achieve comprehensive analysis by combining three core components: first, a shared encoder uses a deep neural network to process input medical images and create a unified feature embedding representation. Second, task-specific heads convert shared features into the output format required by each task, including classification heads, segmentation heads, and question-and-answer heads. Finally, a loss fusion module combines the loss functions of different tasks to optimize the overall model performance, usually using weighted averaging or dynamic weight adjustment strategies. The following disadvantages exist:

[0004] (1) Static parameter configuration leads to low efficiency: existing parameter-efficient fine-tuning methods such as LoRA use fixed rank configurations, which cannot dynamically adjust parameter allocation according to different data characteristics and task complexity. This static configuration, when faced with complex medical images such as pathological myopia, either underfits due to insufficient parameters or overfits due to redundant parameters, ultimately leading to 30-50% parameter waste and performance degradation.

[0005] (2) Serious conflict between task-specific feature representations: existing multi-task learning methods use simple feature sharing strategies, lacking adaptive feature enhancement mechanisms for different task requirements. Classification tasks require global semantic features, segmentation tasks require fine boundary features, and visual question-and-answer requires multi-modal fusion features. This difference in requirements leads to mutual interference between tasks, causing error propagation and performance degradation.

[0006] (3) Insufficient ability to capture complex lesion features: Traditional attention mechanisms cannot effectively handle multi-scale lesion features in pathological myopia images. Retinal atrophy regions have multi-level features ranging from pixel-level minor changes to organ-level structural deformations. Existing methods lack hierarchical feature modeling capabilities, resulting in insufficient recognition accuracy for complex anatomical structures and pathological changes.

[0007] (4) Lack of multi-task collaborative optimization mechanism: Existing solutions typically use simple loss weighting strategies for multi-task optimization, which cannot fully utilize the inherent relevance between tasks. Classification results can guide segmentation region positioning, and segmentation masks can enhance spatial understanding for question answering, but existing technologies lack effective cross-task information interaction and collaborative enhancement mechanisms.

[0008] (5) Excessive computational resource consumption limits clinical deployment: Existing multi-task models generally have large parameter quantities and high computational complexity, making real-time processing difficult in resource-constrained clinical environments. The self-attention computation complexity of large Transformer models is O(n²), which faces serious computational bottlenecks when processing high-resolution medical images. SUMMARY

[0009] The present invention aims to solve the above problems. To this end, the present invention provides a pathological myopia analysis method and system based on multi-task learning, which dynamically optimizes the rank value of LoRA parameters through information entropy, achieving adaptive balance of parameter efficiency and model performance, significantly improving diagnostic accuracy and computational efficiency, providing strong technical support for automated diagnosis and personalized treatment of ophthalmic diseases, and promoting the widespread application of intelligent medical technology in clinical practice.

[0010] The present invention provides a pathological myopia analysis method based on multi-task learning, which adopts the following technical solutions:

[0011] S1: Input the pathological myopia fundus image into the shared feature encoder for feature extraction to obtain an image embedding vector; the shared feature encoder is based on the Swin Transformer architecture and dynamically adjusts the rank value of LoRA parameters through information entropy;

[0012] S2: Perform feature enhancement on the image embedding vector through the hierarchical progressive head attention module to obtain an enhanced image embedding vector;

[0013] S3: Input the enhanced image embedding vector into the multi-task decoding head to obtain an analysis result.

[0014] Further, the shared feature encoder includes a block embedding layer and a plurality of Transformer modules connected in sequence.

[0015] Further, the process of dynamically adjusting the rank value of the LoRA parameter through information entropy is as follows:

[0016] Calculate the entropy of the A matrix and the B matrix respectively, and then calculate the total entropy, wherein the A matrix and the B matrix are trainable low-rank matrices;

[0017] Starting from the current rank, iteratively reduce the rank, calculate the entropy of the corresponding A truncated matrix and B truncated matrix respectively, compare the sum of the two with the total entropy, find the smallest rank that can retain the specified total entropy ratio, and obtain the optimal rank;

[0018] Adjust the A matrix and the B matrix using the optimal rank, and calculate the low-rank increment according to the adjusted A matrix and the B matrix.

[0019] Further, the optimal ranks of all LoRA layers are calculated, and the median is taken as the target rank, and the A matrix and the B matrix of all LoRA layers are adjusted using the target rank.

[0020] Further, by normalizing the absolute value of the matrix element to a probability distribution, the entropy of the A matrix and the B matrix is calculated respectively; the total entropy is the sum of the entropy of the A matrix and the entropy of the B matrix.

[0021] Further, a low-rank increment is introduced into the pre-training weight matrix of the shared feature encoder, and forward propagation is performed for training; after each training, the performance of the shared feature encoder is verified, and if the performance of multiple consecutive training cycles does not improve, the rank value of the LoRA parameter is dynamically adjusted through information entropy.

[0022] Further, the hierarchical progressive head attention module includes two levels of progressive attention mechanisms; the first level of attention mechanism adopts an adaptive pooling operation to establish global pathological feature consistency through cross-head information fusion; the second level of attention mechanism performs statistical pooling along the spatial correlation dimension to capture statistical information of the correlation strength between each spatial position and other positions.

[0023] Further, the processing process of the hierarchical progressive head attention module includes:

[0024] S2.1: Obtain the attention weight matrix in the form of the head according to the image embedding vector;

[0025] S2.2: The attention weight matrix in the form of the head is obtained through the first level of attention mechanism to obtain the attention weight matrix in the form of space;

[0026] The processing process includes: applying maximum pooling and average pooling to the attention weight matrix in the form of the head along the head dimension, then connecting along the head dimension, and then generating global pathological attention weight through a convolutional network; the global pathological attention weight acts on all attention heads to obtain the attention weight matrix in the form of space;

[0027] S2.3: The spatial form attention weight matrix is enhanced by a second-level attention mechanism to obtain an enhanced image embedding vector;

[0028] The processing process includes: the spatial form attention weight matrix is subjected to maximum pooling and average pooling operations along the spatial correlation dimension, and then anatomical structure attention weights are generated through a connection and convolution network; the anatomical structure attention weights are applied to the spatial form attention weight matrix to obtain hierarchical refined attention weights; and the enhanced image embedding vector is obtained according to the hierarchical refined attention weights and the value matrix.

[0029] Further, the multi-task decoding head includes a classification head, a segmentation head and a visual question and answer head.

[0030] The enhanced image embedding vector is input into the classification head, the segmentation head and the visual question and answer head respectively; the classification head outputs a pathological myopia severity classification result, the segmentation head outputs an optic disc segmentation mask, and the visual question and answer head outputs a medical question and answer result. The analysis result includes the pathological myopia severity classification result, the optic disc segmentation mask and the medical question and answer result.

[0031] The application also provides a pathological myopia analysis system based on multi-task learning, which adopts the technical scheme as follows: comprising a shared feature encoder, a hierarchical progressive head attention module and a multi-task decoding head,

[0032] The shared feature encoder is used for feature extraction of a pathological myopia fundus image to obtain an image embedding vector; the shared feature encoder is based on a Swin Transformer architecture and dynamically adjusts the rank value of a LoRA parameter through information entropy.

[0033] The hierarchical progressive head attention module is used for feature enhancement of the image embedding vector to obtain an enhanced image embedding vector.

[0034] The multi-task decoding head is used for obtaining an analysis result according to the enhanced image embedding vector.

[0035] The above one or more technical solutions in the embodiments of the application have at least one of the following technical effects:

[0036] 1. A dynamic parameter optimization mechanism is established: the application develops a dynamic LoRA parameter optimization strategy based on information entropy, which can adaptively adjust the rank value of a parameter matrix according to data characteristics, realize 30-50% parameter reduction while maintaining 99% information retention rate, and improve parameter use efficiency.

[0037] 2. Design task-specific feature enhancement method: The invention builds a hierarchical progressive head attention mechanism, providing differentiated feature representation enhancement for different tasks by integrating self-attention and spatial attention, reducing task interference and promoting collaborative learning.

[0038] 3. Develop multi-scale lesion feature modeling technology: The invention designs a hierarchical attention architecture that can model global pathological feature consistency and local anatomical structure details simultaneously, improving the recognition and positioning accuracy of complex lesions such as retinal atrophy.

[0039] 4. Build cross-task collaborative learning framework: The invention establishes a unified multi-task learning framework, realizing the collaborative optimization and mutual promotion of classification, segmentation and visual question answering tasks through the organic combination of shared feature encoders and multi-task decoding heads.

[0040] 5. Balance computational efficiency and performance: The invention significantly reduces computational complexity while maintaining high accuracy through parameter-efficient fine-tuning and attention mechanism optimization, enabling efficient deployment and real-time analysis of the model in resource-constrained clinical environments.

[0041] Additional aspects and advantages of the present invention will be partially given in the following description, some of which will become apparent from the following description, or will be learned by practice of the present invention. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0043] Figure 1 is a flowchart of the method provided by the present invention.

[0044] Figure 2 is a schematic diagram of the system provided by the present invention.

[0045] Figure 3 is a flowchart of the entropy-driven low-rank adaptive tuning strategy provided by the present invention.

[0046] Figure 4 is a schematic diagram of the hierarchical progressive head attention module provided by the present invention.

[0047] Figure 5 is a schematic diagram of the classification task head provided by the present invention.

[0048] Figure 6 is a schematic diagram of the segmentation task head provided by the present invention.

[0049] Figure 7 is a schematic diagram of a visual question answering head provided by the present application.

[0050] Figure 8 is a visualization result diagram provided by the present application. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. The following embodiments are used to illustrate the present application, but cannot be used to limit the scope of the present application.

[0052] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures or characteristics described in combination with the embodiment or example are contained in at least one embodiment or example of the embodiments of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0053] The following will be described in combination with Figures 1 to 8 The present application is further described in detail, and a pathological myopia analysis method and system based on multi-task learning are described:

[0054] In the present embodiment, as Figure 1 shown, a pathological myopia analysis method based on multi-task learning is provided, comprising the following steps:

[0055] S1: inputting a pathological myopia fundus image into a shared feature encoder for feature extraction to obtain an image embedding vector; the shared feature encoder is based on a Swin Transformer architecture, and the rank value of LoRA parameters is dynamically adjusted by information entropy.

[0056] The shared feature encoder is based on an Entropy-driven LoRA Tuning (ELT) enhanced Swin Transformer architecture, which is specifically designed to extract multi-scale feature representations from input pathological myopia fundus images. The Swin Transformer employs a hierarchical window attention mechanism, which can capture local and global dependencies in images at different scales, thus better understanding complex retinal anatomy and pathological changes. In this method, the encoder parameters are dynamically optimized through the ELT strategy, that is, the rank value of the LoRA parameter is adaptively adjusted in an information entropy guided manner, which can significantly reduce the parameter overhead while maintaining the feature expression capability.

[0057] As shown in Figure 2 , the shared feature encoder includes a block embedding layer and a plurality of Transformer modules connected in sequence. Each Transformer module is equipped with a LoRA layer.

[0058] Existing methods usually adopt a fixed LoRA parameter configuration during model fine-tuning, which cannot adaptively adjust the parameter allocation according to the data features and task complexity. However, static rank value settings often have problems of parameter redundancy or insufficient expression capability when facing complex medical images such as pathological myopia. The goal of this method is to allow the LoRA parameter to dynamically adjust the rank value according to the information entropy, achieving an optimal balance between parameter efficiency and model performance. This method introduces a dynamic rank value optimization mechanism based on information entropy in the traditional LoRA framework, achieving intelligent tuning of parameter configuration, thereby achieving a 30-50% reduction in parameters while maintaining 99% information content.

[0059] The entropy-driven LoRA tuning strategy of the present embodiment is shown in Figure 3 . For a linear layer with a weight matrix , , is a real set, is the feature dimension, LoRA introduces a low-rank increment , , where is the A matrix, is the B matrix, , , and , is the rank, is the input feature dimension of the linear layer, is the output feature dimension of the linear layer, is the minimum value. The forward propagation is defined as:

[0060]

[0061] where, is the linear layer output vector, is the linear layer input vector, is the pre-trained weight matrix, and A and B matrices are trainable low-rank matrices. Unlike traditional LoRA that adopts a fixed rank value, ELT dynamically adjusts the rank value based on information entropy . This embodiment freezes the weight matrix during model fine-tuning and adjusts A and B matrices.

[0062] The core idea of ELT is to balance parameter reduction and information preservation, adjusting the rank value adaptively according to the characteristics of the pathological myopia analysis dataset. The adjustment process is triggered when the validation performance stagnates for multiple cycles. The process calculates the optimal rank for each LoRA layer, uses the median of the optimal rank values between layers to ensure robustness, and applies the adjustment when the current rank value is reduced and exceeds the minimum rank value.

[0063] Low-rank increments are introduced into the pre-trained weight matrix of the shared feature encoder, forward propagation, and training. After each training, the performance of the shared feature encoder is verified, and if the performance does not improve for multiple consecutive training cycles, the rank value of the LoRA parameter is dynamically adjusted through information entropy.

[0064] The process of dynamically adjusting the rank value of the LoRA parameter through information entropy is as follows:

[0065] Calculate the entropy of A and B matrices respectively, and then calculate the total entropy. By normalizing the absolute value of the matrix elements as a probability distribution, the entropy of A and B matrices is calculated respectively; the total entropy is the sum of the entropy of A matrix and the entropy of B matrix.

[0066] The entropy of matrix is calculated by normalizing its absolute value as a probability distribution, and the calculation formula is:

[0067]

[0068] where, is the probability distribution of the element in the i-th row and j-th column of matrix , and is the minimum value.

[0069]

[0070] where, is the element in the i-th row and j-th column of matrix , and is the element in the k-th row and l-th column of matrix .

[0071] ​The entropy of the A matrix and the entropy of the B matrix can be calculated according to the above formula.

[0072] Total entropy The calculation formula of the total entropy is wherein, the entropy of the A matrix, the entropy of the B matrix.

[0073] The ELT starts from the current rank, iteratively reduces the rank, respectively calculates the entropy of the corresponding A truncated matrix and B truncated matrix, compares the sum of the two with the total entropy, finds the smallest rank that can retain the specified total entropy ratio, and obtains the optimal rank.

[0074] The optimal rank The calculation formula is:

[0075]

[0076] wherein, the entropy of the A truncated matrix, the entropy of the B truncated matrix, the information retention ratio coefficient, the rank. In the embodiment, is set to 0.99.

[0077] Calculate the optimal rank of all LoRA layers , and take the median as the target rank wherein, the optimal rank of the first LoRA layer, the optimal rank of the second LoRA layer, the optimal rank of the nth LoRA layer. If the target rank is less than or equal to the minimum rank, the A matrix and the B matrix are not adjusted. If the target rank is greater than the minimum rank, the A matrix and the B matrix of all LoRA layers are adjusted using the target rank, and then the low-rank increment is calculated according to the adjusted A matrix and B matrix. In the next round of training of the shared feature encoder, the adjusted low-rank increment is introduced into the pre-trained weight matrix.

[0078] The technical advantages of the ELT strategy are: first, through the dynamic rank value adjustment guided by information entropy, the adaptive balance of parameter efficiency and expression ability is realized; second, 99% of the information content is maintained while reducing 30-50% of the parameters, which significantly improves the calculation efficiency; finally, the adaptability and stability of the model in the resource-limited clinical environment are enhanced, making it particularly suitable for complex medical image analysis tasks such as pathological myopia.

[0079] S2: The image embedding vector is subjected to feature enhancement through the hierarchical progressive head attention module to obtain an enhanced image embedding vector.

[0080] Hierarchical Progressive Head-specific Attention (HPHA) is a key component connecting the shared feature encoder and the multi-task decoding heads. It provides differentiated feature enhancement for different tasks by integrating self-attention and spatial attention mechanisms. HPHA adopts a two-level progressive attention calculation: the first layer establishes global pathological feature consistency modeling, ensuring that the three tasks have a unified understanding of the global feature distribution through cross-head information fusion; the second layer realizes head-specific spatial relationship modeling, and fine-tuned processing of local anatomical structures is performed according to the differentiated needs of different tasks.

[0081] Existing methods usually use a unified attention mechanism to handle multi-task scenarios, which cannot effectively meet the differentiated needs of different tasks for feature representation. In fundus image analysis, the classification task requires global semantic features, the segmentation task requires fine boundary features, and the visual question answering (VQA) task requires multi-modal fusion features. This difference in needs leads to poor performance of traditional attention mechanisms in handling complex multi-task scenarios. This method designs a progressive attention mechanism that can simultaneously meet the needs of global pathological feature consistency modeling and local anatomical structure detail enhancement. HPHA introduces a hierarchical progressive feature enhancement strategy in the attention framework, achieving the organic combination of global consistency and task-specificity through two-level attention calculation, thereby significantly improving the quality and adaptability of multi-task feature representation.

[0082] The specific structure of the hierarchical progressive head attention module is shown in Figure 4 The working process of HPHA is as follows:

[0083] For the input image embedding vector , , is the height, is the width, is the number of feature channels, first generate the query, key and value shared by multiple tasks through linear transformation. The spatial features are reshaped into a sequence form and decomposed into multi-head query , , where is the number of attention heads, is the feature dimension of a single attention head, , is the sequence length, i.e. the total number of spatial positions, .

[0084] Calculate the initial normalized attention weight matrix , the formula is:

[0085]

[0086] in, Let T be the normalization function, and T be the matrix transpose. For multi-headed key, .

[0087] right The attention weight matrix is ​​reshaped to obtain a head-like form. , This serves as the input for the first layer of attention mechanism.

[0088] The first-level attention mechanism aims to establish a consistent global representation of pathological features. Considering that all three tasks require a unified understanding of the global feature distribution, this method performs information fusion at the head dimension. Adaptive pooling is applied along the head dimension. Max pooling captures the feature responses most sensitive to fundus pathological regions in multi-head attention, while average pooling preserves the statistical distribution information of global pathological features. The two pooling features are concatenated along the head dimension and then used to generate global pathological attention weights through a convolutional network.

[0089]

[0090] in, For global pathological attention weights, It is the sigmoid activation function. For convolution operations, The attention weight tensor is the result of max pooling in the head dimension. This is the attention weight tensor after average pooling of the head dimension.

[0091] Level 1 Enhanced Attention Weights Obtained through element-wise multiplication:

[0092]

[0093] in, This is element-wise multiplication. Through broadcasting, it affects all Individual attention points are used to enhance the consistency of global pathological features.

[0094] right Dimensional reshaping is performed to obtain a spatial attention weight matrix. This serves as the input for the second-layer attention mechanism.

[0095] The second-level attention mechanism is specifically designed to handle the differentiated requirements of various tasks in fundus image processing. To maintain head specificity to adapt to these differentiated requirements, statistical pooling is performed along the spatial correlation dimension. The spatial dimension pooling operation captures the statistical information of the correlation strength between each spatial position and all other positions, providing a basis for modeling the differentiated spatial relationships of different task heads.

[0096] The pooling operation along the spatial dimension is obtained by feature connection:

[0097]

[0098] wherein, is the spatial dimension correlation attention weight tensor, is the spatial dimension maximum-pooled attention weight tensor, is the spatial dimension average-pooled attention weight tensor.

[0099] Then, is reshaped to , is the reshaped spatial dimension correlation attention weight tensor, .

[0100] Generating refined anatomical structure attention weights through a convolutional network :

[0101]

[0102] wherein, is the bias vector.

[0103] Reshaping back to a form compatible with the original tensor, the reshaped anatomical structure attention weight is obtained, .

[0104] The second-level enhanced hierarchical refined attention weight is: .

[0105] Reshaping into a standard multi-head attention form, the reshaped hierarchical refined attention weight is obtained, and then weighted aggregated with the value matrix:

[0106]

[0107] wherein, is the weighted feature, is the multi-head form value.

[0108] After dimension reorganization and linear projection of multi-head output, an enhanced image embedding vector is obtained according to the weighted features , .

[0109] The technical advantages of the HPHA mechanism are as follows: first, the hierarchical and progressive design meets the requirements of global consistency and task specificity at the same time; second, the first-level global consistency modeling establishes a unified pathological feature understanding basis for the three tasks; and finally, the second-level head-specific refinement allows different attention heads to focus on different anatomical structures, realizing fine feature enhancement for different task requirements.

[0110] S3: input the enhanced image embedding vector into the multi-task decoding head to obtain an analysis result.

[0111] The multi-task decoding head architecture includes three specially designed task-specific decoders, including a classification task head, a segmentation task head and a visual question answering head. Each decoding head is independently run, and the shared features enhanced by the HPHA are used to generate task-specific outputs, ensuring efficient and coordinated multi-task performance.

[0112] As shown in Figure 5 , the classification task head adopts the CABNet classifier architecture, which is used for four-class prediction of the severity of pathological myopia. The classifier is used for feature mapping and classification prediction, and outputs a four-class probability distribution to realize the classification results of normal, mild, moderate and severe pathological myopia. The enhanced image embedding vector is input into the classification task head, and the classification result of the severity of pathological myopia is output. As shown in Figure 6 , the segmentation task head is based on the UNet decoder architecture, which restores the spatial resolution layer by layer and fuses multi-scale features through upsampling operation and skip connection, and finally generates a pixel-level optic disc segmentation mask to realize pixel-level accurate segmentation of the optic disc region. The enhanced image embedding vector is input into the segmentation task head, and the optic disc segmentation mask is output. As shown in Figure 7 , the visual question answering head adopts the Surgical-VQA decoder, which combines image features and text questions for multi-modal reasoning, aligns image features and text semantics through attention mechanism, and outputs a natural language answer. The enhanced image embedding vector is input into the visual question answering head, and the medical question answering result is output. The analysis result includes the classification result of the severity of pathological myopia, the optic disc segmentation mask and the medical question answering result.

[0113] The effectiveness of the present application is verified by the following test.

[0114] The method freezes the pre-training parameters of the shared feature encoder when training the multi-task learning model, and realizes efficient fine-tuning through the ELT strategy to dynamically adjust the LoRA parameters; the parameters of the HPHA module and the multi-task decoding head are mainly trained. In this way, rich visual features learned on large-scale datasets can be utilized, while the parameters are efficiently fine-tuned to adapt to the pathological myopia analysis task. All input images are uniformly adjusted to a resolution of 224x224. The Swin Transformer (SwinTransformer) is selected as the basic architecture of the shared feature encoder. The learning rate is set to 5x10 -5 , and the maximum training period is set to 100. The learning rate decay adopts a cosine annealing strategy, and the warm-up period is 5 cycles. The batch size is set to 60. The AdamW optimizer is used for optimization, and the weight decay is set to 1x10 -8 . The initial rank of the entropy-driven low-rank adaptive tuning strategy is set to 64, the minimum rank is 4, the information retention ratio is 0.99, and the rank value adjustment is triggered after 10 cycles of validation performance stagnation. In order to enhance the training stability and improve the comprehensive performance of the model, the cross-entropy loss is used for classification and visual question answering tasks, and the combination of cross-entropy loss and Dice loss is used for segmentation tasks. All experiments are performed on NVIDIA H100 GPUs with 80GB of video memory.

[0115] The present embodiment uses the constructed Pathological Myopia Multi-task Analysis (PMMA) dataset to verify the effectiveness of the present method and several existing multi-task learning methods. In the multi-task learning framework, different parameter efficient fine-tuning strategies and attention mechanisms are compared, and the performance of different strategies in pathological myopia analysis tasks is evaluated. The baseline model is trained completely in the multi-task learning framework, and this result is used as a reference for comparison.

[0116] The experimental results are shown in Tables 1, 2 and 3. The performance indicators of the classification task include accuracy, precision, recall and F1 score, where accuracy is the ratio of correctly classified samples to the total number of samples, precision is the ratio of samples predicted as positive and actually positive to the number of samples predicted as positive, recall is the ratio of samples predicted as positive and actually positive to the number of samples actually positive, and F1 score is the harmonic mean of precision and recall. The performance indicators of the segmentation task include Dice coefficient (Dice) and Hausdorff distance (HD), where Dice coefficient is the ratio of the overlapping part between the predicted foreground region and the real foreground region to their average, and Hausdorff distance is used to measure the accuracy of the segmentation boundary. The performance indicators of the VQA task include accuracy, precision, recall and F1 score, which measure the model's understanding and reasoning ability for medical questions and answers.

[0117] Table 1 Performance comparison results of classification task

[0118]

[0119] Table 1 shows the comparison results of the proposed method and other state-of-the-art methods. Among the other state-of-the-art methods, single-task models use ConvNeXt, ViT, FlexiViT, CaiT, SwinT, SwinT V2, CABNet, FedIIC, and HiFuse, and multi-task models use DeepUNet and ACSNet. In the classification task, the proposed method achieves the highest accuracy of 83.95%, surpassing all baseline methods. Most notably, the proposed method performs outstandingly in recall, reaching 76.05%, which is significantly better than the best baseline method SwinT of 70.05%, with an improvement of 6.00 percentage points, which is crucial for reducing the risk of missed diagnosis in clinical applications. The F1 score result further verifies the effectiveness of the proposed method, reaching 77.41%, which is 7.05 percentage points higher than ConvNeXt of 70.36%. Compared with multi-task learning methods, the proposed method significantly outperforms DeepUNet and ACSNet, with an accuracy improvement of 5.42% and 4.77%, respectively, and an F1 score improvement of 9.04% and 8.18%, respectively.

[0120] Table 2 Performance comparison results on segmentation task

[0121]

[0122] Table 2 shows the comparison results on optic disc segmentation task.

[0123] For the Dice index, the proposed method shows excellent performance, significantly outperforming UNet, Att-UNet, CENet, and UNeXt, etc. architectures compared to the traditional CNN method, with improvement ranging from 2.52% to 8.48%. Compared to the Transformer-based method, the proposed method maintains its advantages, especially surpassing Med-SA and SAM-Med2D; among them, the proposed method improves by 0.78% compared to the best-performing SAM-Med2D. Compared to the KAN-based method, the proposed method surpasses U-KAN, MedVKAN, AttUKAN, and TransUKAN, with a range of 1.08% to 2.20%.

[0124] For the HD index, the proposed method achieves 3.29, which is slightly higher than the optimal 3.09 of Med-SA, but remains competitive and indicates excellent boundary accuracy.

[0125] Table 3 Performance comparison results of visual question answering task

[0126]

[0127] Table 3 shows the performance advantage of the proposed method in the classification-based medical VQA task. The proposed method achieves an accuracy of 95.37%, which is 1.08% higher than the 94.29% of the second-best method RAMM. In terms of precision, the proposed method achieves 89.84%, surpassing the 89.53% of RAMM. More importantly, the proposed method achieves a recall rate of 88.20%, which is a significant improvement of 10.59% compared to the best-performing baseline RAMM of 77.61%, which is particularly important for reducing the risk of missed diagnosis. In terms of F1 score, the proposed method obtains 88.75%, which is 5.72% higher than the 83.03% of RAMM, showing a good balance between precision and recall.

[0128] Figure 8 The top six network segmentation results are visualized in terms of average accuracy, where, Figure 8 (a) in Fig. 1 is the original image, Figure 8 (b) in Fig. 1 is the label, Figure 8 (c) in Fig. 1 is Att-Unet, Figure 8 (d) in Fig. 1 is ACSNet, Figure 8 (e) in Fig. 1 is UTransNet, Figure 8 (f) in Fig. 1 is SAM-Med2D, Figure 8 (g) in Fig. 1 is MedVKAN, Figure 8 (h) in Fig. 1 is the proposed method. The experimental results show that the proposed method achieves excellent segmentation performance.

[0129] The embodiment also provides a pathological myopia analysis system based on multi-task learning, as shown in the figure, and the technical scheme is as follows: comprising a shared feature encoder, a hierarchical progressive head attention module (HPHA) and a multi-task decoding head, Figure 2

[0130] The shared feature encoder is used for feature extraction of a pathological myopia fundus image, and an image embedding vector is obtained; the shared feature encoder is based on a Swin Transformer architecture, and the rank value of a LoRA parameter is dynamically adjusted based on information entropy;

[0131] The hierarchical progressive head attention module is used for feature enhancement of the image embedding vector, and an enhanced image embedding vector is obtained;

[0132] The multi-task decoding head is used for obtaining an analysis result according to the enhanced image embedding vector.

[0133] The entropy-driven low-rank adaptive tuning (ELT) strategy designed in the application dynamically optimizes the rank value of the LoRA parameter based on information entropy, and compared with the traditional fixed rank value configuration method, the parameter allocation can be adaptively adjusted according to the data characteristics and the task complexity. This design enables the model to intelligently balance the parameter efficiency and the expression ability when facing complex medical images such as pathological myopia, and avoids the problems of parameter redundancy or insufficient expression, so the method is more suitable for processing high-precision medical image analysis tasks in a resource-limited environment. Through this dynamic parameter optimization method, the model not only maintains 99% of the information content, but also realizes 30-50% reduction of the parameters, significantly reducing the computational overhead and deployment cost.

[0134] The hierarchical progressive head-specific attention (HPHA) mechanism designed in the application processes the multi-task feature requirements through two-level progressive attention calculation, and compared with the feature processing method of the traditional unified attention mechanism, the difference requirement of the feature representation of different tasks can be more effectively met. This hierarchical progressive processing method enables the model to simultaneously establish global pathological feature consistency and task-specific enhancement, thereby improving the recognition accuracy of the model for multi-scale lesion features in complex fundus images and the task coordination ability. In addition, the dual mechanism of self-attention and spatial attention further enhances the discriminability of the features, so that the model can focus on the key areas of retinal lesions, and therefore our method has higher multi-task analysis accuracy.

[0135] ​The multitask cooperative learning framework designed by the present application can simultaneously process classification, segmentation and visual question answering tasks through a unified architecture, compared with the traditional independent task learning method, the internal correlation and synergistic effect between tasks can be more fully utilized. This cooperative learning method enables the classification result to guide the segmentation region positioning, the segmentation mask to enhance the spatial understanding of the question and answer, and the visual question and answer to provide semantic verification, thereby forming a benign mutual promotion mechanism between tasks, so the method is significantly superior to the single task method in comprehensive performance. Through this cross-task cooperative optimization, the model not only improves the accuracy of a single task, but also enhances the reliability and robustness of overall analysis.

[0136] The present application comprehensively uses the entropy-driven low-rank adaptive tuning strategy, the hierarchical progressive head-specific attention mechanism and the multitask cooperative learning framework, which can effectively utilize advanced parameter optimization techniques and attention mechanisms, and significantly improve the computational efficiency, feature expressiveness and analysis accuracy compared with existing medical image analysis methods. Through these technical innovations, the present application not only realizes high precision and high efficiency of pathological myopia multitask analysis, but also has better parameter economy and clinical deployment adaptability, so that it has significant technical advantages and practical value in intelligent ophthalmic diagnosis applications.

[0137] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for analyzing pathological myopia based on multi-task learning, characterized in that, Includes the following steps: S1: Input the pathological myopia fundus image into the shared feature encoder for feature extraction to obtain the image embedding vector; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy; S2: The image embedding vector is enhanced by passing it through a hierarchical progressive head attention module to obtain an enhanced image embedding vector; The processing steps of the layered progressive head attention module include: S2.1: Obtain the attention weight matrix in head form based on the image embedding vector; S2.2: The head-shaped attention weight matrix is ​​used to obtain the spatial attention weight matrix through the first-level attention mechanism; The processing steps include: applying max pooling and average pooling along the head dimension to the attention weight matrix in head form, then concatenating along the head dimension, and then generating global pathological attention weights through a convolutional network; the global pathological attention weights are applied to all attention heads to obtain a spatial attention weight matrix; S2.3: The spatial form of the attention weight matrix is ​​used to obtain an enhanced image embedding vector through a second-level attention mechanism; The processing steps include: performing max pooling and average pooling operations along the spatially relevant dimensions on the spatial attention weight matrix, and then generating anatomical structure attention weights through connection and convolutional networks; applying the anatomical structure attention weights to the spatial attention weight matrix to obtain hierarchical refinement attention weights; and obtaining enhanced image embedding vectors based on the hierarchical refinement attention weights and the value matrix. S3: Input the enhanced image embedding vector into the multi-task decoding head to obtain the analysis results.

2. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, The shared feature encoder includes a block embedding layer and multiple Transformer modules connected in sequence.

3. A method for analyzing pathological myopia based on multi-task learning as described in claim 1 or 2, characterized in that, The process of dynamically adjusting the rank of the LoRA parameter using information entropy is as follows: Calculate the entropy of matrix A and matrix B respectively, and then calculate the total entropy, where matrix A and matrix B are trainable low-rank matrices; Starting from the current rank, iteratively decrease the rank, calculate the entropy of the corresponding A-truncation matrix and B-truncation matrix respectively, compare the sum of the two with the total entropy, find the smallest rank that can retain a specified proportion of the total entropy, and obtain the optimal rank; Adjust matrices A and B using the optimal rank, and calculate the low-rank increment based on the adjusted matrices A and B.

4. The method for analyzing pathological myopia based on multi-task learning as described in claim 3, characterized in that, Calculate the optimal rank of all LoRA layers, take the median as the target rank, and adjust the A and B matrices of all LoRA layers using the target rank.

5. The method for analyzing pathological myopia based on multi-task learning as described in claim 3, characterized in that, By normalizing the absolute values ​​of the matrix elements to a probability distribution, the entropy of matrix A and matrix B are calculated separately; the total entropy is the sum of the entropy of matrix A and the entropy of matrix B.

6. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, A low-rank increment is introduced into the pre-trained weight matrix of the shared feature encoder, and forward propagation is performed for training. After each training iteration, the performance of the shared feature encoder is verified. If the performance does not improve over multiple consecutive training cycles, the rank value of the LoRA parameter is dynamically adjusted using information entropy.

7. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, The hierarchical progressive head attention module includes a two-level progressive attention mechanism; the first-level attention mechanism adopts adaptive pooling operation to establish global pathological feature consistency through cross-head information fusion. The second-level attention mechanism performs statistical pooling along the spatial correlation dimension, capturing statistical information on the correlation strength between each spatial location and other locations.

8. A method for analyzing pathological myopia based on multi-task learning as described in claim 1 or 7, characterized in that, Multi-task decoding heads include classification heads, segmentation heads, and visual question-answering heads; The enhanced image embedding vectors are input into the classification head, segmentation head, and visual question answering head, respectively. The classification head outputs the classification results of the severity of pathological myopia, the segmentation head outputs the optic disc segmentation mask, and the visual question-and-answer head outputs the medical question-and-answer results.

9. A pathological myopia analysis system based on multi-task learning, characterized in that, A method for performing pathological myopia analysis based on multi-task learning as described in any one of claims 1 to 8, comprising: a shared feature encoder, a hierarchical progressive head attention module, and a multi-task decoding head. The shared feature encoder is used to extract features from pathological myopia fundus images to obtain image embedding vectors; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy. The hierarchical progressive head attention module is used to perform feature enhancement on the image embedding vector to obtain an enhanced image embedding vector. The multi-task decoding head is used to obtain analysis results based on the enhanced image embedding vector.

Images