Multi-task learning-based pathological myopia analysis method and system

Abstract

The invention relates to the technical field of medical image segmentation, and particularly discloses a pathological myopia analysis method and system based on multi-task learning, and the method comprises the steps: inputting a pathological myopia eye bottom image into a shared feature encoder for feature extraction, and obtaining an image embedding vector; the shared feature encoder is based on a Swin Transform architecture, and the rank value of the LoRA parameter is dynamically adjusted through the information entropy; performing feature enhancement on the image embedding vector through a layered progressive head attention module to obtain an enhanced image embedding vector; and inputting the enhanced image embedding vector into a multi-task solution terminal to obtain an analysis result. According to the method, the diagnosis precision and the calculation efficiency can be remarkably improved, powerful technical support is provided for automatic diagnosis and personalized treatment of ophthalmic diseases, and wide application of the intelligent medical technology in clinical practice is promoted.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and in particular to a method and system for analyzing pathological myopia based on multi-task learning. Background Technology

[0002] In the field of medical image analysis, pathological myopia, as one of the leading causes of irreversible vision loss worldwide, makes accurate multi-task analysis crucial for early diagnosis, treatment decisions, and prognostic assessment. With the development of deep learning technology, convolutional neural networks such as UNet and its improved versions UNet++ and Attr-UNet have achieved significant results in medical image segmentation. With the rise of Visual Transformers (ViT), researchers have begun applying the Transformer architecture to medical image analysis. In recent years, multi-task learning has demonstrated significant efficiency and generalization advantages in medical image analysis by simultaneously processing different tasks through shared feature representations. Existing multi-task medical models such as DeepUNet and ACSNet can extract local lesion features, while Transformer models enhance feature representation capabilities in complex scenes by modeling global dependencies. However, these models often suffer from large parameter counts and high computational demands, limiting their deployment in resource-constrained clinical environments.

[0003] Existing multi-task medical image analysis methods achieve comprehensive analysis by combining three core components: First, a shared encoder uses a deep neural network to process the input medical images and create a unified feature embedding representation. Second, a task-specific head converts the shared features into the output format required for each task, including classification, segmentation, and question-answering heads. Finally, a loss fusion module combines loss functions from different tasks to optimize the overall model performance, typically employing weighted averaging or dynamic weight adjustment strategies. However, these methods suffer from the following drawbacks: (1) Static parameter configuration leads to low efficiency: Existing efficient parameter fine-tuning methods, such as LoRA, use fixed rank configurations and cannot dynamically adjust parameter allocation according to different data characteristics and task complexity. When faced with complex medical images such as pathological myopia, this static configuration either leads to underfitting due to insufficient parameters or overfitting due to parameter redundancy, ultimately resulting in 30-50% parameter waste and performance degradation.

[0004] (2) Severe conflicts in feature representation between tasks: Existing multi-task learning methods adopt simple feature sharing strategies and lack adaptive feature enhancement mechanisms for different task requirements. Classification tasks require global semantic features, segmentation tasks require fine boundary features, and visual question answering requires multimodal fusion features. These differences in requirements lead to mutual interference between tasks, causing error propagation and performance degradation.

[0005] (3) Insufficient ability to capture complex lesion features: Traditional attention mechanisms cannot effectively process multi-scale lesion features in pathological myopic images. Retinal atrophy areas have multi-level features ranging from pixel-level minute changes to organ-level structural deformations. Existing methods lack the ability to model hierarchical features, resulting in insufficient accuracy in recognizing complex anatomical structures and pathological changes.

[0006] (4) Lack of multi-task collaborative optimization mechanism: Existing solutions usually adopt a simple loss weighting strategy for multi-task optimization, which cannot make full use of the inherent correlation between tasks. Classification results can guide the localization of segmented regions, and segmentation masks can enhance the spatial understanding of question answering, but existing technologies lack effective cross-task information interaction and collaborative enhancement mechanisms.

[0007] (5) Excessive computational resource consumption limits clinical deployment: Existing multi-task models generally suffer from a large number of parameters and high computational complexity, making it difficult to achieve real-time processing in resource-constrained clinical environments. The self-attention computational complexity of large Transformer models is O(n²), which faces a serious computational bottleneck when processing high-resolution medical images. Summary of the Invention

[0008] This invention aims to solve the aforementioned problems. To this end, this invention provides a method and system for pathological myopia analysis based on multi-task learning. By dynamically optimizing the rank value of LoRA parameters through information entropy, an adaptive balance between parameter efficiency and model performance is achieved. This significantly improves diagnostic accuracy and computational efficiency, providing strong technical support for the automated diagnosis and personalized treatment of ophthalmic diseases, and promoting the widespread application of intelligent medical technology in clinical practice.

[0009] This invention provides a method for analyzing pathological myopia based on multi-task learning, and the technical solution adopted is as follows: including the following steps: S1: Input the pathological myopia fundus image into the shared feature encoder for feature extraction to obtain the image embedding vector; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy; S2: The image embedding vector is enhanced by passing it through a hierarchical progressive head attention module to obtain an enhanced image embedding vector; S3: Input the enhanced image embedding vector into the multi-task decoding head to obtain the analysis results.

[0010] Furthermore, the shared feature encoder includes a block embedding layer and multiple Transformer modules connected in sequence.

[0011] Furthermore, the process of dynamically adjusting the rank of the LoRA parameter through information entropy is as follows: Calculate the entropy of matrix A and matrix B respectively, and then calculate the total entropy, where matrix A and matrix B are trainable low-rank matrices; Starting from the current rank, iteratively decrease the rank, calculate the entropy of the corresponding A-truncation matrix and B-truncation matrix respectively, compare the sum of the two with the total entropy, find the smallest rank that can retain a specified proportion of the total entropy, and obtain the optimal rank; Adjust matrices A and B using the optimal rank, and calculate the low-rank increment based on the adjusted matrices A and B.

[0012] Furthermore, calculate the optimal rank of all LoRA layers, take the median as the target rank, and use the target rank to adjust the A and B matrices of all LoRA layers.

[0013] Furthermore, by normalizing the absolute values ​​of the matrix elements to a probability distribution, the entropy of matrix A and matrix B are calculated separately; the total entropy is the sum of the entropy of matrix A and the entropy of matrix B.

[0014] Furthermore, a low-rank increment is introduced into the pre-trained weight matrix of the shared feature encoder, and forward propagation is performed for training. After each training, the performance of the shared feature encoder is verified. If the performance does not improve for multiple consecutive training cycles, the rank value of the LoRA parameter is dynamically adjusted through information entropy.

[0015] Furthermore, the hierarchical progressive head attention module includes a two-level progressive attention mechanism; the first-level attention mechanism adopts adaptive pooling operation to establish global pathological feature consistency through cross-head information fusion; the second-level attention mechanism performs statistical pooling along the spatial correlation dimension to capture statistical information on the correlation strength between each spatial location and other locations.

[0016] Furthermore, the processing procedure of the hierarchical progressive head attention module includes: S2.1: Obtain the attention weight matrix in head form based on the image embedding vector; S2.2: The head-shaped attention weight matrix is ​​used to obtain the spatial attention weight matrix through the first-level attention mechanism; The processing steps include: applying max pooling and average pooling along the head dimension to the attention weight matrix in head form, then concatenating along the head dimension, and then generating global pathological attention weights through a convolutional network; the global pathological attention weights are applied to all attention heads to obtain a spatial attention weight matrix; S2.3: The spatial form of the attention weight matrix is ​​used to obtain an enhanced image embedding vector through a second-level attention mechanism; The processing steps include: performing max pooling and average pooling operations along the spatially relevant dimensions on the attention weight matrix in spatial form, and then generating anatomical structure attention weights through connection and convolutional networks; applying the anatomical structure attention weights to the spatial form attention weight matrix to obtain hierarchical refinement attention weights; and obtaining enhanced image embedding vectors based on the hierarchical refinement attention weights and the value matrix.

[0017] Furthermore, the multi-task decoding head includes a classification head, a segmentation head, and a visual question-answering head; The enhanced image embedding vectors are input to the classification head, segmentation head, and visual question-answering head, respectively. The classification head outputs the classification result of the severity of pathological myopia, the segmentation head outputs the optic disc segmentation mask, and the visual question-answering head outputs the medical question-answering result. The analysis results include the classification result of the severity of pathological myopia, the optic disc segmentation mask, and the medical question-answering result.

[0018] This invention also provides a pathological myopia analysis system based on multi-task learning, employing the following technical solution: a shared feature encoder, a hierarchical progressive head attention module, and a multi-task decoding head. The shared feature encoder is used to extract features from pathological myopia fundus images to obtain image embedding vectors; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy. The hierarchical progressive head attention module is used to perform feature enhancement on the image embedding vector to obtain an enhanced image embedding vector. The multi-task decoding head is used to obtain analysis results based on the enhanced image embedding vector.

[0019] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: 1. Establishing a dynamic parameter optimization mechanism: This invention develops a dynamic LoRA parameter tuning strategy based on information entropy, which can adaptively adjust the rank of the parameter matrix according to data characteristics, achieving a 30-50% reduction in parameters while maintaining a 99% information retention rate, thereby improving parameter utilization efficiency.

[0020] 2. Design of task-specific feature enhancement methods: This invention constructs a hierarchical progressive head attention mechanism, which integrates self-attention and spatial attention to provide differentiated feature representation enhancement for different tasks, reduce interference between tasks and promote collaborative learning.

[0021] 3. Development of multi-scale lesion feature modeling technology: This invention designs a hierarchical attention architecture that can simultaneously model global pathological feature consistency and local anatomical structural details, improving the accuracy of identification and localization of complex lesions such as retinal atrophy.

[0022] 4. Constructing a cross-task collaborative learning framework: This invention establishes a unified multi-task learning framework, which achieves collaborative optimization and mutual promotion of classification, segmentation and visual question answering tasks through the organic combination of shared feature encoders and multi-task decoders.

[0023] 5. Achieving a balance between computational efficiency and performance: This invention significantly reduces computational complexity while maintaining high accuracy through efficient parameter fine-tuning and attention mechanism optimization, enabling the model to be efficiently deployed and analyzed in real time in resource-constrained clinical environments.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the method provided by the present invention.

[0027] Figure 2 This is a schematic diagram of the system provided by the present invention.

[0028] Figure 3 This is a flowchart of the entropy-driven low-rank adaptive tuning strategy provided by the present invention.

[0029] Figure 4 This is a schematic diagram of the layered progressive head attention module provided by the present invention.

[0030] Figure 5 This is a schematic diagram of the classification task head provided by the present invention.

[0031] Figure 6 This is a schematic diagram of the segmentation task head provided by the present invention.

[0032] Figure 7 This is a schematic diagram of the visual question-and-answer head provided by the present invention.

[0033] Figure 8 This is a visualization result diagram provided by the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0035] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0036] The following is combined with Figures 1 to 8 The present invention will be further described in detail below, including a method and system for analyzing pathological myopia based on multi-task learning: In this embodiment, as Figure 1 As shown, a method for analyzing pathological myopia based on multi-task learning is provided, including the following steps: S1: Input the pathological myopia fundus image into the shared feature encoder for feature extraction to obtain the image embedding vector; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy.

[0037] The shared feature encoder is based on an entropy-driven LoRA Tuning (ELT) enhanced Swin Transformer architecture, specifically designed to extract multi-scale feature representations from input pathological myopia fundus images. The Swin Transformer employs a hierarchical window attention mechanism, capable of capturing local and global dependencies at different scales, thus providing a better understanding of complex retinal anatomy and pathological changes. In this method, the encoder parameters are dynamically optimized using an ELT strategy, adaptively adjusting the rank of the LoRA parameters in a manner guided by information entropy, which significantly reduces parameter overhead while maintaining feature expressiveness.

[0038] like Figure 2As shown, the shared feature encoder includes a block embedding layer and multiple Transformer modules connected in sequence. Each Transformer module is equipped with a LoRA layer.

[0039] Existing methods typically employ fixed LoRA parameter configurations during model fine-tuning, failing to adaptively adjust parameter allocations based on data features and task complexity. However, static rank settings often suffer from parameter redundancy or insufficient expressive power when dealing with complex medical images such as pathological myopia. This method aims to enable LoRA parameters to dynamically adjust their rank values ​​based on information entropy, achieving an optimal balance between parameter efficiency and model performance. This approach introduces a dynamic rank optimization mechanism based on information entropy into the traditional LoRA framework, enabling intelligent tuning of parameter configurations and achieving a 30-50% parameter reduction while maintaining 99% of the information content.

[0040] The entropy-driven low-rank adaptive tuning strategy in this embodiment, such as Figure 3 As shown. For the weight matrix linear layer, , For the set of real numbers, For the feature dimension, LoRA introduces a low-rank increment. , ,in, Let A be the matrix. Let B be the matrix. , ,and , For rank, The input feature dimension is the linear layer. The output feature dimension of the linear layer. To obtain the minimum value, forward propagation is defined as: in, For the output vector of the linear layer, For linear layer input vectors, The weights are pre-trained matrices, and matrices A and B are trainable low-rank matrices. Unlike traditional LoRA which uses fixed-rank values, ELT dynamically adjusts the weights based on information entropy. Value. In this embodiment, the weight matrix is ​​frozen during model fine-tuning. Adjust matrices A and B.

[0041] The core idea of ​​ELT is to balance parameter reduction with information preservation, adaptively adjusting the rank value based on the characteristics of the pathological myopia analysis dataset. The adjustment process is triggered when validation performance stagnates for several cycles. This process calculates the optimal rank for each LoRA layer, uses the median of the optimal rank values ​​across layers to ensure robustness, and applies adjustment when reducing the current rank value exceeds the minimum rank value.

[0042] A low-rank increment is introduced into the pre-trained weight matrix of the shared feature encoder, and training is performed via forward propagation. After each training iteration, the performance of the shared feature encoder is verified. If the performance does not improve over multiple consecutive training cycles, the rank value of the LoRA parameter is dynamically adjusted using information entropy.

[0043] The process of dynamically adjusting the rank of the LoRA parameter using information entropy is as follows: Calculate the entropy of matrix A and matrix B separately, and then calculate the total entropy. The entropy of matrix A and matrix B is calculated by normalizing the absolute values ​​of the matrix elements to a probability distribution; the total entropy is the sum of the entropy of matrix A and matrix B.

[0044] matrix entropy The probability distribution is calculated using its normalized absolute value as the probability distribution, and the calculation formula is as follows: in, For matrix The probability distribution of the element in the i-th row and j-th column. It is a local minimum.

[0045] in, For matrix The element in the i-th row and j-th column, For matrix The element in the k-th row and l-th column.

[0046] The entropy of matrix A and the entropy of matrix B can be calculated using the above formula.

[0047] Total Entropy The calculation formula is: ,in, Let A be the entropy of matrix A. Let be the entropy of matrix B.

[0048] Starting from the current rank, ELT iteratively decreases the rank by calculating the entropy of the corresponding A-truncation matrix and B-truncation matrix, comparing the sum of the two with the total entropy, and finding the rank that retains the smallest proportion of the specified total entropy, thus obtaining the optimal rank.

[0049] Optimal Rank The calculation formula is: in, Let A be the entropy of the truncated matrix. The entropy of the B-truncated matrix. To retain the proportion of information, Rank. In this embodiment, Set it to 0.99.

[0050] Calculate the optimal rank of all LoRA layers Take the median as the target rank ,in, The optimal rank of the first LoRA layer. The optimal rank of the second LoRA layer. Let be the optimal rank of the nth LoRA layer. If the target rank is less than or equal to the minimum rank, then matrices A and B are not adjusted. If the target rank is greater than the minimum rank, then matrices A and B of all LoRA layers are adjusted using the target rank, and then the low-rank increment is calculated based on the adjusted matrices A and B. During the next round of training of the shared feature encoder, the adjusted low-rank increment is introduced into the pre-trained weight matrix.

[0051] The technical advantages of the ELT strategy are as follows: First, by adjusting the dynamic rank value guided by information entropy, an adaptive balance between parameter efficiency and expressive power is achieved; second, by maintaining 99% of the information content while reducing parameters by 30-50%, computational efficiency is significantly improved; and finally, the model's adaptability and stability in resource-constrained clinical environments are enhanced, making it particularly suitable for analyzing complex medical images such as pathological myopia.

[0052] S2: The image embedding vector is enhanced by using a hierarchical progressive head attention module to obtain an enhanced image embedding vector.

[0053] The Hierarchical Progressive Head-Specific Attention (HPHA) module, as a key component connecting the shared feature encoder and the multi-task decoder, provides differentiated feature enhancements for different tasks by integrating self-attention and spatial attention mechanisms. HPHA employs a two-level progressive attention computation: the first layer establishes a consistent model of global pathological features, ensuring a unified understanding of the global feature distribution across the three tasks through cross-head information fusion; the second layer implements head-specific spatial relationship modeling, performing refined processing of local anatomical structures to meet the differentiated needs of different tasks.

[0054] Existing methods typically employ a uniform attention mechanism to handle multi-task scenarios, failing to effectively meet the differentiated feature representation requirements of various tasks. In fundus image analysis, classification tasks require global semantic features, segmentation tasks require fine boundary features, and visual question answering (VQA) tasks require multimodal fusion features. This difference in requirements leads to the poor performance of traditional attention mechanisms in handling complex multi-task scenarios. This method designs a progressive attention mechanism that can simultaneously satisfy global pathological feature consistency modeling and local anatomical structural detail enhancement. HPHA introduces a hierarchical progressive feature enhancement strategy into the attention framework, achieving an organic combination of global consistency and task specificity through two-level attention computation, thereby significantly improving the quality and adaptability of multi-task feature representation.

[0055] The specific structure of the layered progressive head attention module is as follows: Figure 4 As shown. The working process of HPHA is as follows: For the input image embedding vector , , For height, For width, To determine the number of feature channels, a multi-task shared query, key, and value are first generated through a linear transformation. Spatial features are then reshaped into a sequence form and decomposed into multi-head query types. , ,in, For the number of attention heads, For the feature dimensions of a single attention head, , The sequence length is the total number of spatial locations. .

[0056] Calculate the initial normalized attention weight matrix The formula is: in, Let T be the normalization function, and T be the matrix transpose. For multi-headed key, .

[0057] right The attention weight matrix is ​​reshaped to obtain a head-like form. , This serves as the input for the first layer of attention mechanism.

[0058] The first-level attention mechanism aims to establish a consistent global representation of pathological features. Considering that all three tasks require a unified understanding of the global feature distribution, this method performs information fusion at the head dimension. Adaptive pooling is applied along the head dimension. Max pooling captures the feature responses most sensitive to fundus pathological regions in multi-head attention, while average pooling preserves the statistical distribution information of global pathological features. The two pooling features are concatenated along the head dimension and then used to generate global pathological attention weights through a convolutional network. in, For global pathological attention weights, It is the sigmoid activation function. For convolution operations, The attention weight tensor is the result of max pooling in the head dimension. This is the attention weight tensor after average pooling of the head dimension.

[0059] Level 1 Enhanced Attention Weights Obtained through element-wise multiplication: in, This is element-wise multiplication. Through broadcasting, it affects all Individual attention points are used to enhance the consistency of global pathological features.

[0060] right Dimensional reshaping is performed to obtain a spatial attention weight matrix. This serves as the input for the second-layer attention mechanism.

[0061] The second-level attention mechanism is specifically designed to handle the diverse needs of various tasks in fundus image processing. To maintain head specificity to accommodate these diverse needs, statistical pooling is performed along the spatial correlation dimension. Spatial dimension pooling operations capture statistical information about the correlation strength between each spatial location and all other locations, providing a foundation for modeling the differentiated spatial relationships of different task heads.

[0062] Pooling is performed along the spatial dimension, and the result is obtained through feature concatenation: in, For spatial dimension-related attention weight tensors, The attention weight tensor after max pooling in spatial dimension. This is the attention weight tensor after spatial dimension average pooling.

[0063] Then, Remodeling , For the attention weight tensor related to the reshaped spatial dimension, .

[0064] Refined anatomical structure attention weights are generated through convolutional networks. : in, This is the bias vector.

[0065] Will Reconstructing the tensor back to a form compatible with the original tensor yields the reconstructed anatomical attention weights. , .

[0066] The second-level enhanced layered and refined attention weights for: .

[0067] Will Remodeling into a standard multi-head attention form yields remodeled, hierarchical, and refined attention weights. , Then, perform weighted aggregation with the value matrix: in, As a weighted feature, It is a multi-head form value.

[0068] After dimensionality restructuring and multi-head output linear projection, an enhanced image embedding vector is obtained based on weighted features. , .

[0069] The technical advantages of the HPHA mechanism are as follows: First, it satisfies the needs of global consistency and task specificity through a hierarchical and progressive design; second, the first-level global consistency modeling establishes a unified foundation for understanding pathological features for the three tasks; and finally, the second-level head-specific refinement allows different attention heads to focus on different anatomical structures, achieving refined feature enhancement for different task requirements.

[0070] S3: Input the enhanced image embedding vector into the multi-task decoding head to obtain the analysis results.

[0071] The multi-task decoder architecture comprises three specially designed task-specific decoders: a classification task head, a segmentation task head, and a visual question-answering head. Each decoder operates independently, leveraging shared features enhanced by HPHA to generate task-specific output, ensuring efficient and coordinated multi-task performance.

[0072] like Figure 5As shown, the classification task head uses the CABNet classifier architecture for four-class classification prediction of the severity of pathological myopia. It employs a classifier for feature mapping and classification prediction, outputting a four-class probability distribution to achieve classification results for the severity of pathological myopia: normal, mild, moderate, and severe. Enhanced image embedding vectors are input to the classification task head, which outputs the classification results for the severity of pathological myopia. Figure 6 As shown, the segmentation task head is based on the UNet decoder architecture. Through upsampling operations and skip connections, it recovers the spatial resolution layer by layer and fuses multi-scale features to finally generate a pixel-level view disk segmentation mask, achieving pixel-level accurate segmentation of the view disk region. The enhanced image embedding vector is input to the segmentation task head, and the output is the view disk segmentation mask. Figure 7 As shown, the visual question-answering head employs a Surgical-VQA decoder, combining image features and text questions for multimodal reasoning. An attention mechanism aligns image features and text semantics, outputting a natural language answer. Enhanced image embedding vectors are input into the visual question-answering head, outputting medical question-answering results. Analysis results include pathological myopia severity classification results, optic disc segmentation masks, and medical question-answering results.

[0073] The effectiveness of the present invention is verified through the following experiments in this embodiment.

[0074] This method freezes the pre-trained parameters of the shared feature encoder during multi-task learning model training and dynamically adjusts the LoRA parameters using the ELT strategy for efficient fine-tuning; it focuses on training the parameters of the HPHA module and the multi-task decoder. This allows the use of rich visual features learned on large-scale datasets, while efficiently fine-tuning parameters to adapt to the pathological myopia analysis task. All input image resolutions are uniformly adjusted to 224×224. The Swin Transformer is chosen as the basic architecture of the shared feature encoder. The learning rate is set to 5×10⁻⁶. -5 The maximum training epochs are set to 100. Cosine annealing is used for learning rate decay, with a warm-up period of 5 epochs. The batch size is set to 60. The AdamW optimizer is used for optimization, with weight decay set to 1×10⁻⁶. -8 The entropy-driven low-rank adaptation tuning strategy has an initial rank of 64, a minimum rank of 4, and an information retention ratio of 0.99. Rank adjustment is triggered after 10 epochs of performance stagnation. To enhance training stability and improve the overall performance of the model, cross-entropy loss is used for classification and visual question answering tasks, while a combination of cross-entropy loss and Dice loss is used for segmentation tasks. All experiments were conducted on an NVIDIA H100 GPU equipped with 80GB of VRAM.

[0075] This embodiment uses the constructed Pathological Myopia Multi-task Analysis (PMMA) dataset to validate the effectiveness of our method and several existing multi-task learning methods. Within the multi-task learning framework, this embodiment compares different parameter-efficient fine-tuning strategies and attention mechanisms, evaluating the performance of different strategies in the pathological myopia analysis task. This embodiment performs full multi-task training on the baseline model, and the results are used as a comparative reference.

[0076] The experimental results are shown in Tables 1, 2, and 3. Performance metrics for the classification task include accuracy, precision, recall, and F1 score. Accuracy is the ratio of correctly classified samples to the total number of samples; precision is the ratio of samples predicted as positive and actually classified as positive to the number of samples predicted as positive; recall is the ratio of samples predicted as positive and actually classified as positive to the number of samples actually classified as positive; and the F1 score is the harmonic mean of precision and recall. Performance metrics for the segmentation task include the Dice coefficient and Hausdorff distance (HD). The Dice coefficient is the ratio of the overlap between the predicted foreground region and the ground truth foreground region to their average value; and the Hausdorff distance measures the accuracy of the segmentation boundary. Performance metrics for the VQA task include accuracy, precision, recall, and F1 score, measuring the model's understanding and reasoning ability in medical question answering.

[0077] Table 1 Performance comparison results of classification tasks

[0078] Table 1 shows the comparison results between our method and other state-of-the-art methods. Among the other state-of-the-art methods, the single-task models selected were ConvNeXt, ViT, FlexiViT, CaiT, SwinT, SwinT V2, CABNet, FedIIC, and HiFuse, while the multi-task models selected were DeepUNet and ACSNet. In the classification task, our method achieved a highest accuracy of 83.95%, surpassing all baseline methods. Most notably, our method demonstrated excellent recall, reaching 76.05%, significantly better than the best baseline method SwinT's 70.05%, an improvement of 6.00 percentage points. This is crucial for reducing the risk of missed diagnoses in clinical applications. The F1 score further validated the effectiveness of our method, reaching 77.41%, an improvement of 7.05 percentage points compared to ConvNeXt's 70.36%. Compared with multi-task learning methods, our method significantly outperforms DeepUNet and ACSNet, improving accuracy by 5.42% and 4.77%, respectively, and F1 score by 9.04% and 8.18%, respectively.

[0079] Table 2 Performance comparison results on the segmentation task

[0080] Table 2 shows the comparison results for the visual disc segmentation task.

[0081] For the Dice metric, this method demonstrates superior performance, significantly outperforming traditional CNN architectures such as UNet, Att-UNet, CENet, and UNeXt, with improvements ranging from 2.52% to 8.48%. Compared to Transformer-based methods, this method maintains its advantages, particularly surpassing Med-SA and SAM-Med2D; specifically, it achieves a 0.78% improvement over the best-performing SAM-Med2D. Compared to KAN-based methods, this method outperforms U-KAN, MedVKAN, AttUKAN, and TransUKAN, with improvements ranging from 1.08% to 2.20%.

[0082] For the HD metric, our method achieves 3.29, which, while slightly higher than Med-SA's best of 3.09, remains competitive and demonstrates excellent boundary accuracy.

[0083] Table 3 Performance Comparison Results of Visual Question Answering Tasks

[0084] Table 3 shows the performance advantages of our proposed method in the classification-based medical VQA task. Our method achieves an accuracy of 95.37%, a 1.08% improvement over the second-best method, RAMM (94.29%). In terms of precision, our method achieves 89.84%, surpassing RAMM's 89.53%. More importantly, our method achieves a recall of 88.20%, a significant 10.59% improvement over the best-performing baseline, RAMM's 77.61%, which is particularly important for reducing the risk of missed diagnoses. Regarding the F1 score, our method achieves 88.75%, a 5.72% improvement over RAMM's 83.03%, demonstrating a good balance between precision and recall.

[0085] Figure 8 The top six network segmentation results, ranked by average accuracy, are visualized. Figure 8 (a) is the original image. Figure 8 (b) is the label. Figure 8 (c) represents Att-Unet. Figure 8 In the middle (d), ACSNet is represented. Figure 8 (e) represents UCTransNet. Figure 8 In the middle (f), SAM-Med2D is used. Figure 8 In the middle (g), MedVKAN is used. Figure 8 (h) represents the proposed method. Experimental results show that the proposed method achieves excellent segmentation performance.

[0086] This embodiment also provides a pathological myopia analysis system based on multi-task learning, such as... Figure 2 As shown, the technical solution adopted is as follows: it includes a shared feature encoder, a hierarchical progressive head attention module (HPHA), and a multi-task decoding head. The shared feature encoder is used to extract features from pathological myopia fundus images to obtain image embedding vectors; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy. The hierarchical progressive head attention module is used to perform feature enhancement on the image embedding vector to obtain an enhanced image embedding vector. The multi-task decoding head is used to obtain analysis results based on the enhanced image embedding vector.

[0087] The entropy-driven low-rank adaptive tuning (ELT) strategy designed in this invention dynamically optimizes the rank values ​​of LoRA parameters based on information entropy. Compared with traditional fixed-rank configuration methods, it can adaptively adjust parameter allocation according to data characteristics and task complexity. This design enables the model to intelligently balance parameter efficiency and expressive power when dealing with complex medical images such as pathological myopia, avoiding parameter redundancy or insufficient expression. Therefore, this method is more suitable for handling high-precision medical image analysis tasks in resource-constrained environments. Through this dynamic parameter optimization method, the model not only retains 99% of the information content but also achieves a 30-50% reduction in parameters, significantly reducing computational overhead and deployment costs.

[0088] The hierarchical progressive head-specific attention (HPHA) mechanism designed in this invention handles multi-task feature requirements through two-level progressive attention computation. Compared with the traditional unified attention mechanism, it can more effectively meet the differentiated feature representation needs of different tasks. This hierarchical progressive processing method enables the model to simultaneously establish global pathological feature consistency and task-specific enhancement, thereby improving the model's recognition accuracy and task coordination ability for multi-scale lesion features in complex fundus images. Furthermore, the combination of self-attention and spatial attention mechanisms further enhances the discriminative ability of features, allowing the model to focus on key areas of retinal lesions. Therefore, our method has higher multi-task analysis accuracy.

[0089] The multi-task collaborative learning framework designed in this invention simultaneously handles classification, segmentation, and visual question answering tasks through a unified architecture. Compared to traditional independent task learning methods, it can more fully utilize the inherent correlation and synergistic effects between tasks. This collaborative learning method enables classification results to guide segmentation region localization, segmentation masks to enhance spatial understanding of question answering, and visual question answering to provide semantic verification, thus forming a virtuous cycle of mutual promotion among tasks. Therefore, this method significantly outperforms single-task methods in overall performance. Through this cross-task collaborative optimization, the model not only improves the accuracy of individual tasks but also enhances the reliability and robustness of the overall analysis.

[0090] This invention integrates an entropy-driven low-rank adaptive optimization strategy, a hierarchical progressive head-specific attention mechanism, and a multi-task collaborative learning framework. It effectively utilizes advanced parameter optimization techniques and attention mechanisms, significantly improving computational efficiency, feature representation, and analytical accuracy compared to existing medical image analysis methods. Through these technological innovations, this invention not only achieves high precision and efficiency in multi-task analysis of pathological myopia but also possesses better parameter economy and clinical deployment adaptability, giving it significant technical advantages and practical value in intelligent ophthalmological diagnostic applications.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for analyzing pathological myopia based on multi-task learning, characterized in that, Includes the following steps: S1: Input the pathological myopia fundus image into the shared feature encoder for feature extraction to obtain the image embedding vector; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy; S2: The image embedding vector is enhanced by passing it through a hierarchical progressive head attention module to obtain an enhanced image embedding vector; S3: Input the enhanced image embedding vector into the multi-task decoding head to obtain the analysis results.

2. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, The shared feature encoder includes a block embedding layer and multiple Transformer modules connected in sequence.

3. A method for analyzing pathological myopia based on multi-task learning as described in claim 1 or 2, characterized in that, The process of dynamically adjusting the rank of the LoRA parameter using information entropy is as follows: Calculate the entropy of matrix A and matrix B respectively, and then calculate the total entropy, where matrix A and matrix B are trainable low-rank matrices; Starting from the current rank, iteratively decrease the rank, calculate the entropy of the corresponding A-truncation matrix and B-truncation matrix respectively, compare the sum of the two with the total entropy, find the smallest rank that can retain a specified proportion of the total entropy, and obtain the optimal rank; Adjust matrices A and B using the optimal rank, and calculate the low-rank increment based on the adjusted matrices A and B.

4. The method for analyzing pathological myopia based on multi-task learning as described in claim 3, characterized in that, Calculate the optimal rank of all LoRA layers, take the median as the target rank, and adjust the A and B matrices of all LoRA layers using the target rank.

5. The method for analyzing pathological myopia based on multi-task learning as described in claim 3, characterized in that, By normalizing the absolute values ​​of the matrix elements to a probability distribution, the entropy of matrix A and matrix B are calculated separately; the total entropy is the sum of the entropy of matrix A and the entropy of matrix B.

6. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, A low-rank increment is introduced into the pre-trained weight matrix of the shared feature encoder, and forward propagation is performed for training. After each training iteration, the performance of the shared feature encoder is verified. If the performance does not improve over multiple consecutive training cycles, the rank value of the LoRA parameter is dynamically adjusted using information entropy.

7. The method for analyzing pathological myopia based on multi-task learning as described in claim 1, characterized in that, The hierarchical progressive head attention module includes a two-level progressive attention mechanism; the first-level attention mechanism adopts adaptive pooling operation to establish global pathological feature consistency through cross-head information fusion. The second-level attention mechanism performs statistical pooling along the spatial correlation dimension, capturing statistical information on the correlation strength between each spatial location and other locations.

8. A method for analyzing pathological myopia based on multi-task learning as described in claim 1 or 7, characterized in that, The processing steps of the layered progressive head attention module include: S2.1: Obtain the attention weight matrix in head form based on the image embedding vector; S2.2: The head-shaped attention weight matrix is ​​used to obtain the spatial attention weight matrix through the first-level attention mechanism; The processing steps include: applying max pooling and average pooling along the head dimension to the attention weight matrix in head form, then concatenating along the head dimension, and then generating global pathological attention weights through a convolutional network; the global pathological attention weights are applied to all attention heads to obtain a spatial attention weight matrix; S2.3: The spatial form of the attention weight matrix is ​​used to obtain an enhanced image embedding vector through a second-level attention mechanism; The processing steps include: performing max pooling and average pooling operations along the spatially relevant dimensions on the attention weight matrix in spatial form, and then generating anatomical structure attention weights through connection and convolutional networks; applying the anatomical structure attention weights to the spatial form attention weight matrix to obtain hierarchical refinement attention weights; and obtaining enhanced image embedding vectors based on the hierarchical refinement attention weights and the value matrix.

9. A method for analyzing pathological myopia based on multi-task learning as described in claim 1 or 7, characterized in that, Multi-task decoding heads include classification heads, segmentation heads, and visual question-answering heads; The enhanced image embedding vectors are input into the classification head, segmentation head, and visual question answering head, respectively. The classification head outputs the classification results of the severity of pathological myopia, the segmentation head outputs the optic disc segmentation mask, and the visual question-and-answer head outputs the medical question-and-answer results.

10. A pathological myopia analysis system based on multi-task learning, characterized in that, A method for performing pathological myopia analysis based on multi-task learning as described in any one of claims 1 to 9, comprising: a shared feature encoder, a hierarchical progressive head attention module, and a multi-task decoding head. The shared feature encoder is used to extract features from pathological myopia fundus images to obtain image embedding vectors; the shared feature encoder is based on the Swing Transformer architecture and dynamically adjusts the rank value of the LoRA parameter through information entropy. The hierarchical progressive head attention module is used to perform feature enhancement on the image embedding vector to obtain an enhanced image embedding vector. The multi-task decoding head is used to obtain analysis results based on the enhanced image embedding vector.

Images