A polycystic ovary syndrome early screening and subtype identification system and method based on multi-modal modeling and explainability mechanism

Abstract

The application discloses a PCOS early screening and subtype identification system and method based on a multi-modal modeling and explainability mechanism, and belongs to the technical field of medical artificial intelligence. The system comprises a multi-modal data fusion module, a shared-specific multi-task learning module and a double-channel explainability enhancement module connected in sequence. The multi-modal data fusion module generates unified feature representation by using differentiated strategies covering gated weighted early fusion, graph neural network fusion and collaborative attention fusion. The shared-specific multi-task learning module realizes joint prediction through a shared encoder and a double-task branch. The double-channel explainability module synchronously outputs visualized and quantified explanation results. The application can improve multi-modal data fusion effect and model clinical credibility, and provides artificial intelligence auxiliary support for polycystic ovary syndrome early screening.

Description

Technical Field

[0001] This invention belongs to the field of medical artificial intelligence and medical data processing technology, specifically relating to a PCOS early screening and subtype identification system and method based on multimodal modeling and interpretability mechanisms. Background Technology

[0002] Polycystic ovary syndrome (PCOS) is one of the most common endocrine and metabolic disorders in women of reproductive age, with a global incidence of approximately 6% to 20%. The clinical manifestations of this disease are highly heterogeneous, with common symptoms including oligomenorrhea or amenorrhea, hyperandrogenemia, and polycystic ovarian morphology. It is often accompanied by metabolic syndrome manifestations such as insulin resistance, obesity, and lipid metabolism abnormalities. Current clinical diagnosis primarily uses the Rotterdam criteria, but these criteria have several limitations in practical application: the diagnostic criteria rely heavily on subjective or semi-quantitative indicators such as hirsutism scores and ultrasound follicle counts, resulting in limited consistency among physicians; existing subtype classification systems are relatively crude and fail to fully reflect individual differences in metabolic, reproductive, and endocrine dimensions; and individualized predictive tools for subsequent interventions are relatively lacking.

[0003] In recent years, artificial intelligence technology has been gradually applied to the risk prediction and assisted screening of PCOS. Existing related research is mainly divided into two categories: one is based on structured clinical data (including hormone levels, BMI, menstrual cycle parameters, etc.) to build machine learning classification models; the other is based on ovarian ultrasound images to detect, count and classify follicles morphologically. However, the above-mentioned solutions generally suffer from the following technical limitations: First, the information sources are mostly limited to a single modality, and the multimodal fusion methods are simplistic, failing to adapt to the differences in the strength of association between different modalities. This makes it difficult to comprehensively characterize the pathological features of PCOS across multiple dimensions, including imaging, hormones, metabolism, and clinical phenotypes. Even when a few studies have attempted multimodal fusion, they often employ simple feature splicing strategies, failing to differentiate modeling based on the differences in the strength of pathological associations between different modalities, resulting in insufficient adaptability to highly heterogeneous patient populations. Second, there are few studies on refined modeling at the subtype level, limiting the technical support for clinical stratification management. Third, most models output results in an end-to-end black-box manner, failing to provide verifiable decision-making basis and limiting the credibility and acceptability of the models in real clinical scenarios. Fourth, in the face of the common feature loss problem in real-world data collection, simple processing methods such as mean imputation or sample removal are often used, further reducing the generalization performance of the models. Fifth, medical data involves patient privacy, and existing solutions are relatively weak in privacy protection in single-machine training and multi-center collaborative deployment scenarios.

[0004] Therefore, how to use computer technology to simultaneously achieve the effective fusion of multimodal medical data, the collaborative prediction of PCOS disease risk and subtypes, the transparent presentation of decision-making criteria, and the full-process privacy protection within a single system, so as to provide reliable auxiliary reference for clinical screening, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the shortcomings and deficiencies of existing technologies, this invention provides a PCOS early screening and subtype identification system and method based on multimodal modeling and interpretability mechanisms. The system achieves deep integration and intelligent analysis of multi-source heterogeneous medical data through a multimodal data fusion module, a shared-specific multi-task learning module, and a dual-channel interpretability enhancement module connected in sequence. At the data fusion level, this invention employs a differentiated fusion strategy based on the differences in the strength of association between modalities. For strongly associated modalities, gated weighted early fusion is used to retain fine-grained interaction information. For weakly associated modalities, a modal relationship graph is constructed, and the association strength is adaptively adjusted through a learnable edge attention mechanism. Then, global fusion using collaborative attention is used to mine higher-order semantic associations, effectively overcoming the problem that a single fusion strategy cannot adequately consider the association characteristics of different modalities. At the multi-task learning level, common semantic representations are extracted through a shared feature encoder, and end-to-end joint training is performed via a risk prediction branch and a subtype classification branch. An adaptive weighting strategy dynamically balances task losses, solving the problems of disconnected risk prediction and subtype classification and insufficient generalization ability in small sample scenarios in traditional methods. At the interpretability level, the system generates visualization results of the model's focus area through the image interpretation channel, quantifies the contribution of structured features through the phenotypic interpretation channel, and combines a clinical consistency quantitative assessment mechanism to provide verifiable objective evidence for model decision-making, significantly improving the system's credibility and practicality in clinical auxiliary screening.

[0006] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0007] This invention addresses four core technical deficiencies in existing intelligent screening solutions for polycystic ovary syndrome (PCOS): First, the multimodal data fusion method is singular, failing to consider the significant heterogeneity of intrinsic correlations between different medical modalities. Using a unified fusion strategy easily leads to the dilution of strongly correlated information or the amplification of weakly correlated noise, resulting in insufficient multimodal feature representation capabilities and difficulty in comprehensively depicting the multidimensional pathological characteristics of the disease. Second, disease risk identification and subtype classification tasks are disconnected. In clinically common scenarios with small samples and class imbalance, the model's generalization ability is weak. Furthermore, there is a lack of refined subtype label construction schemes that combine data-driven approaches with medical semantics, resulting in insufficient clinical interpretability of the obtained subtypes. Third, the model's decision-making process has low transparency and lacks a quantifiable mechanism for verifying clinical rationality. Interpretation results largely rely on subjective human judgment, leading to poor repeatability and insufficient clinical acceptance. Fourth, real-world medical data has a high missing rate, and privacy compliance requirements are strict. Existing solutions often use mean imputation or simple sample removal to handle missing data, and the privacy protection mechanisms are poorly adaptable, making it difficult to simultaneously meet robustness and compliance requirements. This invention aims to achieve, through computer technology, the effective integration of multimodal medical data, the collaborative prediction of disease risk and subtypes, the transparent presentation of decision-making criteria, and the full-process privacy protection within a single system, providing a reliable auxiliary reference for clinical screening.

[0008] To address the aforementioned technical problems, this invention provides a system for assisted screening and subtype identification of polycystic ovary syndrome based on multimodal modeling and interpretability mechanisms. The system includes a multimodal data fusion module, a shared-specific multi-task learning module, and a dual-channel interpretability enhancement module connected in sequence.

[0009] 1. Multimodal data fusion module

[0010] The multimodal data fusion module receives and preprocesses multi-source heterogeneous medical data for auxiliary screening of polycystic ovary syndrome (PCOS). This multi-source heterogeneous medical data includes at least four categories: ultrasound images, biochemical hormone indicators, metabolic parameters, and clinical information. The module employs a differentiated fusion strategy based on the differences in the correlation strength between modalities to generate a unified multimodal feature representation. This differentiated fusion strategy uses three parallel channels, employing an adapted fusion method for modality pairs with different correlation strengths to avoid the limitations of a single fusion strategy.

[0011] Early Fusion with Gated Weights for Strongly Associated Modalities: For modal pairs with strong semantic associations, such as ultrasound images and structured indicators, the image feature map is first compressed into a one-dimensional vector using global average pooling, and then linearly projected onto a unified dimensional space aligned with the structured features. Simultaneously, the structured features are mapped to the same dimensional space using a multilayer perceptron. Then, learnable gated weights are used to perform element-wise weighted fusion of the two feature paths. This design can dynamically adjust the contribution ratio of different modal features, achieving tightly coupled representation while preserving the original discriminative power of each modality. The network can adaptively adjust the fusion weights according to the characteristics of the input samples, avoiding the rigidity of expression caused by fixed weights. It also achieves early complementarity of associated information, reducing information loss in subsequent feature processing. The dimensionality of the unified dimensional space is a pre-defined optimal parameter based on the modal feature dimensions and task requirements.

[0012] Graph Neural Network Fusion for Weakly Associated Modalities: For modal pairs with relatively weak semantic associations, such as ultrasound morphology and metabolic parameters, or ultrasound morphology and clinical information, a modal relationship graph is constructed using the feature vectors of each modality as nodes. An initial adjacency matrix is ​​set based on prior medical knowledge (higher initial weights are assigned to modal pairs with known strong pathological associations, and lower initial weights are assigned to weakly associated modal pairs; self-loop weights are set to fixed values). During training, a learnable edge attention mechanism adaptively adjusts the association strength of each edge in the graph. After training convergence, the association strength of the corresponding modal pair is automatically determined by comparing the mean of the attention weights on each edge with a preset threshold (above the threshold is considered a strong association, otherwise a weak association). The preset threshold is a modal association judgment standard pre-determined based on large-sample clinical data statistics. This graph neural network preferably uses a two-layer graph convolutional network for message passing between nodes, thereby obtaining a node-level representation containing inter-modal relationship information. Then, mean pooling is performed on all node features to obtain a graph-level fused representation. This design injects medical knowledge into the prior adjacency matrix as a structured inductive bias, guiding the model to focus on modal interaction paths with physiological basis in the early stage of training. Then, it automatically learns the real correlation strength between modalities through the side attention mechanism, avoiding the subjectivity of manually setting adjacency relationships.

[0013] Collaborative attention global fusion: The output of the early gated weighted fusion and the output of the graph neural network fusion are concatenated and fed into a multi-head cross-attention module to mine high-order semantic relationships between modalities, generating a final unified multimodal feature representation. This multi-head cross-attention module, through parallel computation of multiple attention heads, can capture interaction patterns between modalities from different subspaces, enabling the final fused features to possess both fine-grained local interactions and global contextual information.

[0014] As a preferred implementation, the multimodal data fusion module also incorporates a generative imputation unit. This unit employs a generative adversarial imputation network to imput features with a missing rate below a preset threshold, and triggers a dynamic compensation acquisition process for features with a missing rate above the preset threshold. The preset missing rate threshold is an optimal value set for the missing characteristics of medical data. This design distinguishes between repairable and unreliable missing data, avoiding the introduction of systematic bias by forcibly imputing severely missing features and ensuring the baseline quality of the input data. The generative adversarial imputation network consists of a generator and a discriminator. The generator receives an incomplete data matrix, a corresponding binary mask matrix, and random noise as input, and outputs the imputed complete data. The discriminator receives the imputed data and a cue matrix as input, and determines whether each value is a true observation or a generated imputation value for each feature. After training, the original values ​​are retained for observed positions, and missing positions are filled with the generated results, thus ensuring imputation quality while avoiding information loss caused by simple mean imputation or sample removal.

[0015] 2. Shared-Specialized Multitasking Learning Module

[0016] The shared-specific multi-task learning module includes a shared feature encoder, a risk prediction branch network, and a subtype classification branch network. The shared feature encoder receives a unified multimodal feature representation and outputs a shared latent semantic representation. The shared feature encoder is preferably composed of residual blocks. This design aims to force the model to learn common representations that are discriminative for both downstream tasks, avoiding overfitting of single-task models to noise specific to a given task with limited samples, and effectively improving generalization ability in small-sample scenarios.

[0017] The risk prediction branch network output is activated by Sigmoid to obtain binary classification probabilities, indicating the level of disease risk. The subtype classification branch network output is activated by Softmax to obtain probability vectors for four subtypes (metabolic, reproductive, mixed, and background). The two branches are jointly trained end-to-end through a joint loss function, which is a weighted sum of the risk prediction task loss and the subtype classification task loss. The subtype classification task uses a class weighting mechanism to address the class imbalance problem of clinical samples.

[0018] To avoid suboptimal results from manually fixing weighting coefficients, an adaptive adjustment strategy is adopted, including two optional implementation methods: The first is a dynamic weighting method based on uncertainty modeling, which introduces learnable uncertainty parameters to allow the model to automatically perceive the difficulty of each task and allocate loss weights accordingly, reducing the dominant role of difficult tasks in overall optimization. The second is a dynamic balancing method based on gradient normalization, which monitors the gradient magnitude of each task on the shared layer and dynamically normalizes their relative contributions, mitigating optimization direction shifts caused by differences in gradient magnitudes and ensuring that both tasks obtain balanced learning signals in the shared parameter space. This adaptive adjustment strategy eliminates the need for manually setting fixed weights and automatically balances task contributions during training, effectively avoiding gradient conflicts in multi-task training. As the preferred implementation, the model training process incorporates cosine annealing learning rate scheduling and an early stopping mechanism to further improve training stability and performance in small-sample scenarios.

[0019] As a preferred implementation, the system also includes a subtype label construction unit for generating high-quality subtype labels during the training phase. This unit extracts multiple subtype-related quantitative variables from the training samples (including at least BMI, LH, FSH, dehydroepiandrosterone sulfate, sex hormone-binding globulin, total testosterone, fasting insulin, and fasting blood glucose), performs unsupervised hierarchical clustering after standardization, and sets the number of clusters to 3. After clustering, based on preset subtype semantic mapping rules, the three clusters are matched as basic labels for metabolic, reproductive, and background subtypes, respectively. The preset semantic mapping rules are pre-set by clinicians according to the characteristic distribution patterns of the centers of each cluster. Subsequently, mixed-type samples are identified through rule post-processing: samples that simultaneously meet the condition that at least two indicators are within the preset standard deviation range of the centers of the metabolic cluster and at least two indicators are within the preset standard deviation range of the centers of the reproductive cluster are additionally labeled as mixed-type, ultimately resulting in four subtype labels. The preset standard deviation range is an optimal value validated by clinical datasets and can be adaptively adjusted according to the actual sample distribution. The label construction process also includes a consistency verification mechanism to ensure the quality of label annotation. This two-stage label generation mechanism combines data-driven clustering structures with medical prior semantic interpretation, avoiding the over-reliance of purely supervised methods on high-quality labeled data and overcoming the limitation of clusters obtained by purely unsupervised methods lacking clinical interpretability.

[0020] 3. Dual-channel interpretability enhancement module

[0021] The dual-channel interpretability enhancement module provides verifiable decision-making support for the model output. This module includes an image interpretation channel and a phenotypic interpretation channel: The image interpretation channel is based on a visual attribution algorithm (preferably Grad-CAM). It calculates the gradient of the feature map of the last convolutional layer and spatially averages it to obtain the channel weights. Then, it linearly weights all feature maps according to these weights and activates them using ReLU to generate a heatmap of the model's region of interest for the corresponding input ultrasound image. This visually presents the spatial location of the image that the model focuses on when making predictions, allowing clinicians to directly verify whether the model is focusing on pathologically significant anatomical structures. The phenotypic interpretation channel is based on a feature attribution algorithm (preferably the SHAP value framework). It calculates the Shapley value of each structured input feature across all feature subsets, quantifies the marginal contribution of each feature to the prediction result, and outputs the model in order of absolute contribution value. This allows physicians to review the model output from the perspective of mechanistic rationality.

[0022] As a preferred implementation, the dual-channel interpretability enhancement module is also equipped with a clinical consistency quantification unit for objectively quantifying the clinical consistency of the interpretation results. This unit calculates the region matching rate between the image's region of interest and a preset lesion region reference mask (a union of lesion regions pre-annotated by clinicians based on expert consensus), as well as the distribution rationality score between each feature contribution direction and a preset medical prior (feature contribution directions pre-set based on known PCOS pathological mechanisms), thus providing quantifiable verification indicators for the model's interpretability. The use of preset reference standards instead of real-time manual annotation ensures both the automation and repeatability of the evaluation process and pre-constructs human cognitive judgment into executable technical rules for the system, making interpretability verification an internal technical step rather than an external subjective evaluation.

[0023] As a preferred implementation, the system also integrates a differential privacy protection unit, supporting two deployment modes that can be flexibly selected based on the actual deployment environment: In a single-machine training scenario, Gaussian noise is directly added to the shared latent semantic representation output by the shared feature encoder to achieve patient-level differential privacy protection; in a multi-center deployment scenario, based on a federated learning framework, each participant independently calculates the model gradient using local data, prunes the gradient according to the L2 norm, then adds Gaussian noise to the pruned gradient, and the noisy gradient is weighted and averaged through a secure aggregation protocol, preventing the central server from obtaining the original gradient of any single participant. The aggregated global gradient is used to update the global model and distributed to each participant for the next round of local training. Privacy protection parameters for both modes (including pruning threshold, noise multiplier, privacy budget, etc.) are pre-set according to compliance requirements and model performance balance, maintaining model training efficiency while ensuring patient privacy and security.

[0024] Based on the above system, another aspect of the present invention provides a computer-implemented method for multimodal medical data processing and subtype classification of polycystic ovary syndrome, comprising the following steps:

[0025] The first step is to preprocess and extract modal features from the multi-source heterogeneous medical data input for auxiliary screening of polycystic ovary syndrome, and obtain the initial feature representation of each modality.

[0026] The second step involves using a differentiated fusion strategy based on the intermodal correlation strength to perform cross-modal feature fusion, generating a unified multimodal feature representation. The differentiated fusion strategy includes gated weighted early fusion for strongly correlated modalities, graph neural network fusion for weakly correlated modalities, and collaborative attention global fusion of the two results. Among them, the graph neural network fusion constructs a modal relationship graph with each modal feature vector as a node, sets an initial adjacency matrix based on medical priors, and adaptively adjusts the correlation strength of each edge through a learnable edge attention mechanism.

[0027] The third step involves extracting shared latent semantic representations using a shared feature encoder, and then calculating them through the risk prediction branch and the subtype classification branch to output the disease risk classification result and the subtype classification result; the two branches are jointly trained end-to-end through a joint loss function.

[0028] The fourth step involves outputting the visualization results of the image modality of interest region and the quantification results of the structured feature contribution through a dual-channel interpretability mechanism. The results are used to provide reference information for clinical auxiliary screening for non-diagnostic purposes and do not constitute an independent diagnostic conclusion. The final diagnosis and treatment judgment shall be made by qualified professionals in combination with the complete clinical context.

[0029] Compared to existing technologies, this invention and its preferred solution, through a three-channel differentiated fusion architecture, can adaptively select the optimal fusion path based on the actual correlation strength between modalities, avoiding information loss and noise introduction caused by forced alignment or simple splicing, and improving the integrity and discriminative power of multimodal feature representation. Through a shared-specific multi-task learning framework, risk prediction and subtype classification tasks mutually promote each other in a shared semantic space, enhancing the model's robustness and generalization performance under small sample conditions. At the same time, the joint optimization mechanism effectively alleviates gradient conflicts between multiple tasks. Through dual-channel interpretability enhancement and clinical consistency quantitative assessment, the model's internal decision-making logic is transformed into visual evidence that conforms to medical cognition, making the output results traceable and verifiable, and lowering the trust threshold for clinical deployment. Through differential privacy protection and generative missing data imputation and other supporting mechanisms, the model's adaptability to incomplete real-world data is improved while ensuring patient data security, enhancing the system's engineering feasibility and long-term deployment stability. Attached Figure Description

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0031] Figure 1 This is a schematic diagram of the overall steps of the method for auxiliary screening and subtype identification of polycystic ovary syndrome according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the overall architecture and data flow of the polycystic ovary syndrome auxiliary screening and subtype identification system according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the internal structure of the shared-specific multi-task learning module in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of the structure of the dual-channel interpretability enhancement module in an embodiment of the present invention. Detailed Implementation

[0035] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0036] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0037] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0038] In view of the defects and shortcomings of the existing technology, the technical problem to be solved by the present invention is as follows:

[0039] Single-modal information is insufficient to fully characterize the multidimensional pathological features of PCOS. The heterogeneous data fusion method is too simple and cannot adapt to the differences in the correlation strength between different modalities, resulting in insufficient fusion characterization ability.

[0040] Disease risk prediction and subtype classification tasks are disconnected from each other. The model has insufficient generalization ability in scenarios with small samples and class imbalance, and there is a lack of refined subtype label construction schemes that combine data-driven approaches with medical semantics.

[0041] The model's decision-making process lacks transparency and a quantifiable mechanism for verifying its clinical rationality, limiting its clinical acceptance and making it difficult to implement.

[0042] Real-world medical data is often incomplete, and privacy compliance requirements are stringent. Existing solutions have significant shortcomings in terms of data robustness and privacy protection.

[0043] To address the aforementioned technical problems, this invention provides a PCOS intelligent assisted screening and subtype identification system based on multimodal modeling and interpretability mechanisms. The system mainly consists of three parts: a multimodal data fusion module, a shared-specific multi-task learning module, and a dual-channel interpretability enhancement module.

[0044] The multimodal data fusion module receives and preprocesses heterogeneous data from multiple sources, including ultrasound images, biochemical hormone indicators, metabolic parameters, and clinical information. It performs cross-modal semantic alignment and employs corresponding fusion strategies based on the differences in the correlation strength between modalities to generate a unified multimodal feature representation. The shared-specific multi-task learning module, building upon the fused features, extracts latent common semantic representations through a shared encoder. These representations are then processed by two task-specific networks—a risk prediction branch and a subtyping branch—to complete PCOS risk classification prediction and four subtype classifications: metabolic, reproductive, mixed, and background subtypes. The dual-channel interpretability enhancement module consists of an image interpretation channel and a phenotypic interpretation channel. It generates a heatmap of the region of interest in the ultrasound image based on the Grad-CAM algorithm and quantifies the marginal contribution of each structured input feature to the model's decision based on the SHAP value framework, thus providing verifiable decision-making basis for the model output.

[0045] Another aspect of this invention provides a computer-implemented method for PCOS multimodal medical data processing and subtype classification model construction. The method comprises three stages: model construction and training, data inference processing, and model evaluation. In the model construction and training stage, multimodal medical samples with risk and subtype labels are used to train the system end-to-end, resulting in a trained PCOS multimodal classification model. In the data inference processing stage, the patient's multimodal medical data to be processed is input into the trained model, sequentially completing data integration, disease risk classification prediction, and subtype classification. A dual-channel interpretability module outputs decision visualization reference information; the prediction results are only for auxiliary reference by clinicians and do not directly constitute the final clinical diagnostic conclusion. In the model evaluation stage, a multi-dimensional evaluation system including objective and quantifiable subjective indicators is used to comprehensively evaluate the model performance and compare it with existing methods for verification.

[0046] As a further preferred implementation, the present invention also includes a differential privacy protection mechanism for single-machine scenarios, a federated learning extension mechanism for multi-center deployments, a GAIN generative missing data filling mechanism, and a human-machine collaborative closed-loop optimization mechanism, comprehensively improving the engineering feasibility and clinical adaptability of the solution.

[0047] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0048] 1. System Overall Architecture

[0049] like Figure 2 As shown, the PCOS intelligent assisted screening and subtype identification system provided by the present invention is mainly composed of three parts connected in sequence: a multimodal data fusion module, a shared-specific multi-task learning module, and a dual-channel interpretability enhancement module.

[0050] The multimodal data fusion module receives and preprocesses heterogeneous data from multiple sources, including ultrasound images, biochemical hormone indicators, metabolic parameters, and clinical information. It performs cross-modal semantic alignment and employs appropriate fusion strategies based on the differences in the correlation strength between modalities to generate a unified multimodal feature representation. The shared-specific multi-task learning module, building upon the fused features, extracts latent common semantic representations through a shared encoder. These representations are then processed by two task-specific networks—a risk prediction branch and a subtyping branch—to perform PCOS risk classification prediction and four subtype classifications, respectively. The dual-channel interpretability enhancement module consists of an image interpretation channel and a phenotypic interpretation channel, outputting heatmap visualization results and feature contribution quantification results, respectively, providing verifiable evidence for model decision-making.

[0051] Based on the above system, the overall process of the computer-implemented PCOS multimodal medical data processing and subtype classification model construction method provided by this invention is as follows: Figure 1 As shown, this method comprises three core stages: model building and training, data inference processing, and model evaluation. The execution entity is a computer system, the processing object is pre-collected medical data, and the output results serve only as an auxiliary reference for clinical screening and do not directly constitute a disease diagnosis conclusion.

[0052] 2. Preprocessing and missing data imputation for multi-source heterogeneous data

[0053] In the model training and inference application phases, the medical data of PCOS patients to be processed were first separated into four categories according to modality: ultrasound images, biochemical hormone indicators, metabolic parameters, and clinical information. Ultrasound images were cropped to the region of interest and scaled to 224×224 pixels before being fed into the network. During the training phase, additional data augmentation strategies such as random rotation and flipping were employed to improve model robustness. Hormone and metabolic indicators were standardized using Z-score after outlier removal, and categorical variables were encoded using one-hot encoding.

[0054] Considering that the missing rate of some features in the multimodal medical sample collection process can reach over 20%, this invention uses GAIN (Generative Adversarial Imputation Network) to repair features with a missing rate below 20%, and triggers a dynamic compensation collection process for features with a missing rate above 20%. GAIN consists of a generator G and a discriminator D: the generator G receives an incomplete data matrix, the corresponding binary mask matrix, and random noise as input, and adopts a three-layer fully connected structure. Both the input and output dimensions are the feature dimension d, with hidden layer dimensions of 256 and 128 respectively. The hidden layers use BatchNorm and ReLU activation, and the output layer uses Sigmoid activation, outputting the completed data. The discriminator D receives the completed data and a cue matrix as input, also adopting a three-layer fully connected structure. The hidden layer dimensions are 128 and 64 respectively. The hidden layers use LeakyReLU activation with a negative slope of 0.2, and the output layer uses Sigmoid activation. It distinguishes between true and false values ​​for each feature, rather than judging the overall data as true or false.

[0055] To guide the generator in learning a reasonable conditional distribution, the hint matrix is ​​constructed as follows:

[0056]

[0057] Where M is a binary mask matrix, with elements of 1 representing observed data and 0 representing missing data; B is a binary matrix obtained from Bernoulli sampling, with its elements set to 1 with the cue probability and 0 with 1 minus that probability; in this implementation, the cue probability is 0.9; (symbol) This indicates element-wise multiplication.

[0058] The discriminator loss and generator loss are calculated using the following formulas:

[0059]

[0060]

[0061] Where X represents the true value corresponding to the observed portion of the incomplete data. The completed data output by the generator. To reconstruct the error weights, we set the value to 10 in this implementation.

[0062] GAIN training uses the Adam optimizer, and hyperparameter selection... , The learning rates for both the generator and discriminator are set to the same value. The batch size was 64, and the training ran for 200 epochs with an early stopping strategy of patience 20. The final filling result was obtained after training using the following formula:

[0063]

[0064] That is, the original values ​​are retained for observed positions, and missing positions are filled with the generated results. As a preferred implementation, independent GAIN networks can be trained for different modalities to adapt to the differences in feature distributions of each modality.

[0065] 3. Subtype Tag Construction Method

[0066] To support the model's refined identification of PCOS subtypes, this invention designs a two-stage label generation system for the model training phase, enabling the final labels to possess data-driven objective distribution characteristics and interpretability of medical semantics.

[0067] Specifically, eight quantitative variables closely related to PCOS subtypes were extracted from the training samples, including BMI, LH, FSH, dehydroepiandrosterone sulfate (DHEAS), sex hormone-binding globulin (SHBG), total testosterone, fasting insulin, and fasting blood glucose. After Z-score standardization, an unsupervised hierarchical clustering algorithm was used, with Euclidean distance as the metric, to cluster all samples in the standardized feature space. The optimal number of clusters, k, was 3 based on clinical recognition of classic subtypes and data distribution characteristics.

[0068] After clustering, two endocrinologists with at least five years of clinical experience performed semantic mapping based on the quantitative feature distribution patterns of each cluster center:

[0069] Metabolic type: The corresponding cluster centers are characterized by high BMI, high fasting insulin, high fasting blood glucose, and low SHBG;

[0070] Reproductive type: The corresponding cluster centers are characterized by high LH, high SHBG, high AMH, and low BMI;

[0071] Background type: The corresponding cluster center indicators show no significant deviation.

[0072] AMH serves as an auxiliary clinical reference indicator for physicians when performing semantic mapping and is not included in the input feature set of the clustering algorithm.

[0073] Based on the above clustering, mixed-type samples are further identified through rule-based post-processing. Specifically, the rule is as follows: samples that simultaneously meet both the conditions of "at least two indicators being within ±0.5 standard deviations of the center of the metabolic cluster" and "at least two indicators being within ±0.5 standard deviations of the center of the reproductive cluster" are selected and additionally labeled as mixed-type. Mixed-type samples can come from either the metabolic or reproductive clusters; if a sample meeting the above conditions appears in the background cluster, it is relabeled as mixed-type. Thus, the final subtype labels include four categories: metabolic, reproductive, mixed, and background. The aforementioned ±0.5 standard deviation threshold is an optimal value validated using clinical datasets; those skilled in the art can adaptively adjust it according to the actual sample distribution characteristics.

[0074] To control annotation quality, the Kappa value of the annotations from two physicians must be no less than 0.8; inconsistent samples are arbitrated by a third senior physician. As a preferred implementation, after annotation is completed, the number of samples for each subtype can be checked, and categories with significantly fewer samples can be balanced in the subsequent training phase using class-weighted loss or resampling strategies.

[0075] 4. Multimodal feature extraction and cross-modal fusion

[0076] After preprocessing and label construction, the data for each modality are fed into the corresponding sub-network to obtain initial feature representations. Ultrasound images are processed using EfficientNet-B0 as the image sub-network, with an input size of [insert size here]. The extracted intermediate feature map is denoted as In this implementation, C is set to 1280, and both H and W are set to 7. The image sub-network uses ImageNet pre-trained weights for transfer learning. During training, the bottom feature extraction layer is frozen, and only the parameters of the top feature mapping layer and subsequent fusion module are fine-tuned to ensure the stability and generalization ability of feature extraction in small sample scenarios. The structured data consists of approximately 25 variables, including age, BMI, waist-to-hip ratio, menstrual cycle days, total testosterone, free testosterone, LH, FSH, E2, fasting blood glucose, fasting insulin, and HOMA-IR. After Z-score standardization, the data is fed into a three-layer multilayer perceptron (MLP) for encoding, with hidden layer dimensions of 256 and 128, respectively, resulting in a 64-dimensional structured feature vector. .

[0077] To fully utilize the correlation structure between different modalities, the multimodal data fusion module uses three parallel channels to collaboratively achieve cross-modal fusion: early fusion is used for strongly correlated modalities, graph neural networks are used to model multimodal relationships for weakly correlated modalities, and finally, global deep fusion is achieved through a collaborative attention mechanism.

[0078] 4.1 Early Fusion and Gated Weighting of Strongly Correlated Modes

[0079] For modalities with strong semantic correlation, such as ultrasound images and structured indices, a strategy combining early fusion and gated weighting is adopted to achieve tightly coupled representation while preserving the discriminative power of the original modality. Specifically, the image feature map is first... The vector is compressed into a one-dimensional vector using global average pooling, and then linearly projected onto a d-dimensional space aligned with the structured features. The structured features are then mapped to the same d-dimensional space using a multilayer perceptron. This process can be formally represented as follows:

[0080]

[0081]

[0082]

[0083] in, This is a global average pooling operation. The linear projection matrix of the image features. This is a multilayer perceptron mapping operation, where d is the aligned unified feature dimension, which is 64 in this implementation.

[0084] After dimensional alignment is completed, the two features are fused using a gated weighting mechanism:

[0085]

[0086]

[0087] in, This is a vector concatenation operation. For the gated parameter matrix, The Sigmoid activation function outputs gate weights with values ​​in the range [0,1]. This represents element-wise multiplication. In a preferred scheme, as a parallel branch, the aligned features from the two branches can also be directly concatenated and fed into downstream classifiers such as XGBoost to preserve the original interaction information between modalities; the outputs of the two branches are weighted and averaged to become the final output of this channel. The weighting can preferably be 0.5, i.e., the two branches are fused with equal weights, or it can be set as a learnable parameter for adaptive adjustment based on the actual training effect.

[0088] 4.2 Graph Neural Network Modeling of Weakly Associated Modes

[0089] For modal pairs with relatively weak semantic associations, such as ultrasound morphology and metabolic parameters, and ultrasound morphology and clinical information, this invention uses graph neural networks to explicitly model their relationship structure, avoiding the introduction of overly strong coupling assumptions in the early stages of fusion.

[0090] Specifically, constructing modal relationship diagrams (Where V is the set of modal nodes, and E is the set of edges between nodes), node set It comprises four nodes, corresponding to ultrasound image features, hormonal features, metabolic features, and clinical features, respectively. The features of each node are uniformly mapped to [the appropriate node type] via linear projection. 3D space, as the initial node feature matrix The initial adjacency matrix is ​​set based on prior medical knowledge. The following node pairs are defined as follows: ultrasound and hormones, hormones and metabolism, and metabolism and clinical information are designated as strongly correlated edges with an initial weight of 1.0; ultrasound and metabolism, and ultrasound and clinical information are designated as weakly correlated edges with an initial weight of 0.3; the weight of self-loops is uniformly set to 1.0. These initial weights are optimal values ​​based on medical priors and can be adjusted according to the actual data distribution. Considering the moderate pathological correlation between hormone indicators and clinical information, the initial weight of the hormone and clinical information node pair is set to 0.7; for other node pairs not explicitly listed, their initial weights can be flexibly set based on medical priors, and all can be adaptively adjusted to the optimal value during training through the aforementioned learnable edge attention mechanism.

[0091] Above this relationship graph, a two-layer graph convolutional network (GCN) is used for message passing between nodes:

[0092]

[0093] in, To add a self-loop adjacency matrix, I is the identity matrix. for The degree matrix, and For learnable weight matrix, The ReLU activation function is used, where l takes the value 0 or 1, corresponding to the graph convolutions in the first and second layers respectively. The output of the second layer is... .

[0094] To automatically determine the strength of intermodal associations and avoid over-reliance on prior adjacency matrices, this invention introduces a learnable edge attention mechanism during training. For any edge in the graph... Its attention weight is calculated using the following formula:

[0095]

[0096] in, For learnable attention vectors, To share the linear transformation matrix, and These are the current layer feature vectors of nodes i and j, respectively, with symbols... This represents vector concatenation; softmax normalizes the edges by the dimension of the outgoing edges of the target node, ensuring that the sum of the attention weights of all outgoing edges of the same node is 1. After training convergence, the average attention weight of each edge is taken. If it is greater than 0.4, the two modalities connected by the edge are determined to be strongly correlated; otherwise, they are determined to be weakly correlated. This achieves automatic determination of correlation strength and serves as an effective supplement to the prior adjacency matrix. The aforementioned 0.4 threshold is the preferred determination threshold and can be adaptively adjusted according to the training dataset.

[0097] After completing the node-level representation update, mean pooling is performed on all node features output by the second-layer graph convolution to obtain the graph-level fused representation:

[0098]

[0099] This graph-level representation is spliced ​​together with the output of the earlier fusion channel in the channel dimension and fed into the subsequent collaborative attention fusion layer.

[0100] 4.3 Global Fusion of Collaborative Attention

[0101] To further explore higher-order semantic relationships between channels, the outputs of the two channels are concatenated and fed into a multi-head cross-attention module for deep fusion. In this implementation, four attention heads are used, with each attention head having a dimension equal to the input feature dimension divided by the number of heads, ensuring that the overall dimension of the concatenated multi-head output matches the input dimension. The multi-head cross-attention module maps the concatenated features to three subspaces: query, key, and value. Through parallel multi-head attention computation, higher-order relationships between modalities are explored, maintaining consistency between the output and input dimensions, ultimately generating a unified multimodal feature representation. This serves as the input to the shared-specific multi-task learning module. As a preferred implementation, the number of attention heads and the intermediate dimension can be adjusted appropriately based on the data size; for scenarios with small sample sizes, the number of attention heads can be reduced to alleviate overfitting.

[0102] 5. Shared-specific multi-task learning mechanism

[0103] The structure of the shared-specific multi-task learning module is as follows: Figure 3 As shown. Unified multimodal feature representation. First, it undergoes further abstraction using a shared feature encoder, outputting a 512-dimensional shared latent semantic representation. The shared encoder consists of two residual blocks used to capture global common characteristics and core discriminative patterns of the PCOS patient population.

[0104] Above the shared semantic representation, two task-specific branches are set up: a risk prediction branch network and a genotyping branch network. The risk prediction branch is a two-layer fully connected structure, with the hidden dimension mapped from 512 to 128 and then back to 1. The output is activated by a Sigmoid function to obtain the binary probability of PCOS risk. The subtype branch is also a two-layer fully connected structure. The hidden dimension is mapped from 512 to 128 and then to 4. The output is activated by Softmax to obtain four subtype probability vectors, including metabolic, reproductive, mixed and background types.

[0105] The two tasks are optimized end-to-end using a joint loss function:

[0106]

[0107] The risk prediction task employs a binary cross-entropy loss:

[0108]

[0109] The classification task employs category-weighted cross-entropy loss:

[0110]

[0111] In the formula, and These are the true labels and predicted probabilities for the risk prediction task; and These are the true label and predicted probability of subtype category k, respectively; K is the total number of subtype categories, which is set to 4 in this implementation. This is the weight coefficient for category k, and its value is inversely proportional to the number of training samples for each subtype. It is used to alleviate the class imbalance problem. These are task weighting coefficients used to dynamically balance the optimization directions of the two tasks.

[0112] To avoid manual adjustments The resulting suboptimal nature, this invention addresses... An adaptive adjustment strategy is adopted. As a preferred implementation, the overall loss can be rewritten based on uncertainty modeling principles as follows:

[0113]

[0114] in, and These are the learnable uncertainty parameters for the corresponding tasks, which are then used as trainable variables and jointly optimized with other network parameters to automatically adjust the relative weights of the loss for each task according to its learning difficulty. Furthermore, a gradient normalization strategy can be employed to dynamically normalize the relative contributions of each task based on the gradient magnitude on the shared layer, further mitigating gradient interference between tasks.

[0115] The entire training process uses the Adam optimizer with an initial learning rate of The learning rate was scheduled using a cosine annealing strategy with a batch size of 16 and a total of 150 training rounds.

[0116] 6. Differential privacy protection mechanism

[0117] Considering that medical data involves patient privacy, this invention introduces a differential privacy protection mechanism during the training process of the shared-specific multi-task learning module, and provides two specific implementation methods according to different deployment scenarios.

[0118] 6.1 Single-machine training scenario

[0119] In a single-machine training scenario, since the training process does not involve multi-party parameter interaction, this invention provides a hidden layer representation of the shared encoder output. Directly add Gaussian noise to achieve patient-level differential privacy protection:

[0120]

[0121] Where C is the upper bound of feature sensitivity, corresponding to the gradient clipping threshold, which is set to 1.0 in this implementation; is the noise multiplier, which is 0.5 in this implementation; I is the identity matrix with the same dimension as the hidden layer feature dimension; The distribution is Gaussian, also known as a normal distribution. The first term within parentheses is the distribution mean, and the second term is the covariance matrix. Under the above parameter configuration, the total privacy budget... Set the relaxation parameter to 0.3. Pick ,satisfy - Definition of differential privacy. This solution can run in a standalone environment without relying on multi-party collaboration, yet it provides formal protection for individual patient privacy.

[0122] 6.2 Multi-center deployment scenario

[0123] In multi-center deployment scenarios, when the system is deployed across multiple medical institutions and data from each party is not allowed to leave their local locations, this invention further provides an extended implementation based on federated learning. Each participant independently computes the model gradient using local PCOS patient data, and the gradient is pruned according to the L2 norm while satisfying the following:

[0124]

[0125] Then Gaussian noise is added to the clipped gradient:

[0126]

[0127] The noisy gradients are weighted and averaged using a secure aggregation protocol, preventing the central server from obtaining the original gradients from any single participant. The aggregated global gradients are used to update the global model and distributed to each participant for the next round of local training. In this implementation, the pruning threshold C is set to 1.0, and the privacy budget for each round of communication is... Take 0.05, total communication rounds T is 60, total privacy budget We set the value to 0.3. This extended solution achieves multi-center collaborative optimization and privacy protection of the model while ensuring that data from all parties remains within their local machines, and can complement the single-machine solution.

[0128] 7. Dual-channel interpretability enhancement mechanism

[0129] This invention superimposes a dual-channel interpretability enhancement module on top of a shared-specific multi-task learning module, providing two independent interpretation channels for image modalities and structured modalities respectively. An example of the module output is shown below. Figure 4 As shown.

[0130] 7.1 Image Interpretation Channel

[0131] The image interpretation channel, based on the Grad-CAM algorithm, generates heatmaps of regions of interest from ultrasound images for risk prediction or classification tasks, thereby locating key spatial regions of interest for the model when making predictions. (This is related to task category scores.) First, calculate the feature map of the last convolutional layer. The gradient on the feature map is calculated and averaged according to spatial location to obtain the channel weights corresponding to the feature map.

[0132]

[0133] Then, all feature maps are linearly weighted according to the above weights and activated by ReLU to obtain the heatmaps for the corresponding categories:

[0134]

[0135] in, For the k-th feature map, Location in the feature map Z is the pixel value of the feature map, which is the product of H and W, representing the total number of pixels in the feature map. The function is a linear rectified function. The generated heatmap is normalized and upsampled before being superimposed onto the original ultrasound image. This allows for a direct visualization of the model's focus when identifying typical PCOS imaging features (such as densely distributed polycystic follicles and enlarged ovarian areas), providing physicians with auxiliary references and verification criteria.

[0136] 7.2 Phenotypic Interpretation Channel

[0137] The phenotypic interpretation channel, based on the SHAP value framework, quantifies the marginal contribution of structured input features to risk prediction or classification decisions. For structured input features... Its SHAP value satisfy:

[0138]

[0139] Where F is the set of all structured input features, and S is any subset of features that does not contain feature i. For model prediction function, and These represent the number of elements in the corresponding set. In engineering implementation, KernelExplainer can be used to estimate the SHAP value based on background samples. After calculation, the features are sorted according to the absolute value of their contribution and output in the form of bar charts and waterfall plots, which facilitates physicians to review the model output from the perspective of mechanism rationality.

[0140] 8. Comprehensive Evaluation System

[0141] This invention establishes a multi-dimensional evaluation system that includes objective indicators and quantifiable subjective indicators, which is used to comprehensively verify the model's classification and prediction capabilities, genotyping capabilities, and interpretability.

[0142] 8.1 Objective Indicators

[0143] Objective metrics include AUC (area under the ROC curve), accuracy, and F1 score, which are used to quantitatively evaluate the model's prediction accuracy in PCOS risk prediction and subtype classification tasks.

[0144] 8.2 Subjective Quantitative Indicators

[0145] Subjective indicators include two categories: Grad-CAM regional matching rate and SHAP distribution rationality score, which transform clinical interpretability into calculable quantitative indicators.

[0146] (1) Grad-CAM region matching rate

[0147] Grad-CAM region matching rate is used to quantify the consistency between the model's region of interest and the lesion regions labeled by clinicians. In its implementation, the normalized heatmap generated by Grad-CAM is first... Perform binarization:

[0148]

[0149] Where the threshold Setting it to 0.5 yields the binary mask for the model's region of interest. Two sonographers with at least five years of experience independently annotated the lesion areas, including areas of dense polycystic follicle distribution and areas of enlarged ovarian volume. The annotated data was the union of the two annotators' findings. This serves as a reference area. The matching rate is calculated using the following formula:

[0150]

[0151] in, Indicates pixel count, This represents a pixel-by-pixel logical AND operation. The average matching rate of the two corresponding physicians is taken as the final indicator, and a matching rate of not less than 75% is considered to indicate consistent regional localization. The above threshold is a preferred criterion and can be adjusted according to application requirements. As a preferred implementation, a standard intersection-over-union ratio (IoU) can also be calculated simultaneously as an auxiliary measure.

[0152]

[0153] The two complement each other, comprehensively measuring the regional positioning accuracy.

[0154] (2) Rationality score of SHAP distribution

[0155] The SHAP distribution rationality score is used to quantify the consistency between the model's perceived contribution direction of structured features and prior clinical mechanisms. After calculating the SHAP value for each input feature for each sample, the mean of the SHAP values ​​for each feature is calculated by grouping by subtype category. The prior directions are defined as follows: an increased LH / FSH ratio is positive (i.e., increased PCOS risk), an increased HOMA-IR is positive, an increased BMI is positive, an increased AMH is positive, an increased total testosterone is positive, an increased SHBG is negative (i.e., decreased PCOS risk), and an increased follicle count is positive. If the sign of the mean SHAP value for a certain feature is consistent with the prior direction, then the SHAP direction for that feature is considered reasonable. The reasonableness score is defined as follows:

[0156]

[0157] To further measure the discriminative power of SHAP contributions across different subtypes, this invention also designs a SHAP subtype discriminative score:

[0158]

[0159] Where K is the total number of subtypes. Let be the SHAP mean of feature i in subtype k. The baseline of the SHAP mean of feature i in the entire sample. Let be the global standard deviation of the SHAP values ​​of feature i in the entire sample. Using the global standard deviation as the normalization denominator can avoid calculation anomalies when the baseline mean is close to zero. The larger the value of this index, the more significant the difference in SHAP contribution of each feature among different subtypes, and the higher the subtype discrimination.

[0160] The aforementioned objective and subjective indicators together constitute a complete evaluation system, which can visually compare and display the subjective evaluation results with the objective evaluation results, making it easier for the clinical team to make a comprehensive judgment on the model output. 9. Specific Implementation Examples

[0162] Example 1: System Construction Example

[0163] This embodiment implements a complete PCOS intelligent assisted screening system under the PyTorch framework. The overall system architecture is as follows: Figure 2 As shown in the diagram, in the multimodal data fusion module, the ultrasound image input size is 224×224×3, and a 1280-dimensional image feature vector is extracted by EfficientNet-B0. The structured data includes approximately 25 variables such as age, BMI, waist-to-hip ratio, menstrual cycle days, total testosterone, free testosterone, LH, FSH, E2, fasting blood glucose, fasting insulin, and HOMA-IR. After Z-score standardization, these variables are input into a three-layer multilayer perceptron with hidden dimensions of 256 and 128 respectively, and output a 64-dimensional structured feature vector.

[0164] In the early fusion channel of strong correlation, the image feature map is compressed to 1280 dimensions by global average pooling, and then subjected to linear projection matrix. The features are mapped to 64 dimensions; simultaneously, the structured features are encoded to 64 dimensions via the aforementioned multilayer perceptron. After the two feature paths are aligned, one path is directly concatenated and fed into the XGBoost classifier, while the other path is fused through a gated weighting mechanism, where the gate parameter matrix... The outputs of the two branches are weighted and averaged to become the output of this channel. In the weakly correlated graph neural network fusion channel, a relationship graph containing four nodes—image, hormone, metabolism, and clinical—is constructed. The features of each node are uniformly mapped to 128 dimensions through linear projection. The initial adjacency matrix is ​​constructed in the aforementioned manner. Two-layer graph convolution is used to complete message passing, and the correlation strength is automatically determined through an edge attention mechanism. The outputs of the two channels are concatenated and then fed into a multi-head cross-attention module with 4 heads for global deep fusion.

[0165] In the shared-specific multi-task learning module, the shared encoder receives the aforementioned unified fusion features and outputs a 512-dimensional shared latent semantic representation after two residual blocks. The risk prediction branch consists of two fully connected layers (512→128→1) that are activated by a sigmoid function and output the disease risk probability. The subtyping branch consists of two fully connected layers (512→128→4) that are activated by a softmax function and output four subtype probability vectors. Training uses the aforementioned joint loss function and task weighting coefficients. The initial value is set to 0.5 and automatically adjusted through uncertainty modeling. A gradient normalization strategy is also introduced to mitigate inter-task interference. The optimizer is Adam, and the initial learning rate is... The learning rate was scheduled using a cosine annealing strategy, with a batch size of 16, and a total of 150 training rounds were performed.

[0166] The dual-channel interpretability enhancement module simultaneously outputs a Grad-CAM heatmap and SHAP contribution ranking for each sample during the inference phase. In a single-machine training scenario, it provides the 512-dimensional hidden layer representation output by the shared encoder. Gaussian noise is added in the manner described above, thereby achieving patient-level differential privacy protection without relying on multi-party collaboration.

[0167] Example 2: Data Processing Flow Example

[0168] This embodiment describes a typical data processing flow of the system in a clinical setting, and the overall flow corresponds to... Figure 1 The steps are as follows. For a suspected PCOS patient to be processed, firstly, their ultrasound images, hormone and metabolic indicators, and clinical information are separated according to modality. The ultrasound images are cropped to the region of interest and scaled to 224×224 pixels. Hormone and metabolic indicators are normalized using Z-score after outlier removal, and categorical variables are encoded using one-hot encoding. For features with a missing rate of less than 20%, a GAIN generative filling network is used for repair as described above; for features with a missing rate of more than 20%, a dynamic compensation acquisition process is triggered.

[0169] The preprocessed modal data are fed into the PCOS intelligent assisted screening system, where they undergo feature extraction, cross-modal fusion, shared coding, and task-specific branching to obtain the patient's PCOS risk probability and four subtype probability vectors. On a typical test set, the system achieves an AUC of 0.94 for risk prediction and a subtype classification accuracy of 82.3%.

[0170] Subsequently, the system outputs decision visualization reference information through a dual-channel interpretability enhancement module: the image interpretation channel generates an ultrasound image heatmap based on the Grad-CAM algorithm, overlaying the key spatial regions of interest to the model on the patient's image as a thermal distribution. Typically, multiple densely populated areas of small follicles at the ovarian margin can be observed. The phenotypic interpretation channel calculates the marginal contribution of each structured input feature based on the SHAP value framework and outputs them in order of absolute contribution value. Typically, indicators such as the LH / FSH ratio, HOMA-IR, and total testosterone are among the top performers. Clinicians can then verify the model output. If the region of interest shown in the heatmap matches the actual densely populated follicle area visible in the ultrasound image, and the features ranked high in the SHAP ranking are consistent with known PCOS pathological mechanisms such as hyperandrogenemia and insulin resistance, then the model's classification conclusion has verifiable mechanistic support and can serve as an auxiliary reference for clinical decision-making. It should be noted that the output of this system is only for auxiliary reference; the final clinical diagnosis should be made by a qualified clinician.

[0171] Example 3: Extended Example of Multi-center Federated Learning

[0172] This embodiment describes an extended implementation of the system in a multi-institutional collaborative training scenario. In this scenario, the PCOS patient data of each participant must not leave their local machine; therefore, a federated learning framework based on differential privacy protection is used for multi-center collaborative training. Each participant independently performs forward computation and backpropagation using its local data, and after obtaining the local gradient, it is pruned according to the L2 norm with a pruning threshold C of 1.0. Subsequently, Gaussian noise is added to the pruned gradient in the manner described above. The noise gradients are obtained. The noise gradients of each participant are weighted and averaged on the central server using a secure aggregation protocol, preventing the server from obtaining the original gradients of any single participant. The aggregation result is used to update the global model and distributed to each participant for the next round of training. The privacy budget for each round of communication in this embodiment... Take 0.05, total communication rounds T is 60, total privacy budget We set the value to 0.3. This extended scheme achieves multi-center collaborative optimization and privacy protection of the model while ensuring that the data of each party does not leave their local machine. It can be regarded as a natural extension of the single-machine differential privacy scheme in a multi-center scenario.

[0173] Example 4: Human-Machine Collaborative Closed-Loop Optimization Example

[0174] This embodiment describes the continuous optimization mechanism after system deployment. After the system is deployed and launched, clinicians review the risk prediction results and subtype classification results output by the model. For cases where the clinician's review conclusion is inconsistent with the model output, the clinician marks the correct result and uploads it to the feedback database. The system triggers an incremental learning process every two weeks, using newly accumulated feedback data to fine-tune the model: during the fine-tuning phase, the learning rate is reduced to... The system was trained for only 10 epochs, with the remaining optimizer configurations remaining consistent with the initial training phase. After fine-tuning and validation on an independent test set, the model was officially deployed. Under typical deployment conditions, after approximately 6 iterations, the system's risk prediction AUC improved from the initial 0.94 to approximately 0.96, and the subtype classification accuracy improved from 82.3% to approximately 86.5%. This closed-loop mechanism allows the system to continuously absorb real-world clinical feedback during long-term use, gradually aligning with the data distribution characteristics of specific clinical scenarios.

[0175] In summary, compared with the prior art, the advantages of the solution provided by this invention include:

[0176] First, the multimodal data fusion module integrates four types of heterogeneous data—ultrasound images, biochemical hormone indicators, metabolic parameters, and clinical information—within a unified framework. It employs three parallel channels—early fusion, graph neural network modeling, and collaborative attention deep fusion—to process modal combinations with different correlation strengths. This allows the system to fully utilize the complementarity of information from each modality and comprehensively characterize the multidimensional pathological features of PCOS in imaging, endocrine, and metabolic dimensions. Compared to a single-modal approach, this significantly improves the system's adaptability to heterogeneous patient populations.

[0177] Second, by using a shared-specific multi-task learning module, the disease risk prediction task and the subtyping task are modeled in a unified manner at the shared encoder level. End-to-end optimization is performed using dynamic weighting strategies such as joint loss function, uncertainty modeling, or gradient normalization. This allows the two tasks to promote each other with the support of shared representations, alleviate gradient interference between tasks, and improve the model's generalization ability under conditions of small samples and class imbalance. Combined with a two-stage subtyping label generation system for the annotation stage, which combines hierarchical clustering with physician semantic mapping, and identifies cross-phenotype samples through hybrid rule post-processing, the system can output four refined subtypes, including metabolic, reproductive, mixed, and background subtypes. Compared with the existing coarse subtype classification, this can better support individualized hierarchical management in clinical practice.

[0178] Third, the dual-channel interpretability enhancement module provides two types of decision-making criteria for the model output: a spatial Grad-CAM heatmap and a feature-dimensional SHAP contribution ranking. Quantifiable indicators such as Grad-CAM region matching rate and SHAP distribution rationality score are used to objectively measure the consistency between the interpretation results and clinical priors. This transforms the model output from a black-box prediction into verifiable and traceable auxiliary reference information, effectively improving the model's credibility and acceptability in real clinical scenarios.

[0179] Fourth, by introducing the GAIN generative imputation network to repair features with moderate missing rates, and combining it with a dynamic compensation acquisition mechanism to handle features with high missing rates, the system can still maintain stable feature expression and prediction performance under the realistic condition that data missingness is common in multimodal medical samples.

[0180] Fifth, a formal privacy guarantee is provided for the system through a differential privacy protection mechanism: in a single-machine training scenario, Gaussian noise is directly added to the hidden layer representation output by the shared encoder to achieve lightweight patient-level privacy protection; in a multi-center deployment scenario, a federated learning framework combined with gradient pruning and Gaussian noise is further used to achieve collaborative training where data from all parties does not leave their local machines. The two implementation methods can be flexibly selected according to the actual deployment environment, enabling the system to maintain both single-machine availability and cross-institutional scalability while protecting patient privacy.

[0181] Sixth, by incorporating the review and feedback of clinicians into the continuous training process of the model through a human-machine collaborative closed-loop optimization mechanism, the system can continuously adapt to the data distribution evolution in real diagnosis and treatment scenarios during long-term deployment, thereby achieving a gradual improvement in performance.

[0182] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanisms, characterized in that, It includes a multimodal data fusion module, a shared-specific multi-task learning module, and a dual-channel interpretability enhancement module connected in sequence; The multimodal data fusion module receives and preprocesses multi-source heterogeneous medical data for auxiliary screening of polycystic ovary syndrome, and generates a unified multimodal feature representation based on the strength of correlation between modalities using a differentiated fusion strategy. The differentiated fusion strategy includes gated weighted early fusion of strongly correlated modalities, graph neural network fusion of weakly correlated modalities, and collaborative attention global fusion of the two-way results. The graph neural network fusion constructs a modal relationship graph using feature vectors of each modality as nodes, sets an initial adjacency matrix based on medical priors, and adaptively adjusts the correlation strength of each edge in the graph through a learnable edge attention mechanism. The shared-specific multi-task learning module includes a shared feature encoder, a risk prediction branch network, and a subtype classification branch network. The shared feature encoder receives a unified multimodal feature representation and outputs a shared latent semantic representation. The two branches output disease risk classification results and subtype classification results, respectively. The two branches achieve end-to-end joint training through a joint loss function. The dual-channel interpretability enhancement module includes an image interpretation channel and a phenotypic interpretation channel. The image interpretation channel generates a visualization of the model's region of interest corresponding to the input ultrasound image, while the phenotypic interpretation channel outputs a quantified result of the contribution of each structured input feature to the prediction result.

2. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The gated weighted early fusion of the strongly correlated modalities is specifically as follows: after the image features are processed by global average pooling and linear projection, they are dimensionally aligned with the structured features mapped by the multilayer perceptron, and then element-wise weighted fusion is performed through learnable gated weights.

3. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The graph neural network fusion of the weakly associated modalities uses a two-layer graph convolutional network for message passing between nodes; the learnable edge attention mechanism calculates the attention weight for each edge in the modal relationship graph, and determines the strength of association between corresponding modal pairs based on the comparison between the average attention weight after training convergence and a preset threshold.

4. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The collaborative attention global fusion specifically involves: concatenating the early fusion output of the gated weighted system with the fusion output of the graph neural network, feeding them into a multi-head cross-attention module to mine high-order semantic relationships between modalities, and generating a final unified multimodal feature representation.

5. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The joint loss function is a weighted sum of the loss from the risk prediction task and the loss from the subtype classification task; the weighting coefficients of the weighted sum adopt an adaptive adjustment strategy, including a dynamic weighting method based on uncertainty modeling or a dynamic balancing method based on gradient normalization.

6. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The system also includes a subtype label construction unit, which is used to extract multiple quantitative variables related to polycystic ovary syndrome subtypes from the training samples, perform unsupervised hierarchical clustering after standardization, generate basic subtype labels based on preset subtype semantic mapping rules, and identify mixed samples through rule post-processing to obtain four types of subtype labels: metabolic, reproductive, mixed, and background.

7. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The dual-channel interpretability enhancement module is also equipped with a clinical consistency quantification unit, which is used to calculate the region matching rate between the image region of interest and the preset lesion region reference mask, as well as the distribution rationality score of the feature contribution direction and the preset medical prior, and to quantitatively evaluate the clinical consistency of the interpretation results.

8. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The system also integrates a differential privacy protection unit, supporting two deployment modes: in a single-machine training scenario, Gaussian noise is added to the shared latent semantic representation to achieve patient-level privacy protection; in a multi-center deployment scenario, Gaussian noise is added after local gradient pruning based on a federated learning framework, and the global model is updated collaboratively through a secure aggregation protocol.

9. The PCOS early screening and subtype identification system based on multimodal modeling and interpretability mechanism according to claim 1, characterized in that: The multimodal data fusion module also has a built-in generative imputation unit, which uses a generative adversarial imputation network to imput features with a missing rate below a preset threshold, and triggers a dynamic compensation acquisition process for features with a missing rate above a preset threshold.

10. A computer-implemented method for multimodal medical data processing and subtype classification of polycystic ovary syndrome, characterized in that, Includes the following steps: S1. Preprocess and extract modal features from the input multi-source heterogeneous medical data used for auxiliary screening of polycystic ovary syndrome to obtain the initial feature representation of each modality; S2. Based on the intermodal correlation strength, a differentiated fusion strategy is adopted to perform cross-modal feature fusion and generate a unified multimodal feature representation. The differentiated fusion strategy includes early fusion with gated weighting of strongly correlated modalities, fusion of graph neural networks of weakly correlated modalities, and global fusion of two-way results with collaborative attention. The graph neural network fusion uses the feature vectors of each modality as nodes to build a graph, sets an initial adjacency matrix based on medical priors, and adaptively adjusts the correlation strength of each edge through a learnable edge attention mechanism. S3. The shared latent semantic representation is extracted by the shared feature encoder, and calculated through the risk prediction branch and the subtype classification branch respectively, and the disease risk classification result and the subtype classification result are output; the two branches are jointly trained end-to-end through the joint loss function. S4. The visualization results of the image modality of interest region and the quantification results of the structured feature contribution are output through a dual-channel interpretability mechanism. The results are used to provide reference information for clinical auxiliary screening for non-diagnostic purposes.

Images