Method and system for multi-disease classification based on semantic guidance hybrid experts

Abstract

This invention provides a semantically guided hybrid expert-based multi-disease classification method and system, relating to the fields of artificial intelligence and medical information technology. The method includes: forming robust disease classification and expert assignment strategies by performing semantic embedding and clustering analysis on data samples; constructing a multi-disease classification network model based on disease categories and the number of experts; the multi-disease classification network model uses a hybrid gating mechanism to generate weight vectors that fuse the classification probabilities output by multiple experts, and is jointly trained using a dynamic classification loss including dynamic grouping to full classification and a gating loss including assignment conflict penalties. This method effectively decomposes complex classification tasks into manageable sub-tasks, handles different disease types or feature patterns through specialized mechanisms, simplifies the learning of complex decision boundaries, and improves the accuracy and robustness of multi-category disease classification.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical information technology, and in particular to a multi-disease classification method and system based on semantically guided hybrid experts. Background Technology

[0002] Medical imaging reports (such as magnetic resonance imaging (MRI) and computed tomography (CT) reports) provide detailed natural language descriptions of medical images, serving as crucial evidence for clinicians in making diagnoses and developing treatment plans. In recent years, developing automated disease classification methods to assist clinical decision-making has become a research hotspot. Early research primarily focused on disease classification based on medical images, resulting in numerous clinical diagnostic frameworks that utilize convolutional neural networks (CNNs) and Transformer models to extract complex 3D image features, aiming to achieve rapid and automated disease diagnosis and significantly reduce the burden on clinicians. However, in clinical settings, 3D medical images have extremely high information dimensionality, often containing complex features difficult to discern with the naked eye, posing a significant challenge even for experienced physicians. More importantly, due to the high dimensionality of the raw image data and the difficulty in learning the optimal correlation between disease types and high-dimensional image features, the accuracy of these image-based methods is often limited when handling challenging, fine-grained disease classification tasks.

[0003] Meanwhile, due to the high cost of medical data collection and annotation, the size of most 3D medical datasets is limited by the sample size. This means that many deep learning-based 3D image classification methods can only achieve very limited diagnostic performance. Furthermore, existing 3D methods are either too simplistic in their extraction and processing of image features, or their design is limited to a specific task or dataset, lacking universality across disease diagnosis, which severely limits their application scenarios. Subsequently, research shifted to disease classification based on imaging reports. The reports are condensed summaries of imaging findings by radiologists, with a much lower information dimension than the original images. This makes learning the association between image representations and disease types easier and more efficient. Existing report classification methods typically use traditional machine learning algorithms or Transformer-based language models (such as BERT, RoBERTa, and their various biomedical variants, ClinicalBERT, RadBERT, etc.) for text encoding and classification prediction.

[0004] Despite the rapid development of deep learning technology in the medical field, existing technologies still have significant shortcomings in achieving high-precision and universal multi-category disease diagnosis: a large amount of non-image modality data (such as image reports, patient information, etc.) is generated naturally during clinical diagnosis. Collecting this data does not require additional costs. However, there is currently a lack of multimodal methods that can make full use of various clinical data such as 3D clinical images and image reports for efficient diagnosis. Many existing multimodal methods are limited to the processing and utilization of 2D medical images and text information, and most of them rely on pre-training processes on publicly available 2D multimodal medical datasets, lacking general support for 3D clinical scenarios.

[0005] Furthermore, while report classification methods effectively reduce data dimensionality, when handling multi-category disease classification tasks, especially when the number of disease types to be distinguished is large or multimodal subtypes are involved, the decision hypersurface that a single classification model needs to learn becomes highly complex. This complexity significantly increases the training difficulty and the risk of overfitting. Simultaneously, some disease types may exhibit multiple significant and distinct patterns or subtypes in the semantic feature space (i.e., multimodal distributions). Traditional single classification models struggle to effectively distinguish diseases with multimodal feature distributions using a unified set of features or classification logic, further exacerbating the difficulty of classification tasks and limiting the model's ability to identify subtle differences.

[0006] Therefore, existing technologies urgently need a new disease classification framework that can effectively decompose complex classification tasks into smaller, more manageable sub-problems and handle different disease types or feature patterns separately through specialized mechanisms to simplify the learning of decision boundaries, thereby improving the accuracy and robustness of multi-category disease classification. Summary of the Invention

[0007] This invention provides a multi-disease classification method and system based on semantically guided hybrid experts to solve one or more technical problems existing in the prior art.

[0008] On the one hand, a multi-disease classification method based on semantically guided hybrid experts is characterized by including: Step 1: Use a pre-trained language model to obtain the semantic embedding vectors of the data samples in the training set; Step 2: Perform cluster analysis based on semantic embedding vectors. Determine the disease category subset, number of experts, and mapping relationship between experts and disease categories based on the clustering results to form a robust expert allocation strategy. Through Step 1 and Step 2, determine the number of experts in the multi-disease classification network model and the types of diseases each expert is responsible for. At the same time, store the cluster center position and expert allocation strategy corresponding to each expert in the semantic prior module.

[0009] Step 3: Construct a multi-disease classification network model based on a robust expert assignment strategy; Step 4: Train the multi-disease classification network model using data samples from the training set; Step 5: Input the text of the medical examination report to be classified into the trained multi-disease classification network model to obtain the final disease classification result; The data sample contains medical examination report texts and their actual disease category labels.

[0010] In one feasible implementation, before determining expert assignments, the clustering results are subjected to disease category denoising processing. Specifically, the method includes: for the first... Clusters, The subset of disease categories it was assigned It only includes disease categories with sufficient sample size, expressed by the formula: , in, This indicates that the training set data samples contain the first... Disease category, Less than or equal to the total number of disease categories contained in the training set data samples , Indicates disease categories in the training set Total number of data samples Indicates the first Disease categories in each cluster Number of data samples This represents the optimal number of semantic clusters. Noise reduction processing ensures that each disease category is assigned to at least one cluster.

[0011] In one feasible implementation, the multi-disease classification network model includes a semantic prior module, a disease classification expert module, a disease category routing module, a gating network module, and a disease classification result output module. The semantic prior module is used to determine the expert assignment weights for input data samples based on the stored semantic prior information. The semantic clustering prior information includes the cluster center positions and expert assignment strategies. The semantic prior module provides prior domain knowledge guidance for gating decisions, guiding experts to make selections and reducing the learning difficulty for experts.

[0012] The disease classification expert module includes multiple disease classification expert units and a shared semantic encoder, used to semantically encode features of input data samples and the disease classification probability vectors given by each disease classification expert for the input data samples; the number of disease classification expert units is determined by the expert allocation strategy; the shared semantic encoder by multiple experts makes the model lightweight while ensuring the consistency of feature expression, enabling the gating network to more accurately combine semantic priors for routing and distribution.

[0013] The disease category routing module is used to map the disease classification probability vector output by the disease classification expert module to a full probability vector, thereby achieving the fusion of the outputs of multiple disease classification expert units.

[0014] The gating network module is used to generate a weight vector for fusing the disease classification probabilities output by each disease classification expert based on the input data samples and the semantic prior information provided by the semantic prior module; by introducing semantic prior information, it ensures that the expert gating conforms to the preset disease clustering distribution.

[0015] The disease classification result output module is used to aggregate the full probability vector obtained from the disease category routing module using the weight vector generated by the gating network module to obtain the final disease classification result; this allows expert activation to both conform to semantic priors and have the flexibility of data-driven approaches.

[0016] In one feasible implementation, the classification probability vector output by the disease classification expert unit is mapped to the total probability vector using a permutation matrix, as expressed by the formula: , , , , in, Indicates the first An expert For input samples The output is the classification probability vector of the disease category it is responsible for; Represents the total probability vector; Represents the permutation matrix; Indicates the first An expert The set of disease categories to be processed greater than 0 and less than or equal to integers, This represents the number of disease classification experts in a multi-disease classification network model; Represents a set of disease categories The cardinality; Describing the permutation matrix The elements in the disease category In category collection Index in Sure; Describing the permutation matrix The element index number in the array.

[0017] In one feasible implementation, the method of aggregating the total probability vector using a weight vector includes: , in, Represents the weight vector generated by the hybrid gating mechanism. The first in One portion, This represents the probability of the final disease classification.

[0018] In one feasible implementation, the method for calculating the weight vector includes: Step a1: The semantic prior module uses the Euclidean distance between the semantic embedding vector of the input sample and each expert cluster center. Calculate prior gating weights The calculation formula includes: , in, for Dumb variables; this is computationally efficient, and the more sufficient the prior knowledge, the more accurate the prior gating weights.

[0019] Step a2, input sample The semantic embedding vector is concatenated with the encoded features output by the disease classification expert module. The concatenated fused features are then input into a gating network to generate a data-driven gating weight vector. ; Step a3, each weight component in the weight vector By the prior gating weights and the data-driven gating weights The calculation is obtained through combination of methods, and the calculation formula includes: , in, Represents the data-driven gating weight vector The first in Each component.

[0020] In one feasible implementation, the loss function used for training the multi-disease classification neural network includes dynamic classification loss and gating loss, and the calculation formula includes: , in Represents dynamic classification loss. Indicates gated loss, This represents a constant factor that controls the weight of the gated loss contribution.

[0021] In one feasible implementation, the method for calculating the dynamic classification loss includes: , , Among them, for the target disease category is The training samples, The loss is defined as the loss calculated from the aggregated predictions of the experts responsible for that disease category. This represents the standard cross-entropy loss. Represents the standard cross-entropy function. For indicator functions, when experts Responsible for disease categories The value is 1 if the condition is met, and 0 otherwise. , This indicates a dependency on the current training round. Dynamic weights are defined as follows: , This indicates the total number of training rounds.

[0022] In one feasible implementation, the gating loss is the cross-entropy loss between the sum of the gating weights of all experts responsible for the target disease category and the expected value, and the calculation method includes: .

[0023] Dynamic classification loss allows for a smooth transition from learning from a specific group of experts to learning from all experts, while gating loss penalizes conflicting expert choices. Compared to traditional classification loss, this invention improves both the professional competence of individual experts and the collaborative capabilities of multiple experts through joint training and optimization.

[0024] On the other hand, the present invention provides a multi-disease classification system based on semantically guided hybrid experts, including an expert allocation strategy determination subsystem and a multi-disease classification subsystem; The expert allocation strategy determination subsystem includes a semantic embedding unit and a clustering analysis unit, used to determine the disease category subset partitioning and expert allocation, forming a robust expert allocation strategy. Specifically, the semantic embedding unit performs semantic embedding on data samples in the training set to obtain semantic embedding vectors; the clustering analysis unit performs clustering analysis based on the semantic embedding vectors and determines the expert allocation strategy based on the clustering results. The expert allocation strategy includes disease category subset partitioning, the number of experts, and the mapping relationship between disease categories and experts. The multi-disease classification subsystem is used to construct a multi-disease classification network model based on an expert allocation strategy, and to train the multi-disease classification network model using data samples from the training set; the data samples to be classified are input into the trained multi-disease classification network model to obtain the final disease classification result; The data sample contains medical examination report texts and their actual disease category labels.

[0025] By adopting the above technical solution, this invention has the following beneficial effects: This invention proposes a novel hybrid expert framework (SGMoE) for multi-disease classification based on medical examination report text. SGMoE first determines the disease category division, the number of experts, and the mapping relationship between experts and disease categories by performing semantic embedding and cluster analysis on the medical examination report text, forming a robust disease expert allocation strategy. Based on this expert allocation strategy, a multi-disease classification network model containing multiple disease classification experts is constructed, allowing different experts to focus only on learning the disease subset with more compact feature distribution assigned to them. This significantly simplifies the decision boundary that each "expert" needs to learn, reduces the overall learning difficulty of the model, and improves the accuracy and robustness of classification. Simultaneously, this expert allocation strategy provides each expert with a clear learning objective that conforms to the inherent distribution of the data, guiding expert specialization from the source and effectively reducing the blindness and difficulty of expert learning.

[0026] In model construction, the multi-disease classification network model employs a hybrid gating mechanism that combines data-driven approaches with prior feature distribution guidance to generate weight vectors that fuse classification probabilities from multiple experts. This ensures, on the one hand, that samples are assigned to the semantically most relevant experts, aligning with the prior patterns of the overall data distribution; on the other hand, the learnable gating network provides flexibility to handle out-of-distribution samples or complex situations. After fusing the two weights, the model can more accurately and rationally route samples to the most suitable experts, achieving efficient and intelligent collaboration among experts. Furthermore, the multi-disease classification experts in the model use a shared encoder. Parameter sharing significantly reduces memory usage and improves inference speed while ensuring consistency in feature representation. This allows the gating network to more accurately combine semantic priors for routing, thereby significantly enhancing its knowledge transfer and generalization capabilities in complex, multi-label medical scenarios while maintaining model lightweightness.

[0027] To effectively train this complex multi-expert collaborative model, SGMoE designed a specialized joint training loss function. This function includes both a dynamic classification loss (dynamic grouping to full classification) and a gating loss with a penalty for conflicting expert selections. The dynamic classification loss dynamically adjusts the learning focus during training, gradually shifting the training focus from specific expert groups responsible for the same category to all experts through weight adjustments over time. This design avoids interference from irrelevant tasks in the early stages, accelerating convergence and improving the specialization of each expert. The gating loss penalizes conflicting expert selections, encouraging the model to assign larger weights only to experts responsible for disease categories, suppressing the activation of irrelevant experts, effectively regulating the behavior of the gating network, and making expert collaboration clearer and more efficient.

[0028] Furthermore, while the structure of the multi-disease classification network model in the SGMoE framework is fixed, its core configurations (such as the number of experts, disease subset partitioning, and the mapping relationship between experts and diseases) are dynamically determined through semantic embedding and clustering. For different disease classification tasks, it is only necessary to rerun the semantic analysis and clustering on the new dataset and update the key information in the semantic prior module, without redesigning the network structure. Attached Figure Description

[0029] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 A flowchart of a multi-disease classification method based on semantically guided hybrid experts provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall design scheme of the SGMoE framework provided by the present invention; Figure 3 A schematic diagram of the expert allocation strategy provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a category routing mechanism that integrates multiple expert outputs, provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of the overall loss calculation method for model training provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] The present invention will be further explained below with reference to specific embodiments.

[0033] This invention proposes a Semantics-Guided Mixture of Experts (SGMoE) framework. The core idea of ​​this framework is to solve the challenge of classifying complex multi-disease texts by using semantic priors to drive a multi-expert division of labor mechanism. Traditional single classifiers need to learn complex decision boundaries among multiple diseases, while the SGMoE framework employs multiple classification "experts," each responsible for processing a specific subset of the feature space (i.e., different disease categories or patterns) based on the semantic characteristics of the medical examination report text. In this way, the model does not need to find a single complex classification boundary, but instead determines multiple relatively simple decision boundaries, thereby significantly simplifying the classification task and reducing its difficulty.

[0034] like Figure 2 The diagram shown illustrates the SGMoE framework technical solution. Figure 2 (a) illustrates the complex distribution of imaging reports in the semantic embedding space (e.g., overlapping areas of diseases A, B, and C). Through semantic clustering and semantic embedding, this distribution is decomposed into several simpler sub-distributions (e.g., category 1, category 2, and category 3). A robust assignment strategy is then used to assign different sub-distributions to different experts for classification, intuitively demonstrating the design philosophy of decomposing complex tasks into sub-problems. SGMoE first robustly determines expert assignments for disease categories through semantic clustering; after determining the expert assignments, the overall architecture of SGMoE is as follows: Figure 2 As shown in (b), SGMoE includes multiple experts determined by semantic clustering. Following each expert is a class routing module used to merge classification results from different disease categories. A gating network is responsible for assigning input samples to appropriate experts. Its gating decisions consider both prior knowledge from semantic clustering and specific features of the input samples. The outputs of different experts are aggregated after class routing based on previous clustering results and data-driven gating to obtain the final diagnostic result.

[0035] Based on the above design concept and SGMoE framework, this invention provides a multi-disease classification method based on semantically guided hybrid experts, including: Step 1: Use a pre-trained language model to obtain the semantic embedding vectors of the data samples in the training set.

[0036] The SGMoE framework aims to partition the disease feature space, enabling each disease classification expert to specialize in classifying a subset of disease categories within a specific feature space. Simultaneously, it decomposes the multimodal distribution of disease categories into multiple simple distributions, each processed separately by a different disease classification expert. To this end, SGMoE employs a pre-trained language model to obtain semantic embeddings from data samples in the training set, resulting in high-dimensional semantic embedding vectors. These vectors constitute points in the feature space. This invention uses a pre-trained RoBERTa model as an example for illustration.

[0037] The data sample contains medical examination report text and its actual disease category label. This invention uses imaging examination reports as an example for illustration. The steps, methods, and the structure of each module and unit in the system are also applicable to other medical examination reports.

[0038] Step 2: Perform cluster analysis based on semantic embedding vectors, and determine the disease category subset, number of experts, and mapping relationship between experts and disease categories based on the clustering results to form a robust expert allocation strategy.

[0039] The essence of expert assignment is to treat categories with similar feature distributions as a cluster. Clustering analysis algorithms are used to cluster the semantic embedding vectors of data samples. In this embodiment, the K-means clustering algorithm is used, and the optimal number of clusters is determined based on the silhouette coefficient. More specifically, the K-means clustering algorithm is used on the imaging reports in the training set of known disease categories. Based on the clustering results, all disease categories are divided into multiple disease category subsets. Each disease classification expert is responsible for one disease category subset. Different experts may be responsible for overlapping disease categories, but the union of all expert-responsible disease category subsets can cover all disease categories. Since semantic clustering results are insensitive to input perturbations, it ensures that similar samples are always assigned to the same expert. Each expert focuses on processing samples in a specific semantic region, forming an expert-semantic region correspondence.

[0040] like Figure 3 As shown, a schematic diagram illustrating expert allocation with two clusters (clustering groups) is used to explain the robust expert allocation strategy provided by this invention. As mentioned above, SGMoE first uses a pre-trained RoBERTa model to extract the semantic embeddings of all training reports, representing them as points in a high-dimensional feature space. Then, the K-means clustering algorithm is used to group these semantic embedding points. For ease of understanding, Figure 3 The left side shows an example of the distribution after dimensionality reduction of high-dimensional feature clustering data. The clear distinction between cluster group 1 (square points) and cluster group 2 (triangular points) on the two-dimensional plane vividly illustrates this process. The optimal number of clusters is automatically determined by the silhouette coefficient to ensure compactness within clusters and good separation between different clusters. In this way, each cluster defines a semantically similar disease feature subspace, providing a basis for expert division of labor. Figure 3On the right, assume there are two disease experts (Expert 1 and Expert 2) and four disease categories (A, B, C, D). Each expert is trained to specialize in handling diseases within a specific cluster subspace. Disease features in cluster 1 are primarily handled by Expert 1, and disease features in cluster 2 are primarily handled by Expert 2. It's important to note that the data distribution for clustering in practice is complex. For example, subspace 1 after clustering might contain all four disease categories A, B, C, and D. A threshold of 1 / N (where N is the number of experts and also the number of subspaces, meaning one expert corresponds to one subspace) is set. If samples of category A in subspace 1 account for more than 1 / N of the total number of samples in category A, then that category is assigned to Expert 1; otherwise, category A is not included in subspace 1. This decomposes the multimodal disease category distribution into multiple simple distributions, which are then assigned to different experts for processing.

[0041] Furthermore, since semantic embeddings may contain noise and / or clustering bias, some clusters may contain only a very small number of disease categories. Directly assigning all disease categories in a cluster to the same disease classification expert could lead to severe class imbalance. Therefore, before determining expert assignments, the clustering results are subjected to disease category denoising processing. Specific methods include: For the Clusters The subset of disease categories it was assigned It only includes disease categories with sufficient sample size, expressed by the formula: , in, This indicates that the training set data samples contain the first... Disease category, Less than or equal to the total number of disease categories contained in the training set data samples That is, the total number of target disease categories. Indicates disease categories in the training set Total number of data samples Indicates the first Disease categories in each cluster Number of data samples This represents the optimal number of semantic clusters.

[0042] The above formula can eliminate samples that are mis-clustered due to noise, while ensuring that each disease category is assigned to at least one cluster. It should be noted that after denoising, some clusters may contain only a single disease category. In this case, the classification task for the disease classification expert becomes extremely simple, causing the expert to lose the crucial ability to distinguish similar diseases. To solve this problem, disease categories in clusters containing only a single category are merged and assigned to the same expert. Samples in these clusters are compactly distributed in the feature space, making them easier to distinguish.

[0043] Assuming that the expert allocation strategy guided by the above semantic clustering is ultimately determined 1 disease classification expert, of which the first An expert Responsible for processing disease category subsets That is, classifying all types of diseases into Each disease belongs to at least one disease category subset, and each disease classification expert is responsible for one disease category subset.

[0044] Through steps 1 and 2, SGMoE decomposes the original problem with complex decision boundaries into multiple simpler sub-problems based on the semantics of medical examination reports. Furthermore, unlike data-driven expert specifications in traditional standard expert hybrid systems, the expert allocation strategy in this invention naturally incorporates prior knowledge about feature distributions, reducing the difficulty of expert learning.

[0045] Step 3: Construct a multi-disease classification network model based on a robust expert assignment strategy.

[0046] Furthermore, the multi-disease classification neural network model includes a semantic prior module, a disease classification expert network module, a disease category routing module, a gating network module, and a disease classification result output module.

[0047] The semantic prior module is used to determine the expert assignment weights for the input data samples based on the stored semantic prior information. The semantic clustering prior information includes the cluster center location and the expert assignment strategy. This module generates prior expert weights by calculating the distance between the semantic embedding of the input sample and the responsibility area (cluster center location) of each expert, quantifies the semantic fit between the sample and each expert, provides domain knowledge guidance for gating decision-making, and reduces the learning difficulty for experts.

[0048] The disease classification expert network module includes multiple disease classification expert units and a semantic encoder shared by the experts, used to semantically encode features of the input data samples, as well as the disease classification probability vector given by each disease classification expert for the input data samples; the number of disease classification expert units is determined by the expert allocation strategy.

[0049] Unlike existing architectures where experts are composed of independent encoders, this model employs a shared semantic encoder. Existing models often allocate a complete end-to-end network to each expert, leading to a significant increase in parameters with the number of experts. Furthermore, because each expert operates in an isolated encoding space, it is difficult to capture the common patterns of medical semantics. In contrast, this model utilizes a shared encoder to uniformly map radiology reports to the same high-dimensional semantic manifold, with experts consisting only of the top-level classification head. This design not only significantly reduces memory usage and improves inference speed through parameter sharing, but more importantly, it ensures the consistency of feature representation. This allows the gating network to more accurately combine semantic priors for routing and distribution, thereby significantly enhancing its knowledge transfer and generalization capabilities in complex, multi-label medical scenarios while maintaining a lightweight model.

[0050] The disease category routing module is used to map the disease classification probability vector output by the disease classification expert network module into a total probability vector.

[0051] The gated network module is used to generate a weight vector for fusing the disease classification probabilities output by each disease classification expert based on the input data samples and the semantic prior information provided by the semantic prior module. By introducing prior distribution knowledge into the gated network, multiple experts can collaborate more effectively.

[0052] The disease classification result output module is used to aggregate the full probability vector obtained from the disease category routing module using the weight vector generated by the gating network module to obtain the final disease classification result.

[0053] Furthermore, the disease classification expert network module in SGMoE uses RoBERTa as the base network, with subsequent connections including classification heads containing linear transformations and softmax activation functions; it is also compatible with other network structures with the same functionality, such as BERT, ClinicalBERT, CLIP, etc., as classification networks based on this network.

[0054] Furthermore, since the same input sample may activate multiple disease classification expert units, it is necessary to fuse the classification probability results output by different expert units. However, the classification probability vectors of different experts correspond to different category combinations, making direct fusion impossible. Therefore, the SGMoE framework designs a category routing mechanism (disease category routing module) to fuse the outputs of multiple disease classification expert units. Specifically, the classification probability vectors output by each disease classification expert unit are mapped to a full probability vector using a permutation matrix. Specific methods include: , , , , in, Indicates the first An expert For input samples The output is the classification probability vector of the disease category it is responsible for; Represents the total probability vector; Represents the permutation matrix; Indicates the first An expert The set of disease categories to be processed greater than 0 and less than or equal to integers, This represents the number of disease classification experts in a multi-disease classification network model; Represents a set of disease categories The cardinality; This represents the total number of target disease categories, i.e., the total number of disease categories contained in the data samples in the training set; Describing the permutation matrix The elements in the disease category In category collection Index in Sure; Describing the permutation matrix The element index number in the permutation matrix is ​​used to... Mid-positioning expert The output of the first Which global disease category does the dimensional probability correspond to? For example, the first disease category for Expert 2 corresponds to... The middle corresponds to the fourth category It equals 4; The number of disease classification experts in a multi-disease classification network model is less than or equal to .

[0055] Through this operation, the output of all disease classification expert units is reshaped into a format of length [length missing]. vector Furthermore, the categories of each element in the vector are aligned, and the probability value corresponding to the category not assigned to this expert is 0. For example... Figure 4 The diagram illustrates a category routing mechanism that integrates the outputs of multiple experts, assuming there are a total of 16 disease categories and designated disease classification experts. The disease classification experts are responsible for determining the second and third disease categories. The given disease classification probability vector The length is 2, obtained through the permutation matrix. Mapped to a full probability vector of length 16 .

[0056] Furthermore, to better integrate the outputs of multiple disease classification expert units, the gating network module in the SGMoE framework employs a hybrid gating mechanism to calculate the gating weights for each expert. Unlike standard hybrid expert models, the gating mechanism in this invention incorporates prior knowledge of semantic clustering in addition to referencing the input sample data, enabling expert activation to both conform to semantic priors and possess data-driven flexibility.

[0057] The semantic prior module is the core of SGMoE's ability to implement hybrid gating, combined with... Figure 2 Explain its operating logic. Input data sample. First, the data is encoded by two encoders. The first encoder uses the same encoding model as the clustering (and is not trained), resulting in feature f1. Prior distances are calculated between f1 and the location of each cluster center (each cluster corresponds to one expert), and these distances are converted into a set of expert weights, i.e., prior gating weights. Then, f1 and f2 are encoded into feature f2 by a shared encoder fine-tuned using LoRA. Both f1 and f2 are then input into a gating network to obtain the second set of expert weights, i.e., data-driven gating weights. The two sets of expert weights are combined to obtain the final weights. f2 is then input to each expert for classification. Each expert's classification result is mapped from their assigned subset to the entire class, and then weighted with the corresponding expert weights to obtain the final classification result.

[0058] Specifically, the semantic prior module internally stores the data from the offline phase ( Figure 2 (a) Using the N semantic cluster centers determined by k-means clustering, calculate the current input sample. The Euclidean distance between the semantic vector and the cluster center location corresponding to each expert. If an expert is responsible for multiple sub-clusters, the minimum distance among them is taken. The distance is transformed into a probability distribution using the Softmax function, with closer distances receiving higher weights.

[0059] Prior gating weights represent the prior probability, from a purely semantic perspective, that the current input sample belongs to the jurisdiction of the m-th expert, ensuring that expert gating conforms to the pre-defined disease clustering distribution. Data-driven gating weights are generated in real-time by a multilayer perceptron (MLP) gating network based on the features of the current input sample, providing flexibility in handling anomalous samples. Although mathematically all experts participate in the computation, under the constraint of gating loss during model training, the model spontaneously generates sparsity, assigning high weights only to those experts who are semantically most relevant to the sample, while suppressing the activation of irrelevant experts, thereby achieving accurate classification decisions and enabling experts to collaborate more effectively.

[0060] Furthermore, the weight vector generated using the hybrid gating mechanism is used to adjust the total probability vector. The aggregation process yields the final disease classification results. Specific methods include: , in, This indicates that the hybrid gating mechanism generates the weight vector. One of the components, This represents the final disease classification probability vector.

[0061] This aggregation process ensures that the final classification decision integrates the expertise of all experts, while prioritizing the contributions of the most relevant experts based on the gating output. Each weight component in the weight vector... By prior gating weights and data-driven gating weights The composition and specific calculation methods include: Step a1: The semantic prior module uses the Euclidean distance between the semantic embedding vector of the input sample and each expert cluster center. Calculate prior gating weights The calculation formula includes: , in, for dummy variables.

[0062] It is important to note that for disease classification experts responsible for multiple single-category clusters, the minimum distance between them and each cluster center should be used for calculation. This method is highly efficient, and the more comprehensive the prior knowledge, the more accurate the prior gating weights.

[0063] Step a2, input sample The semantic embedding vector is concatenated with the encoded feature vector output by the disease classification expert module. The concatenated fused feature is then processed by a multilayer perceptron (MLP) to generate a data-driven gating weight vector. ; For example, the MLP contains three fully connected layers. The number of units in the first two hidden layers is 768 and 1536, respectively. The first two layers use the ReLU activation function, and the last layer uses the softmax activation function.

[0064] Step a3, each weight component in the weight vector By the prior gating weights and the data-driven gating weights The calculation is obtained through combination of methods, and the calculation formula includes: , in, Represents the data-driven gating weight vector Any component in.

[0065] Step 4: Train the multi-disease classification network model using data samples from the training set.

[0066] like Figure 5 The diagram illustrates the overall loss calculation method provided in this embodiment of the invention. The SGMoE framework utilizes input data samples and their actual disease category labels to jointly train the expert network and the gating network. Since SGMoE involves a complex design of multiple experts (each with different distribution patterns) and their collaboration mechanisms, to better achieve the fusion of expert learning and results, SGMoE uses dynamic classification loss and gating loss to train the multi-disease classification neural network. Dynamic classification loss allows for a smooth transition from learning from a specific expert group to learning from all experts, while gating loss penalizes conflicting expert choices. Compared to traditional classification losses, this invention improves both the learning ability of expert norms and collaboration through joint training and optimization.

[0067] Furthermore, the overall loss for training a multi-disease classification neural network includes dynamic classification loss and gating loss, and the calculation formula includes: , in, Represents dynamic classification loss. Indicates gated loss, This is a constant factor representing the weight of the control gating loss. Dynamic classification loss. Classification loss based on standard cross-entropy An improvement is made by adjusting the weights along the time dimension, gradually shifting the training focus from specific expert groups responsible for the same category to all experts. This design avoids experts being misled by irrelevant sub-problems before they fully understand their assigned category. Furthermore, to improve the learning process of expert aggregation, SGMoE proposes a gated loss. This is used to penalize invalid experts whose assignments contradict expert allocations.

[0068] Furthermore, the dynamic classification loss Design such as Figure 5 As shown in the lower left section, the standard cross-entropy loss is used... Loss Classification by Experts Introduce time-varying weights between them, so that Experts who specialize in dealing with the same subproblems The loss is defined as the loss calculated from the aggregated predictions of the experts responsible for that category. Methods for calculating dynamic classification loss include: , , Among them, for the target disease category is The training samples, The loss is defined as the loss calculated from the aggregated predictions of the experts responsible for that disease category. This represents the standard cross-entropy loss. Represents the standard cross-entropy function. For indicator functions, when experts Responsible for disease categories The value is 1 if the condition is met, and 0 otherwise. , This indicates a dependency on the current training round. Dynamic weights are used to ensure that training initially focuses on relevant expert groups and suppresses the influence of irrelevant experts. The value of gradually decays as training progresses, defined as . , This indicates the total number of training rounds, and the target disease category is the actual disease category of the data sample.

[0069] Furthermore, gate loss Design such as Figure 5 As shown in the right half of the diagram, the gating mechanism aims to improve the expert selection strategy of the gating network and enhance expert collaboration. This is achieved by penalizing the gating weights assigned to experts responsible for non-target categories. Specifically, for disease categories... The training samples, gating loss encourages only the use of samples responsible for disease categories. The experts are assigned relatively large weights. It should be noted that because multiple disease classification experts may share responsibility for the same disease category, it's impossible to determine which expert should be activated for a specific sample; however, at least one expert responsible for that disease category must be activated. Therefore, The design draws on the concept of multi-instance learning, optimizing the sum of the weights of all experts responsible for the target disease category.

[0070] The gating loss is the cross-entropy loss between the sum of the gating weights of all experts responsible for the target disease category and the expected value, and the calculation method includes: .

[0071] The above formula is the binary cross-entropy loss, which is calculated for all categories responsible for the target disease. The sum of the weights of the disease classification experts.

[0072] The multi-disease classification network model constructed and trained using the methods and steps described above simplifies the model structure, reduces the learning difficulty for experts, and improves the accuracy of multi-disease classification by introducing prior information and multi-expert collaboration. To verify the effectiveness, advantages, and versatility of the semantically guided hybrid expert classification framework based on imaging reports designed in this invention, it was validated on multiple different datasets and compared with other models. Details can be found in the experimental testing and data sections below.

[0073] Step 5: Input the medical examination report text to be classified into the trained multi-disease classification neural network to directly obtain the final disease classification result.

[0074] Example 2 like Figure 2 As shown, this embodiment provides a multi-disease classification system based on semantically guided hybrid experts. The system includes: an expert allocation strategy determination subsystem and a multi-disease classification subsystem. The expert allocation strategy determination subsystem includes a semantic embedding unit and a clustering analysis unit, used to determine the disease category subset partitioning and expert allocation, forming a robust expert allocation strategy. Specifically, the semantic embedding unit performs semantic embedding on data samples in the training set to obtain semantic embedding vectors; the clustering analysis unit performs clustering analysis based on the semantic embedding vectors and determines the expert allocation strategy based on the clustering results. The expert allocation strategy includes disease category subset partitioning, the number of experts, and the mapping relationship between disease categories and experts. The multi-disease classification subsystem is used to construct a multi-disease classification network model based on an expert allocation strategy, and to train the multi-disease classification network model using data samples from the training set; the data samples to be classified are input into the trained multi-disease classification network model to obtain the final disease classification result; The data sample contains medical examination report texts and their actual disease category labels.

[0075] Furthermore, the multi-disease classification network model includes a semantic prior module, a disease classification expert module, a disease category routing module, a gating network module, and a disease classification result output module; The semantic prior module is used to determine the expert assignment weights for the input data samples based on the stored semantic prior information, wherein the semantic clustering prior information includes the cluster center position and the expert assignment strategy. The disease classification expert module includes multiple disease classification expert units and a semantic encoder shared by the experts, used to perform semantic encoding features on the input data samples, as well as the disease classification probability vector given by each disease classification expert for the input data samples. The disease category routing module is used to map the disease classification probability vector output by the disease classification expert module to the full probability vector. The gated network module is used to generate a weight vector for fusing the disease classification probabilities output by various disease classification experts, based on the input data samples and the semantic prior information provided by the semantic prior module. The disease classification result output module is used to aggregate the full probability vector obtained from the disease category routing module using the weight vector generated by the gating network module to obtain the final disease classification result.

[0076] Experimental tests and data To verify the effectiveness, advantages, and versatility of the semantically guided hybrid expert classification framework based on radiology reports designed in this invention, the invention was validated on three different datasets. Specifically, this invention used three datasets: Dataset BT16: An internal dataset containing 11,864 brain MRI reports for classifying brain tumors in 16 different fine-grained categories. Dataset BD7: An internal dataset containing 5,400 brain MRI reports for classifying brain diseases in 7 different coarse-grained categories. Dataset CT-RATE: A publicly available dataset of chest CT reports containing 1,862 reports for classifying 40 different combinations of presentations (categories).

[0077] To verify the effectiveness of the method designed in this invention, several mainstream text classification models were compared in the experiment. Specifically, the comparison models included: LSTM: Represents the traditional sequence model; RoBERTa: A robustly optimized general language model; BioBERT, ClinicalBERT, RadBERT, PubMedBERT, and BlueBERT are all Transformer-based models, but they have been pre-trained on corpora in specific domains such as biomedical, clinical, or radiological reports.

[0078] The performance of each model on the three tasks is shown in Table 1, where classification accuracy (ACC) and F1 score were used as metrics to evaluate model performance.

[0079] Table 1: Performance comparison of the SGMoE model on three datasets (ACC% / F1%)

[0080] As shown in Table 1, the SGMoE framework achieved consistently best classification results across all evaluation metrics on three datasets with different tasks and numbers of classes, demonstrating significant performance improvements over traditional text classification models and advanced domain-specific pre-trained models. Since this invention uses the same base encoder as RoBERTa, SGMoE's average ACC (80.8%) across all tasks is a significant 4.2% improvement over RoBERTa's (76.6%). This fully and directly proves the effectiveness of the core technical solution designed in this invention—namely, decomposing complex multi-class problems through semantically guided expert assignment.

[0081] Furthermore, this invention was compared with current state-of-the-art large-scale language models (LLMs), including domain-specific (e.g., Radiology-LLaMA2, PMC-LLaMA-13B, LLaMA-medx-v3.2) and general (e.g., LLaMA-3.1-8B-Instruct). The comparisons were performed on the BT16 and BD7 datasets, and the results are shown in Table 2.

[0082] Table 2: Performance Comparison of the Model and the Fine-tuned Large Language Model (LLM) (ACC% / F1%)

[0083] As shown in Table 2, on the more challenging BT16 fine-grained classification task, the SGMoE of this invention (72.5% ACC) outperforms all compared LLMs, including the domain-tuned LLaMA-medx-v3.2 (71.1% ACC). On the BD7 task, although PMC-LLaMA-13B and LLaMA-medx-v3.2 perform well, the SGMoE of this invention still achieves comparable performance (78.1% ACC) and outperforms LLaMA-3.1 (77.9% ACC). More importantly, this invention has a significant advantage in terms of computing resources: SGMoE training and inference require only about 14GB and 6GB of GPU memory, respectively, while LLMs (such as PMC-LLaMA-13B) require up to 80GB and 50GB of GPU memory, respectively. This invention achieves superior performance on complex tasks and comparable performance on other tasks with lower resource consumption, making it more suitable for regular practical deployments.

[0084] In addition, the present invention was compared with domain experts (radiologists) on a subset of the BT16 dataset, as shown in Table 3.

[0085] Table 3: Performance Comparison of Models and Domain Experts on the Brain Tumor BT16 Dataset

[0086] As can be seen from Table 3, the classification accuracy of the SGMoE of this invention (65.6%) is better than that of a senior expert with 15 years of experience (61.5%), demonstrating that the invention has the potential to surpass human experts when dealing with complex, fine-grained multi-class classification tasks.

[0087] In this invention, all patient data used in the experimental verification phase were anonymized at Beijing Tiantan Hospital. The downloaded report data underwent a preprocessing operation to standardize its length (long reports were truncated, and short reports were padded with zeros).

[0088] The technical solutions and comparative methods used in the experiments provided in this invention were all written in Python, version 3.8. The experiments were deployed on an NVIDIA RTX 3090 GPU, running Ubuntu 22.04. (Gated loss) Weighting factors The classification performance on the validation set was used to determine the model's performance. The RoBERTa encoder in SGMoE was initialized with default pre-trained weights, and then fine-tuned using the LoRA method based on the training data. The LoRA rank was set to 16, and the alpha value was set to 32. A task-specific classification head was also attached and trained. Different experts used different random seeds for initialization. The model was trained using the AdamW optimizer with a learning rate set to... The batch size is 32, and the training epochs are 200 to ensure model convergence.

[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-disease classification method based on semantically guided hybrid experts, characterized in that, include: Step 1: Use a pre-trained language model to obtain the semantic embedding vectors of the data samples in the training set; Step 2: Perform cluster analysis based on semantic embedding vectors, and determine the disease category subset, number of experts, and mapping relationship between experts and disease categories based on the clustering results to form a robust expert allocation strategy; Step 3: Construct a multi-disease classification network model based on a robust expert assignment strategy; Step 4: Train the multi-disease classification network model using data samples from the training set; Step 5: Input the text of the medical examination report to be classified into the trained multi-disease classification network model to obtain the final disease classification result; The data sample contains medical examination report texts and their actual disease category labels.

2. The multi-disease classification method according to claim 1, characterized in that, Before determining the expert assignments, the clustering results are subjected to disease category denoising processing. Specific methods include: for the... Clusters, The subset of disease categories it was assigned It only includes disease categories with sufficient sample size, expressed by the formula: , in, This indicates that the training set data samples contain the first... Disease category, Less than or equal to the total number of disease categories contained in the training set data samples , Indicates disease categories in the training set Total number of data samples Indicates the first Disease categories in each cluster Number of data samples This represents the optimal number of semantic clusters.

3. The multi-disease classification method according to claim 1, characterized in that, The multi-disease classification network model includes a semantic prior module, a disease classification expert module, a disease category routing module, a gating network module, and a disease classification result output module; The semantic prior module is used to determine the expert assignment weights for the input data samples based on the stored semantic prior information, wherein the semantic clustering prior information includes the cluster center position and the expert assignment strategy. The disease classification expert module includes multiple disease classification expert units and a semantic encoder shared by the experts, used to perform semantic encoding features on the input data samples, as well as the disease classification probability vector given by each disease classification expert for the input data samples. The disease category routing module is used to map the disease classification probability vector output by the disease classification expert module to the full probability vector. The gated network module is used to generate a weight vector for fusing the disease classification probabilities output by various disease classification experts, based on the input data samples and the semantic prior information provided by the semantic prior module. The disease classification result output module is used to aggregate the full probability vector obtained from the disease category routing module using the weight vector generated by the gating network module to obtain the final disease classification result. The number of disease classification expert units is determined by the expert allocation strategy.

4. The multi-disease classification method according to claim 3, characterized in that, The classification probability vector output by the disease classification expert unit is mapped to the total probability vector using a permutation matrix, as expressed by the formula: ， ， ， ， in, Indicates the first An expert For input samples The output is the classification probability vector of the disease category it is responsible for; Represents the total probability vector; Represents the permutation matrix; Indicates the first An expert The set of disease categories to be processed greater than 0 and less than or equal to integers, This represents the number of disease classification experts in a multi-disease classification network model; Represents a set of disease categories The cardinality; Describing the permutation matrix The elements in the disease category In category collection Index in Sure; Describing the permutation matrix The element index number in the array.

5. The multi-disease classification method according to claim 4, characterized in that, Methods of aggregating the total probability vector using weight vectors include: ， in, Represents the weight vector generated by the hybrid gating mechanism. The first in One portion, This represents the probability of the final disease classification.

6. The multi-disease classification method according to claim 5, characterized in that, The method for calculating the weight vector includes: Step a1: The semantic prior module uses the Euclidean distance between the semantic embedding vector of the input sample and each expert cluster center. Calculate prior gating weights The calculation formula includes: ， in, for dummy variables; Step a2, input sample The semantic embedding vector is concatenated with the encoded features output by the disease classification expert module. The concatenated fused features are then input into a gating network to generate a data-driven gating weight vector. ; Step a3, each weight component in the weight vector By the prior gating weights and the data-driven gating weights The calculation is obtained through combination of methods, and the calculation formula includes: ， in, Represents the data-driven gating weight vector The first in Each component.

7. The multi-disease classification method according to claim 1, characterized in that, The loss function used for training a multi-disease classification neural network includes dynamic classification loss and gating loss, and the calculation formulas are as follows: ， in Represents dynamic classification loss. Indicates gated loss, This represents a constant factor that controls the weight of the gated loss contribution.

8. The multi-disease classification method according to claim 7, characterized in that, The method for calculating the dynamic classification loss includes: ， ， Among them, for the target disease category is The training samples, The loss is defined as the loss calculated from the aggregated predictions of the experts responsible for that disease category. Represents the standard cross-entropy loss. Represents the standard cross-entropy function. For indicator functions, when experts Responsible for disease categories The value is 1 if the condition is met, and 0 otherwise. , This indicates a dependency on the current training round. The dynamic weight is defined as follows: , This indicates the total number of training rounds.

9. The multi-disease classification method according to claim 7, characterized in that, The gating loss is the cross-entropy loss between the sum of the gating weights of all experts responsible for the target disease category and the expected value, and the calculation method includes: 。 10. A multi-disease classification system employing any one of the methods described in claims 1-9, characterized in that, The system includes: an expert allocation strategy determination subsystem and a multi-disease classification subsystem. The expert allocation strategy determination subsystem includes a semantic embedding unit and a clustering analysis unit, used to determine the disease category subset partitioning and expert allocation, forming a robust expert allocation strategy. Specifically, the semantic embedding unit performs semantic embedding on data samples in the training set to obtain semantic embedding vectors; the clustering analysis unit performs clustering analysis based on the semantic embedding vectors and determines the expert allocation strategy based on the clustering results. The expert allocation strategy includes disease category subset partitioning, the number of experts, and the mapping relationship between disease categories and experts. The multi-disease classification subsystem is used to construct a multi-disease classification network model based on an expert allocation strategy, and to train the multi-disease classification network model using data samples from the training set; the data samples to be classified are input into the trained multi-disease classification network model to obtain the final disease classification result; The data sample contains medical examination report texts and their actual disease category labels.

Images