Intelligent ear-nose-throat disease screening system and method based on multi-modal data

Abstract

The invention belongs to the technical field of disease screening, and particularly relates to an ear-nose-throat disease intelligent screening system and method based on multi-modal data, and the method comprises the steps: constructing an ear-nose-throat anatomical parameterized model according to a clinical image, and generating a virtual image; extracting interpretable anatomical features according to the mixed image data set, mapping the interpretable anatomical features to an ear-nose-throat disease knowledge graph, and establishing a diagnosis association rule through predicate logical reasoning; the method comprises the following steps: reading pixel features of a clinical image, matching a map rule, iterating a closed loop through a manual correction rule, and converting diagnosis adjustment into map rule supplementation; constructing a federal rule sharing framework, uploading rule correction parameters, and aggregating results through a federal average algorithm; and designing a hierarchical screening process according to the diagnosis rule model, and pushing high-confidence positive cases to superior hospitals to form a screening system. According to the invention, on the basis of virtual-real fusion data construction, a highly-adaptive and universal ear-nose-throat disease intelligent screening system is constructed through symbolized logical reasoning and federal rule sharing.

Description

Technical Field

[0001] This invention belongs to the field of disease screening technology, specifically an intelligent screening system and method for ear, nose and throat diseases based on multimodal data. Background Technology

[0002] The intelligent ENT screening system is primarily based on deep learning and computer vision technologies. This technology originates from the standardization of medical data, algorithm optimization, and enhanced computing power, aiming to assist doctors in making efficient and objective preliminary diagnoses. It plays a particularly important role in primary healthcare and early cancer screening, and is a typical application of the close integration of artificial intelligence and clinical medicine.

[0003] Traditional intelligent screening systems for ear, nose, and throat diseases suffer from significant technical and application shortcomings. They are heavily reliant on data and suffer from insufficient sample coverage, often relying on real clinical images for training. This makes it difficult to obtain samples of rare lesions, leading to uneven data distribution, low model accuracy for screening rare cases, and a lack of mechanisms for integrating virtual and real data and adapting to different domains, resulting in poor data universality. The diagnostic process is opaque, often employing deep learning black-box models that fail to clearly present the logical relationship between features and lesions, resulting in a lack of interpretability in the output results. Clinicians struggle to verify diagnostic evidence, limiting trust and applicability. Furthermore, they have poor scenario adaptability, relying on high-end equipment and complex model inference. Due to equipment and technical barriers, these systems are difficult to implement in primary healthcare institutions, failing to support the needs of tiered medical services. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes an intelligent screening system for ear, nose, and throat diseases based on multimodal data. This invention primarily addresses the problems of traditional intelligent screening systems for ear, nose, and throat diseases, such as difficulty in obtaining rare lesion samples and the lack of virtual-real data fusion and domain adaptation mechanisms.

[0005] The intelligent screening system for ear, nose and throat diseases based on multimodal data provided by this invention includes: a virtual and real data module, which is used to construct an anatomical parametric model of the ear, nose and throat based on clinical images, define lesion features in combination with pathological knowledge, generate virtual images through a regularization algorithm, align virtual and real features through adversarial domain adaptation, and output a hybrid image dataset.

[0006] The rule modeling module is used to extract interpretable anatomical features from the mixed image dataset, map them to the ENT disease knowledge graph, establish diagnostic association rules through predicate logic reasoning, and output a symbolic diagnostic reasoning model.

[0007] The feature matching module is designed based on the symbolic diagnostic reasoning model. It reads the pixel features of clinical images, matches them with atlas rules, and outputs the feature matching confidence score.

[0008] The iterative optimization module is used to feed back the feature matching confidence score to the doctor. Through manual correction and rule iteration closed loop, the diagnosis adjustment is transformed into a graph rule supplement, the feature matching weight is optimized, and the iterative diagnostic reasoning rule set is output.

[0009] The federated aggregation module is used to build a federated rule sharing framework based on the iterative diagnostic reasoning rule set, upload rule correction parameters, aggregate the results through the federated averaging algorithm, and output a diagnostic rule model.

[0010] The tiered screening module is used to design tiered screening processes based on diagnostic rule models. Primary healthcare institutions can directly conduct initial screening for ear, nose, and throat diseases through the diagnostic rule models and push high-confidence cases to higher-level hospitals, forming a screening system that does not rely on equipment and outputting ear, nose, and throat disease screening results.

[0011] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes a virtual and real data module comprising:

[0012] The parametric modeling unit is used to extract the coordinates and morphological parameters of key anatomical sites such as vocal cord curvature and sinus ostium diameter in ENT clinical images, construct an anatomical parametric model, and output anatomical parameter templates.

[0013] The virtual generation unit is used to define the anatomical distortion features and thresholds corresponding to the lesion based on the anatomical parameter template and pathological knowledge, and uses a regularization algorithm to drive the template to deform according to the distortion rules, and outputs a virtual image set.

[0014] The domain adaptation unit is used to align the feature distributions of the virtual image set and the original clinical image set with adversarial domain adaptation, and output a hybrid image dataset.

[0015] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes a rule modeling module comprising:

[0016] The feature extraction unit is used to extract interpretable anatomical features that are strongly correlated with clinicopathology from the mixed image dataset, quantize the features into structured vectors, and output a standard anatomical feature set.

[0017] The atlas mapping unit is used to associate each quantified feature with an entity node in the ENT disease knowledge graph based on the standard anatomical feature set, annotate the correspondence between features and lesion types and grades, and output a feature-lesion association atlas.

[0018] The rule generation unit is used to construct diagnostic association rules based on the characteristic lesion association map using first-order predicate logic reasoning, convert the rules into symbolic expressions, and output the diagnostic reasoning model.

[0019] According to the intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention, the specific steps for outputting the diagnostic reasoning model in the rule generation unit are as follows:

[0020] Based on the feature lesion association map, first-order predicate logic is used to sort out the causal relationship that the feature threshold satisfies the lesion determination, and output a preliminary causal logic list.

[0021] Based on the preliminary causal logic list, a confidence interval is set for each logical rule, the scope of lesions to which the rule applies and the boundary conditions are determined, and a draft diagnostic rule is formed.

[0022] Based on the draft diagnostic rules, the rules described in natural language are transformed into standardized symbolic logical expressions, and a diagnostic reasoning model is output.

[0023] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes a feature matching module comprising:

[0024] The rule parsing unit is used to parse the standardized symbolic logical expressions in the symbolic diagnostic reasoning model, extract feature types, threshold ranges, and confidence benchmarks, and transform them into a structured rule matching library.

[0025] The feature extraction unit is used to read the pixel features of clinical images and perform quantization processing according to the feature types defined in the structured rule matching library, and generate a set of clinical image feature vectors.

[0026] The precise matching unit compares the clinical image feature vector set with the feature threshold range in the structured rule matching library, calculates the matching fit based on the confidence benchmark, and outputs the feature matching confidence score.

[0027] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes the following specific steps for outputting the feature matching confidence score in the precise matching unit:

[0028] Establish a corresponding association between the clinical image feature vector set and the structured rule matching library, determine the feature parameter value, threshold range and confidence benchmark corresponding to each rule, and output the feature rule matching index table.

[0029] The deviation between the feature parameter value and the threshold range is calculated based on the feature rule matching index table. It is then determined whether the parameter meets the threshold condition. If it does, the state is marked as valid, and the rule parameter lookup table is output. Otherwise, the matching fit of the rule is assigned to zero.

[0030] Calculate the fit of valid rules based on the rule parameter lookup table, summarize the fit values ​​of all rules, and output the feature matching confidence score table.

[0031] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes an iterative optimization module comprising:

[0032] The feedback acquisition unit is used to synchronize the feature matching confidence score and corresponding clinical images to the doctor's end, present the details of each feature matching and the preliminary diagnostic tendency, and form a structured correction record.

[0033] The rule transformation unit is used to sort out the feature adjustment direction and threshold optimization range corresponding to the correction opinions based on the structured correction records, and transform the diagnostic adjustment content into new rules that can be added to the knowledge graph.

[0034] The weight optimization unit is used to integrate new rules into the existing diagnostic reasoning rule system, adjust the matching weights of corresponding features according to clinical credibility, and construct a temporary rule set that takes into account both the original logic and clinical correction experience.

[0035] The closed-loop validation unit is used to re-match historical clinical image samples based on the temporary rule set, compare and validate the diagnostic accuracy and confidence distribution of the optimized rules, and output an iterative diagnostic inference rule set.

[0036] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes a federated aggregation module comprising:

[0037] The framework building unit is used to build a federated rule sharing framework based on the core principles of the rule set and output the federated rule sharing framework.

[0038] The parameter extraction unit is used to guide participating institutions to adapt their applications based on local clinical image samples and iterative diagnostic reasoning rule sets according to the federal rule sharing framework, calculate the correction coefficients of the rules in the local scenario, and output the rule correction parameter set.

[0039] The parameter aggregation unit is used to perform weighted aggregation calculations on homogeneous parameters using a federated average algorithm based on the rule-corrected parameter set. Weights are assigned according to the sample size and data quality of each institution, and the aggregated rule parameter set is output.

[0040] The model generation unit is used to embed the aggregated rule parameter set into the original framework of the iterative diagnostic reasoning rule set, complete the global optimization of rule feature weights and judgment thresholds, and generate a diagnostic rule model.

[0041] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by the present invention includes a hierarchical screening module comprising:

[0042] The process adaptation unit is used to extract the core diagnostic rules and feature judgment thresholds from the diagnostic rule model, transform them into simple screening logic that can be executed at the grassroots level, and output a set of basic rules for hierarchical screening.

[0043] The initial screening execution unit is used to perform feature matching on the ear, nose, and throat images or physical signs data of the examinee based on the general terminal at the grassroots level, perform initial screening diagnosis according to the basic rule set of stratified screening, generate confidence scores for each case, and output a list of cases screened at the grassroots level.

[0044] The case triage unit is used to determine the triage threshold based on the confidence score of cases in the primary screening case list, mark high-confidence positive cases as key cases that require further diagnosis, organize basic case information and initial screening data, and output a case transfer list.

[0045] The results integration unit is used to compare and correlate the initial screening data at the grassroots level with the confirmed diagnosis conclusions at the higher level based on the case transfer list, correct the confidence threshold of the basic rule set for stratified screening, and output the screening results for ear, nose and throat diseases.

[0046] This invention also provides an intelligent screening method for ear, nose, and throat diseases based on multimodal data, including:

[0047] A parametric model of otolaryngology anatomy is constructed based on clinical images. Pathological features are defined by combining them with pathological knowledge. Virtual images are generated through a regularization algorithm. Virtual and real features are aligned by adversarial domain adaptation, and a hybrid image dataset is output.

[0048] Interpretable anatomical features are extracted from the mixed image dataset, mapped to the ENT disease knowledge graph, diagnostic association rules are established through predicate logic reasoning, and a symbolic diagnostic reasoning model is output.

[0049] A lightweight feature matching module is designed based on the symbolic diagnostic reasoning model. It reads the pixel features of clinical images and matches them with atlas rules, and outputs the feature matching confidence score.

[0050] The feature matching confidence score is fed back to the doctor. Through a closed loop of manual correction and rule iteration, the diagnosis adjustment is transformed into a graph rule supplement, the feature matching weight is optimized, and an iterative diagnostic reasoning rule set is output.

[0051] A federated rule sharing framework is constructed based on the iterative diagnostic reasoning rule set. Rule correction parameters are uploaded, and the results are aggregated by the federated averaging algorithm to output a diagnostic rule model.

[0052] Based on the diagnostic rule model, a tiered screening process is designed. Primary healthcare institutions can directly conduct initial screening for ear, nose, and throat diseases through the diagnostic rule model and push high-confidence cases to higher-level hospitals, forming a screening system that does not rely on equipment and outputs the screening results for ear, nose, and throat diseases.

[0053] The intelligent screening system for ear, nose, and throat diseases based on multimodal data provided by this invention has the following beneficial effects:

[0054] 1. This invention constructs a parametric anatomical template through a virtual-real data module, defines a distortion threshold based on pathological knowledge, generates highly realistic virtual lesion images, and then aligns the feature distribution of the virtual and real images using unsupervised adversarial domain adaptation technology to output a hybrid image dataset. This dataset retains the real anatomical features of clinical images while supplementing rare lesion samples through virtual generation, addressing the pain points of scarce and unevenly distributed clinical data in ENT diseases. Simultaneously, the parametric modeling unit accurately extracts core anatomical parameters such as vocal cord curvature and sinus ostium diameter, laying a quantitative foundation for subsequent feature analysis, significantly improving the completeness and reliability of diagnostic data, and providing high-quality data support for rule-based modeling.

[0055] 2. This invention, through a federated rule parameter sharing framework and a manual correction-rule iteration closed loop, aggregates multi-center rule correction parameters instead of transmitting original data and models. Under the premise of ensuring the privacy and security of medical data, it integrates multi-center clinical experience, continuously optimizes diagnostic logic, significantly improves the system's adaptability and generalization ability in different medical scenarios, and achieves dynamic and accurate matching between technology and clinical practice.

[0056] 3. This invention constructs a hierarchical screening system and a lightweight rule matching module to extract core rules suitable for primary care. It can complete primary screening and accurate triage of high-confidence cases at the primary care level by relying on general-purpose terminals. This breaks the dependence of traditional intelligent diagnosis on high-end equipment and professional technology, reduces the technical threshold and equipment investment cost of primary care, promotes the large-scale implementation of early screening and diagnosis of ear, nose and throat diseases, and improves the screening efficiency and coverage of the overall medical service system. Attached Figure Description

[0057] The invention will now be further described with reference to the accompanying drawings.

[0058] Figure 1 This is a block diagram of the intelligent screening system for ear, nose, and throat diseases based on multimodal data provided in an embodiment of the present invention;

[0059] Figure 2 This is a flowchart of the intelligent screening system for ear, nose, and throat diseases based on multimodal data provided in an embodiment of the present invention;

[0060] Figure 3 This is a flowchart illustrating the steps of the intelligent screening method for ear, nose, and throat diseases based on multimodal data provided in this embodiment of the invention. Detailed Implementation

[0061] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below according to specific embodiments.

[0062] like Figures 1 to 3 As shown in the figure, the intelligent screening system for ear, nose, and throat diseases based on multimodal data provided in this embodiment of the invention includes:

[0063] The virtual and real data module is used to construct parametric anatomical templates based on key anatomical sites in clinical images, define anatomical distortion thresholds for lesions by combining pathological knowledge, generate virtual lesion images by driving template distortion through a regularized algorithm, and align the anatomical feature distribution of virtual and real images through unsupervised adversarial domain adaptation to output a hybrid image dataset.

[0064] The parametric modeling unit is used to extract the coordinates and morphological parameters of key anatomical sites such as vocal cord curvature and sinus ostium diameter in ENT clinical images, construct an anatomical parametric model, and output anatomical parameter templates.

[0065] The virtual generation unit is used to define the anatomical distortion features and thresholds corresponding to the lesion based on the anatomical parameter template and pathological knowledge, and uses a regularization algorithm to drive the template to deform according to the distortion rules, and outputs a virtual image set.

[0066] The domain adaptation unit is used to align the feature distributions of the virtual image set and the original clinical image set with adversarial domain adaptation, and output a hybrid image dataset.

[0067] The rule modeling module is used to extract interpretable anatomical feature vectors that are strongly correlated with clinical pathology from the mixed image dataset, map them to entity nodes in the ENT disease knowledge graph, establish diagnostic association rules of anatomical feature distortion-lesion type-grading standard through first-order predicate logic reasoning, and output a symbolic diagnostic reasoning model with transparent decision links.

[0068] The feature extraction unit is used to extract interpretable anatomical features that are strongly correlated with clinicopathology from the mixed image dataset, quantize the features into structured vectors, and output a standard anatomical feature set.

[0069] Based on the mixed image dataset, dominant anatomical features strongly correlated with clinical pathology, such as vocal cord curvature, sinus cavity density, and ossicular chain angle, were selected. Redundant features due to noise interference were eliminated to complete the selection and localization of core features. The selected features were quantified according to anatomical standard thresholds, converting physical indicators such as curvature and density into standardized numerical vectors, unifying the dimensions and value ranges of the features. The quantified feature vectors were classified and archived according to lesion type, establishing a one-to-one correspondence between image features and numerical vectors, and outputting a standard anatomical feature set.

[0070] The atlas mapping unit is used to associate each quantified feature with an entity node in the ENT disease knowledge graph based on the standard anatomical feature set, annotate the correspondence between features and lesion types and grades, and output a feature-lesion association atlas.

[0071] Import the standard anatomical feature set and the knowledge graph of ear, nose and throat diseases, match each quantized vector in the feature set with the lesion entity node in the graph, and label the lesion type and pathological stage corresponding to the feature.

[0072] Based on clinical consensus, weighting coefficients are added to the matched feature-lesion associations to distinguish between core diagnostic features and auxiliary reference features, thereby enhancing the clinical relevance of the associations.

[0073] By integrating all matching results and weight information, a visual feature-lesion association map is constructed, clearly presenting the correspondence logic between features and lesions, and outputting an association map that can be directly used for rule reasoning.

[0074] The rule generation unit is used to construct diagnostic association rules based on the feature lesion association map, using first-order predicate logic reasoning, to determine the logical relationship between feature thresholds and lesion determination, to convert the rules into symbolic expressions, and to output a diagnostic reasoning model.

[0075] Based on the characteristic lesion association map, first-order predicate logic is used to analyze the causal relationship between feature threshold satisfaction and lesion determination. Considering the clinical risk of missed / misdiagnosis, a confidence interval is set for each logical rule, clarifying the applicable lesion range and boundary conditions, thus forming a preliminary draft of diagnostic rules. The natural language-described rule draft is then transformed into symbolic logical expressions, and a unified symbol system is used to standardize the rule expression, outputting a symbolic diagnostic reasoning model that can be directly embedded into the system.

[0076] The feature matching module is designed based on a symbolic diagnostic reasoning model to create a lightweight module that directly reads pixel-level anatomical features from clinical images and matches them to a map rule base. It does not require complex model reasoning and outputs preliminary screening conclusions and feature matching confidence scores.

[0077] The rule parsing unit is used to parse the standardized symbolic logical expressions in the symbolic diagnostic reasoning model, extract feature types, threshold ranges, and confidence benchmarks, and transform them into a structured rule matching library.

[0078] The standardized first-order predicate logic expression of the symbolic diagnostic reasoning model is parsed, and three core elements are extracted from the expression: feature type, threshold range, and confidence benchmark. These elements are then classified and integrated according to the anatomical location of the ear, nose, and throat and the type of lesion. The abstract symbolic rules are transformed into a structured rule matching library containing feature identifier, threshold range, confidence coefficient, and lesion mapping fields.

[0079] The feature extraction unit is used to read the pixel features of clinical images and perform quantization processing according to the feature types defined in the structured rule matching library, and generate a set of clinical image feature vectors.

[0080] Feature extraction templates are constructed based on all feature types defined in the structured rule matching library and their corresponding anatomical localization requirements. The ENT clinical images to be screened are imported, and the anatomical structure pixel regions in the images that match the feature types are located using image segmentation algorithms. Targeted quantization algorithms are used to numerically transform the pixel regions, converting unstructured pixel information into standardized numerical parameters. According to the feature sorting order in the structured rule matching library, all quantization parameters are integrated into a clinical image feature vector set with consistent dimensions and uniform format.

[0081] The precise matching unit compares the clinical image feature vector set with the feature threshold range in the structured rule matching library, calculates the matching fit based on the confidence benchmark, and outputs the feature matching confidence score.

[0082] The clinical image feature vector set and structured rule matching library are compared feature by feature. First, the difference between each individual feature parameter in the vector set and the corresponding threshold range in the rule library is calculated to determine if the parameter falls within the valid threshold range. If the parameter meets the threshold condition, the matching degree of a single rule is calculated based on the confidence benchmark in the rule library using a bias weighting coefficient. If the parameter does not meet the threshold condition, the matching degree of that rule is directly assigned to zero. After completing the comparison calculation for all features, the matching degree values ​​for each rule are summarized to generate a feature matching confidence score table.

[0083] The formula for the deviation weighting coefficient is expressed as:

[0084]

[0085] In the formula, Let be the bias weighting coefficient for the i-th feature. The absolute value of the threshold deviation of the i-th feature. is the threshold interval width for the i-th feature.

[0086] The formula for single-rule matching fit is expressed as:

[0087]

[0088] In the formula, Let be the matching score for the i-th rule. Let be the confidence baseline value for the i-th rule. Let be the threshold deviation of the i-th feature parameter, and be the deviation weighting coefficient.

[0089] The formula for summing the total confidence scores is as follows:

[0090]

[0091] In the formula, The total confidence score for feature matching. Let be the clinical weight coefficient of the i-th feature, and n be the total number of features involved in the matching.

[0092] The iterative optimization module is used to feed back the initial screening results and confidence scores to the doctor's interface, build a closed loop of manual correction and rule iteration, transform the doctor's diagnostic adjustments into rule supplements for the knowledge graph rather than model parameter updates, optimize the logical weights of feature matching, and output the iterative diagnostic reasoning rule set.

[0093] The feedback acquisition unit is used to synchronize the feature matching confidence score and corresponding clinical images to the doctor's end, present the details of each feature matching and the preliminary diagnostic tendency, and form a structured correction record.

[0094] The rule transformation unit is used to sort out the feature adjustment direction and threshold optimization range corresponding to the correction opinions based on the structured correction records, and transform the diagnostic adjustment content into new rules that can be added to the knowledge graph.

[0095] Based on the clinical diagnostic correction opinions, differences in image feature annotations, and the basis for adjusting diagnostic conclusions contained in the structured correction records, the system systematically sorts out the direction of image feature adjustment corresponding to each correction opinion, clarifies the optimization range of key features such as lesion boundary clarity and density value range, and extracts the diagnostic logic connections implicit in the correction opinions. These fragmented diagnostic adjustment contents are standardized and structured to form new rule entries that can be directly added to the medical imaging diagnostic knowledge graph. These new rule entries include both the judgment criteria of feature dimensions and the derivation logic of diagnostic conclusions, constituting the core output of the rule transformation unit.

[0096] The weight optimization unit is used to integrate new rules into the existing diagnostic reasoning rule system, adjust the matching weights of corresponding features according to clinical credibility, and construct a temporary rule set that takes into account both the original logic and clinical correction experience.

[0097] The new rules are integrated into the existing diagnostic reasoning rule system. By combining indicators such as the credibility of clinical cases and the degree of expert consensus corresponding to the new rules, the matching weights of image features related to the new rules in the original rule system are dynamically adjusted. For features corresponding to new rules with sufficient clinical validation and high credibility, their matching weights in diagnostic reasoning are increased. For features that are controversial or have few validation samples, their weights are appropriately reduced. Through this iterative optimization of weights, a temporary rule set is constructed that retains the original diagnostic logic framework while integrating the latest clinical correction experience. This temporary rule set serves as the core input for the weight optimization unit to pass down.

[0098] The closed-loop validation unit is used to call the temporary rule set to re-match the features of historical clinical image samples, compare and validate the diagnostic accuracy and confidence distribution of the optimized rules, and output the iterative diagnostic reasoning rule set after confirming that it meets clinical needs.

[0099] Based on the temporary rule set, full feature matching and diagnostic reasoning are performed on historical clinical image samples. By comparing the diagnostic results derived from the temporary rule set with the actual clinical diagnostic conclusions of historical samples, the core indicators such as diagnostic accuracy, sensitivity, and specificity of the optimized rules are statistically calculated. At the same time, the distribution of diagnostic results in different confidence intervals is analyzed to verify the applicability and stability of the temporary rule set in different types of image samples and different disease classification scenarios. When the verification results confirm that they meet the clinical diagnostic needs, the temporary rule set is formally determined as the iterative diagnostic reasoning rule set and output, realizing the process connection from rule adjustment to verification closed loop.

[0100] The federated aggregation module is used to build a federated rule parameter sharing framework based on the iterative diagnostic inference rule set. Each institution only uploads the rule correction coefficients, and the rule parameters are aggregated by the federated averaging algorithm instead of the model weights to avoid data privacy risks. The output is a unified diagnostic rule model with multi-center generalization ability, without any model transmission or training throughout the process.

[0101] The framework building unit takes the iterative diagnostic reasoning rule set as its core input, deeply analyzes the dimensions of ENT disease image feature judgment, the weight allocation logic of each feature, and the derivation paradigm from feature matching to diagnostic conclusion output covered in the rule set. Based on this, a federated rule sharing framework is built. During the framework building process, the parameter upload standards of each participating medical institution are clarified, and the specific parameter types and format requirements such as rule correction coefficients and threshold fine-tuning parameters that need to be uploaded are defined. At the same time, cross-institutional interaction protocols are formulated to standardize parameter transmission processes, data encryption methods, and node communication rules, delineate the permission boundaries of each institution, and clarify the permission scope of operations such as data viewing, parameter modification, and result retrieval. The federated rule sharing framework is then output, laying the foundation for subsequent multi-center parameter collaboration.

[0102] The parameter extraction unit guides participating institutions in adapting their local clinical image samples and iterative diagnostic reasoning rule sets according to the federated rule sharing framework. Each institution runs the rule set independently locally, compares the diagnostic results output by the rule set with the actual local clinical diagnostic conclusions, locates the adaptation deviation points of the rules in the local scenario, and then calculates the rule correction coefficient. At the same time, it adjusts the feature judgment threshold and obtains the threshold fine-tuning parameter by combining the feature distribution characteristics of the local samples, optimizes the matching weight of each feature and generates feature weight adaptation values. Each institution summarizes and organizes these local adaptation parameters, and finally outputs a rule correction parameter set containing correction coefficients, threshold fine-tuning parameters and feature weight adaptation values. The entire process does not require uploading local original image data and model files.

[0103] The parameter aggregation unit is used to modify the parameter set according to rules. It employs a federated averaging algorithm to perform weighted aggregation calculations on homogeneous parameters, assigning corresponding weights based on the sample size and data quality of each institution. This eliminates single-center data bias and outputs an aggregated rule parameter set that integrates multi-center clinical experience. During the calculation process, corresponding weight coefficients are assigned based on the local sample size and data quality level of each participating institution. Institutions with larger sample sizes and higher data quality have higher parameter weights, thus ensuring the reliability of the aggregation results. The weighted averaging operation eliminates rule adaptation bias caused by uneven distribution of single-center data and sample bias, effectively integrating the local modification experience of each institution. Finally, it outputs a set of aggregated rule parameter sets that integrates multi-center clinical practice experience and has global adaptability.

[0104] The model generation unit is used to back-embed the aggregated rule parameter set into the original framework of the iterative diagnostic reasoning rule set, complete the global optimization of rule feature weights and decision thresholds, and generate a unified diagnostic rule model that takes into account both multi-center generalization ability and clinical diagnostic accuracy, without any original data or model transmission throughout the process.

[0105] The tiered screening module is used to design tiered screening processes based on diagnostic rule models. Primary care staff only need to use ordinary imaging tools to read image features, input them into the rule model to complete the initial screening, and push high-confidence positive cases to higher-level hospitals for review, forming a screening system that does not rely on additional equipment and outputs ENT disease screening results with reasoning basis.

[0106] The process adaptation unit is used to extract the core diagnostic rules and feature judgment thresholds applicable to grassroots scenarios from the diagnostic rule model, remove complex feature items that require professional equipment support, transform them into simple screening logic that can be executed at the grassroots level, and output a hierarchical screening basic rule set adapted to grassroots conditions. This rule set clarifies the core features and judgment criteria for initial screening.

[0107] The initial screening execution unit is used to obtain the basic rule set for stratified screening, perform feature matching on the ear, nose and throat images or physical signs data of the examinee based on the general terminal at the grassroots level, complete the initial screening diagnosis according to the rule set, generate confidence scores for each case simultaneously, and output a list of grassroots screening cases containing the initial screening results and confidence labels.

[0108] The case triage unit is used to determine the triage threshold based on the confidence score of cases in the primary screening case list, mark high-confidence positive cases as key cases that require further diagnosis, organize basic case information and initial screening data, and output a case transfer list marked with triage indicators.

[0109] The results integration unit is used to correlate and compare the initial screening data at the grassroots level with the confirmed diagnosis conclusions at the higher level based on the case transfer list, correct the confidence threshold of the basic rule set for stratified screening, and output a complete ENT disease screening result report containing the initial screening results, triage path and confirmed diagnosis conclusion.

[0110] like Figure 3 As shown, the present invention also provides an intelligent screening method for ear, nose, and throat diseases based on multimodal data, comprising:

[0111] A parametric model of otolaryngology anatomy is constructed based on clinical images. Pathological features are defined by combining them with pathological knowledge. Virtual images are generated through a regularization algorithm. Virtual and real features are aligned by adversarial domain adaptation, and a hybrid image dataset is output.

[0112] Interpretable anatomical features are extracted from the mixed image dataset, mapped to the ENT disease knowledge graph, diagnostic association rules are established through predicate logic reasoning, and a symbolic diagnostic reasoning model is output.

[0113] A lightweight feature matching module is designed based on the symbolic diagnostic reasoning model. It reads the pixel features of clinical images and matches them with atlas rules, and outputs the feature matching confidence score.

[0114] The feature matching confidence score is fed back to the doctor. Through a closed loop of manual correction and rule iteration, the diagnosis adjustment is transformed into a graph rule supplement, the feature matching weight is optimized, and an iterative diagnostic reasoning rule set is output.

[0115] A federated rule sharing framework is constructed based on the iterative diagnostic reasoning rule set. Rule correction parameters are uploaded, and the results are aggregated by the federated averaging algorithm to output a diagnostic rule model.

[0116] Based on the diagnostic rule model, a tiered screening process is designed. Primary healthcare institutions can directly conduct initial screening for ear, nose, and throat diseases through the diagnostic rule model and push high-confidence cases to higher-level hospitals, forming a screening system that does not rely on equipment and outputs the screening results for ear, nose, and throat diseases.

[0117] In summary, this embodiment provides an intelligent screening system and method for ENT diseases based on multimodal data. It constructs parameterized anatomical templates through virtual and real data modules, defines distortion thresholds based on pathological knowledge, and generates highly realistic virtual lesion images. These images are then aligned with the feature distribution of the virtual and real images using unsupervised adversarial domain adaptation technology, outputting a hybrid image dataset. This dataset retains the true anatomical features of clinical images while supplementing rare lesion samples through virtual generation, addressing the pain points of scarce and unevenly distributed clinical data for ENT diseases. Simultaneously, the parameter modeling unit accurately extracts core anatomical parameters such as vocal cord curvature and sinus ostium diameter, laying a quantitative foundation for subsequent feature analysis, significantly improving the completeness and reliability of diagnostic data, and providing high-quality data support for rule-based modeling.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent screening system for ear, nose, and throat diseases based on multimodal data, characterized in that, include: The virtual and real data module is used to construct an ENT anatomical parametric model based on clinical images, define lesion features by combining pathological knowledge, generate virtual images through a regularization algorithm, align virtual and real features through adversarial domain adaptation, and output a hybrid image dataset. The rule modeling module is used to extract interpretable anatomical features from the hybrid image dataset, map them to the ENT disease knowledge graph, establish diagnostic association rules through predicate logic reasoning, and output a symbolic diagnostic reasoning model. The feature matching module is used to design a lightweight feature matching module based on the symbolic diagnostic reasoning model, read the pixel features of clinical images and match them with atlas rules, and output the feature matching confidence score. The iterative optimization module is used to feed back the feature matching confidence score to the doctor's end, and transform the diagnosis adjustment into a graph rule supplement through manual correction-rule iteration closed loop, optimize the feature matching weight, and output the iterative diagnosis reasoning rule set; The federated aggregation module is used to construct a federated rule sharing framework based on the iterative diagnostic reasoning rule set, upload rule correction parameters, aggregate the results through a federated averaging algorithm, and output a diagnostic rule model. The tiered screening module is used to design a tiered screening process based on the diagnostic rule model. Primary healthcare institutions can directly conduct initial screening for ear, nose, and throat diseases through the diagnostic rule model and push high-confidence cases to higher-level hospitals, forming a screening system that does not rely on equipment and outputting ear, nose, and throat disease screening results.

2. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The virtual and real data modules include: The parametric modeling unit is used to extract the coordinates and morphological parameters of key anatomical sites such as vocal cord curvature and sinus ostium diameter in ENT clinical images, construct an anatomical parametric model, and output anatomical parameter templates. The virtual generation unit is used to define the anatomical distortion features and thresholds corresponding to the lesion based on the anatomical parameter template and pathological knowledge, use a regularization algorithm to drive the template to deform according to the distortion rules, and output a virtual image set. The domain adaptation unit is used to align the feature distributions of the two types of images—the virtual image set and the original clinical images—through adversarial domain adaptation to output a hybrid image dataset.

3. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The rule modeling module includes: The feature extraction unit is used to extract interpretable anatomical features that are strongly correlated with clinicopathology from the mixed image dataset, quantize the features into structured vectors, and output a standard anatomical feature set. The atlas mapping unit is used to associate each quantified feature with an entity node in the ENT disease knowledge graph according to the standard anatomical feature set, annotate the correspondence between features and lesion types and grades, and output a feature-lesion association atlas. The rule generation unit is used to construct diagnostic association rules based on the feature lesion association map using first-order predicate logic reasoning, convert the rules into symbolic expressions, and output a diagnostic reasoning model.

4. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 3, characterized in that: The specific steps for outputting the diagnostic reasoning model in the rule generation unit are as follows: Based on the feature lesion association map, first-order predicate logic is used to sort out the causal relationship that the feature threshold satisfies the lesion determination, and output a preliminary causal logic list. Based on the preliminary causal logic list, a confidence interval is set for each logic rule to determine the lesion range and boundary conditions to which the rule applies, thus forming a draft diagnostic rule; Based on the aforementioned draft diagnostic rules, the rules described in natural language are transformed into standardized symbolic logical expressions, and a diagnostic reasoning model is output.

5. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The feature matching module includes: The rule parsing unit is used to parse the standardized symbolic logical expression in the symbolic diagnostic reasoning model, extract feature types, threshold ranges, and confidence benchmarks, and transform them into a structured rule matching library. The feature extraction unit is used to read the pixel features of the clinical image and perform quantization processing according to the feature types defined in the structured rule matching library to generate a clinical image feature vector set. The precise matching unit is used to compare the clinical image feature vector set with the feature threshold range in the structured rule matching library, calculate the matching fit degree in combination with the confidence benchmark, and output the feature matching confidence score.

6. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 5, characterized in that: In the precise matching unit, the specific steps for outputting the feature matching confidence score are as follows: Establish a corresponding association between the clinical image feature vector set and the structured rule matching library, determine the feature parameter value, threshold range and confidence benchmark corresponding to each rule, and output the feature rule matching index table; The deviation between the feature parameter value and the threshold range is calculated based on the feature rule matching index table. It is then determined whether the parameter meets the threshold condition. If it does, the valid state is marked and the rule parameter lookup table is output. Otherwise, the matching fit of the rule is assigned to zero. The matching degree of effective rules is calculated based on the rule parameter lookup table. All rule matching degree values ​​are summarized, and a feature matching confidence score table is output.

7. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The iterative optimization module includes: The feedback acquisition unit is used to synchronize the feature matching confidence score and corresponding clinical images to the doctor's end, present the details of each feature matching and the preliminary diagnostic tendency, and form a structured correction record; The rule transformation unit is used to sort out the feature adjustment direction and threshold optimization range corresponding to the correction opinions based on the structured correction record, and transform the diagnostic adjustment content into new rules that can be added to the knowledge graph. The weight optimization unit is used to integrate the newly added rules into the original diagnostic reasoning rule system, adjust the matching weight of the corresponding features according to clinical credibility, and construct a temporary rule set that takes into account both the original logic and clinical correction experience. The closed-loop verification unit is used to re-match the features of historical clinical image samples according to the temporary rule set, compare and verify the diagnostic accuracy and confidence distribution of the optimized rules, and output the iterative diagnostic reasoning rule set.

8. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The federated aggregation module includes: The framework construction unit is used to build a federated rule sharing framework based on the core criteria in the rule set and output the federated rule sharing framework. The parameter extraction unit is used to guide participating institutions to adapt their applications based on local clinical image samples and iterative diagnostic reasoning rule sets according to the federated rule sharing framework, calculate the correction coefficients of the rules in the local scenario, and output the rule correction parameter set. The parameter aggregation unit is used to perform weighted aggregation calculations on homogeneous parameters using a federated average algorithm based on the modified parameter set according to the rules, assigning weights according to the sample size and data quality of each institution, and outputting the aggregation rule parameter set. The model generation unit is used to embed the aggregated rule parameter set into the original framework of the iterative diagnostic reasoning rule set, complete the global optimization of rule feature weights and judgment thresholds, and generate a diagnostic rule model.

9. The intelligent screening system for ear, nose, and throat diseases based on multimodal data according to claim 1, characterized in that: The tiered screening module includes: The process adaptation unit is used to extract the core diagnostic rules and feature judgment thresholds in the diagnostic rule model, transform them into simple screening logic that can be executed at the grassroots level, and output a set of basic rules for hierarchical screening. The initial screening execution unit is used to perform feature matching on the ear, nose and throat images or physical signs data of the examinee based on the general terminal at the grassroots level, perform initial screening diagnosis according to the basic rule set of the stratified screening, generate confidence scores for each case, and output a list of cases screened at the grassroots level. The case triage unit is used to determine the triage threshold based on the confidence score of the cases in the primary screening case list, mark high-confidence positive cases as key cases that need further diagnosis, organize basic case information and initial screening data, and output a case transfer list. The results integration unit is used to compare and correlate the initial screening data at the grassroots level with the confirmed diagnosis conclusions at the higher level based on the case transfer list, correct the confidence threshold of the basic rule set for stratified screening, and output the screening results for ear, nose and throat diseases.

10. A method for intelligent screening of ear, nose, and throat diseases based on multimodal data, comprising the intelligent screening system for ear, nose, and throat diseases based on multimodal data as described in any one of claims 1 to 9, characterized in that, The screening method includes: A parametric model of otolaryngology is constructed based on clinical images, and pathological features are defined in combination with pathological knowledge. Virtual images are generated through a regularization algorithm, and virtual and real features are aligned through adversarial domain adaptation to output a hybrid image dataset. Interpretable anatomical features are extracted from the hybrid image dataset, mapped to an ENT disease knowledge graph, diagnostic association rules are established through predicate logic reasoning, and a symbolic diagnostic reasoning model is output. A lightweight feature matching module is designed based on the symbolic diagnostic reasoning model to read the pixel features of clinical images and match them with atlas rules, and output the feature matching confidence score. The feature matching confidence score is fed back to the doctor. Through manual correction and rule iteration closed loop, the diagnosis adjustment is transformed into atlas rule supplementation, the feature matching weight is optimized, and the iterative diagnosis reasoning rule set is output. A federated rule sharing framework is constructed based on the iterative diagnostic reasoning rule set, rule correction parameters are uploaded, and the results are aggregated by the federated averaging algorithm to output a diagnostic rule model. Based on the diagnostic rule model, a tiered screening process is designed. Primary healthcare institutions can directly conduct initial screening for ear, nose, and throat diseases through the diagnostic rule model and push high-confidence cases to higher-level hospitals, forming a screening system that does not rely on equipment and outputting ear, nose, and throat disease screening results.

Images