Heat-onset suspicious intelligent diagnosis system and method based on hierarchical cascaded large language model

Abstract

The application discloses a fever-to-be-investigated intelligent diagnosis system and method based on a hierarchical cascade large language model, which comprises an information extraction module, a hierarchical diagnosis module and a knowledge pushing module. The hierarchical diagnosis module adopts a two-stage cascade architecture, a hierarchical long text encoding method is used by a coarse-grained classification unit to preliminarily classify infectious and non-infectious fever, a parameter-efficient fine-tuning large language model is used by a fine-grained diagnosis unit to perform fine-grained etiological differential diagnosis, and a cascade decision fusion unit fuses the coarse and fine-grained results according to Bayes theorem to output the final diagnosis suggestion. The knowledge pushing module provides evidence-based medical support based on retrieval enhancement generation technology. The application significantly improves the accuracy and efficiency of fever-to-be-investigated diagnosis, effectively solves the "black box" problem of the large language model, enhances the diagnosis explainability, balances real-time response and diagnosis accuracy, and is suitable for various clinical scenes such as outpatient first diagnosis, inpatient evaluation and multi-center consultation.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and clinical medical diagnosis, specifically to an intelligent diagnostic system and method for fever based on a hierarchical cascaded large language model. Background Technology

[0002] Fever of Unknown Origin (FUO) refers to a febrile illness with a body temperature exceeding 38.3°C for more than 3 weeks, where the cause remains unclear despite one week of inpatient examination or outpatient follow-up. Since its initial definition by Petersdorf and Beeson in 1961, FUO has consistently been one of the most challenging diagnostic problems in clinical practice. Its causes encompass over 200 diseases, including infectious diseases, non-infectious inflammatory diseases, and tumors. Globally, infectious diseases remain the leading cause, but the proportion of non-infectious inflammatory diseases is showing a significant upward trend. Meanwhile, approximately 5%–23% of patients remain undiagnosed even after comprehensive examination. The etiology of FUO exhibits significant spatiotemporal heterogeneity, with infectious diseases accounting for a significantly higher proportion in Southeast Asia than in Europe, while non-infectious inflammatory diseases have the highest proportion in the Western Pacific region. The complexity of FUO leads to a time-consuming and costly diagnostic process, heavily reliant on the physician's personal experience. In clinical practice, most FUO patients receive empirical antibiotic treatment upon initial diagnosis, which may not only delay the diagnosis of non-infectious diseases but also exacerbate antibiotic resistance. The aforementioned challenges urgently require the development of intelligent auxiliary tools that can accurately distinguish between infectious and non-infectious causes of disease at an early stage.

[0003] In recent years, Large Language Models (LLMs), represented by ChatGPT, Gemini, DeepSeek, and Doubao, have shown broad application prospects in the medical field, covering tasks such as clinical decision support, patient engagement, and medical literature analysis. Multiple studies have compared and evaluated the performance of mainstream LLMs in medical scenarios both domestically and internationally. In full-process clinical workflow simulation, Doubao's performance in differential diagnosis, examination recommendations, and treatment management is comparable to ChatGPT-4, and LLMs outperform emergency department resident physicians in diagnostic examinations and treatment management. In complex case clinical decision-making, DeepSeek-R1 and GPT-4o showed no significant difference in decision accuracy, both significantly outperforming the Kimi, Gemini, and LLaMa3 series models.

[0004] However, the "illusion" phenomenon of LLMs can generate medical content that contradicts reality, and its incidence and clinical severity have been confirmed in multiple studies. Furthermore, their reasoning lacks interpretability, and their "black box" nature makes it difficult for clinicians to trace and verify diagnostic logic. Despite a significantly expanded context window, LLMs still perform poorly in temporal reasoning from longitudinal electronic health records and in rare disease prediction. Simultaneously, LLMs generally exhibit metacognitive deficiencies, tending to provide high-confidence outputs even with insufficient evidence, and may inherit and amplify systematic biases from training data. In dynamic clinical scenarios, the diagnostic performance of LLMs is significantly lower than in static question-answering environments, with a diagnostic accuracy of only about 43.3% for rare diseases. These limitations collectively restrict the safe clinical application of LLMs. Summary of the Invention

[0005] The main purpose of this application is to provide an intelligent diagnostic system and method for fever diagnosis based on a hierarchical cascaded large language model, aiming to solve the problems of metacognitive defects in LLMs and low diagnostic accuracy in dynamic clinical scenarios.

[0006] To achieve the above objectives, this application provides an intelligent diagnostic system for fever diagnosis based on a hierarchical cascaded large language model, comprising: The information extraction module is used to extract key medical information from doctor-patient dialogues or electronic medical records in a structured manner. The key medical information includes the patient's basic information, fever time, body temperature curve, accompanying symptoms, history of chronic diseases, test results and imaging examination summary. A hierarchical diagnostic module, based on the aforementioned key medical information, employs a two-stage cascaded architecture to achieve progressive diagnostic reasoning from coarse-grained to fine-grained. The hierarchical diagnostic module includes: The coarse-grained classification unit uses a hierarchical bidirectional encoder (Hierarchical BERT) binary classification model based on hierarchical long text encoding to perform preliminary classification of structured medical record text into infectious and non-infectious fever, and outputs classification results and confidence scores. The fine-grained diagnostic unit employs the Qwen2.5-VL-7B-Instruct model with highly efficient parameter fine-tuning. Based on the classification results, it performs fine-grained etiological differential diagnosis and generates a candidate diagnosis list sorted in descending order of confidence. Qwen2.5-VL-7B-Instruct is the large-scale model with fine-tuning of the Tongyi Qianwen 2.5 visual language 7B instructions (Qwen = Tongyi Qianwen, VL = Vision-Language, 7B = 7 billion parameters, Instruct = instruction fine-tuning). The cascaded decision fusion unit fuses the classification results with the candidate diagnosis list according to Bayes' theorem, calculates the posterior probability of each candidate diagnosis, and outputs the final diagnosis suggestion. The knowledge push module, based on search-enhanced generation technology, provides evidence-based medicine support for the final diagnostic recommendations, including relevant medical literature retrieval, diagnostic basis generation, and treatment recommendation push.

[0007] Optionally, the coarse-grained classification unit uses a sliding window overlapping segmentation mechanism to process long text medical records, performs block-level semantic encoding through Chinese-RoBERTa-wwm-ext, and uses a multi-layer Transformer encoder to fuse inter-block information. Finally, it generates an adaptive document representation through an attention pooling mechanism.

[0008] Optionally, the fine-grained diagnostic unit uses a low-rank adaptation method to perform task-specific fine-tuning on the Qwen2.5-VL-7B-Instruct model, and achieves diagnostic reasoning ability for fever-related diseases by conditional generation modeling and supervised fine-tuning objectives.

[0009] Optionally, the intelligent diagnostic system for fever to be investigated adopts an edge-cloud collaborative deployment architecture: The edge terminal is used to deploy the Hierarchical BERT binary classification model to achieve real-time coarse screening; The cloud platform uses the speech recognition API to call the finely tuned Qwen2.5-VL-7B-Instruct model to process fine-grained diagnostic requests.

[0010] Optionally, the information extraction module includes: The speech-to-text conversion channel uses a speech recognition API to convert doctor-patient dialogues into text in real time, and extracts structured information through an information extraction unit based on a basic model. The text parsing channel directly extracts admission records, progress notes, laboratory test results, imaging results, and discharge summaries from the electronic medical record system. The extracted medical information is structured according to a preset JSON Schema, including fields such as patient basic information, fever time, body temperature curve, accompanying symptoms, past medical history, and test results.

[0011] Optionally, the knowledge push module includes: The vector index building unit divides medical text into blocks and builds vector indexes through an embedding model; The dynamic retrieval unit dynamically retrieves Rank-K related text fragments as context based on the current medical record characteristics; The knowledge-enhancing generation unit uses relevant text fragments retrieved as context to guide the large model in generating traceable medical recommendations; The automated medical record generation unit automatically generates outpatient medical records based on integrated structured information, including chief complaint, present illness, past medical history, physical examination, and treatment recommendations.

[0012] This application also provides an intelligent diagnostic method for fever based on a hierarchical cascaded large language model, including the following steps: Step S1: Obtain electronic medical record data of patients with fever of unknown origin, de-identify the data, and have the expert group determine the final diagnosis as the gold standard label; Step S2: Extract key medical information from doctor-patient dialogues or electronic medical records in a structured manner. The key medical information includes the patient's basic information, fever time, body temperature curve, accompanying symptoms, history of chronic diseases, laboratory test results, and imaging examination summary. Step S3: Use the Hierarchical BERT binary classification model based on hierarchical long text encoding to perform preliminary classification of infectious and non-infectious fever in structured medical record texts, and output the classification results and confidence scores. Step S4: Perform temperature scaling calibration and Platt scaling calibration on the confidence scores; Step S5: Using the Qwen2.5-VL-7B-Instruct model with efficient parameter fine-tuning, perform fine-grained etiological differential diagnosis based on the classification results, and generate a candidate diagnosis list sorted in descending order of confidence. Step S6: Based on Bayes' theorem, the classification results are cascaded and fused with the candidate diagnosis list to calculate the posterior probability of each candidate diagnosis and output the final diagnosis suggestion; Step S7: Based on search enhancement generation technology, provide evidence-based medical support for the final diagnostic recommendations, including relevant medical literature retrieval, diagnostic basis generation, and treatment recommendation delivery.

[0013] Optionally, in step S3, the hierarchical long text encoding method includes: The input medical record document is divided into multiple text blocks using a sliding window overlapping segmentation mechanism. Each text block has a length of 510 tokens, an overlap of 128 tokens, and a step size of 382. The Chinese-RoBERTa-wwm-ext pre-trained model is used to perform semantic encoding on each text block, and the output of the last layer [CLS] position is taken as the block-level semantic representation. Inter-block information fusion is performed on the block-level representation matrix using a 4-layer Transformer encoder; An attention pooling mechanism is used to generate an adaptive document representation; The classification layer outputs the binary probability distribution and confidence score of infectious and non-infectious fever.

[0014] Optionally, in step S5, the efficient parameter fine-tuning method includes: A low-rank adaptation method is used to fine-tune the pre-trained large language model. For the pre-trained weight matrix, the parameters are updated by low-rank decomposition.

[0015] Optionally, in step S6, the cascade fusion method includes: Calculate the posterior probability of the candidate diagnosis based on Bayes' theorem; Temperature scaling and Platt scaling calibrations are applied to the confidence scores of coarse-grained classification. The diagnosis with the highest posterior probability is taken as the final output.

[0016] Compared with the prior art, this application has the following beneficial effects: (1) This application realizes collaborative reasoning between a lightweight BERT classification model and a fine-tuned large language model through a two-stage cascaded architecture of "coarse screening-fine diagnosis". The coarse-grained classification unit adopts a hierarchical long text encoding method, which can effectively handle long electronic medical record texts that exceed the length limit of traditional pre-trained models. Through sliding window overlapping segmentation, block-level semantic encoding and inter-block information fusion, it realizes rapid and accurate classification of infectious and non-infectious fever. The fine-grained diagnosis unit adopts a large language model with efficient parameter fine-tuning, and performs fine-grained etiological differential diagnosis based on the coarse-grained classification results, which greatly improves the diagnostic accuracy. Experiments show that the system's first-level diagnostic accuracy reaches more than 90%, and the second-level diagnostic accuracy reaches more than 85%, which is significantly better than the single model architecture.

[0017] (2) Effectively solves the "black box" problem of large language models and enhances diagnostic interpretability: This application provides traceable evidence-based medical support for each diagnostic suggestion through retrieval enhancement generation technology and knowledge push module. The system dynamically retrieves relevant medical literature and clinical guidelines based on vector index, and uses the retrieved relevant text fragments as context to guide the large model to generate detailed explanations containing diagnostic basis and treatment suggestions. This mechanism not only improves the credibility of diagnosis, but also enables clinicians to understand and verify the diagnostic logic of AI, significantly increasing the willingness to adopt it clinically. At the same time, the cascaded decision fusion unit uses Bayes' theorem to calculate the posterior probability, making the diagnostic process mathematically interpretable.

[0018] (3) Balancing real-time response and diagnostic accuracy to adapt to diverse clinical scenarios: This application adopts an edge-cloud collaborative deployment architecture. A lightweight Hierarchical BERT classifier is deployed at the edge to achieve real-time coarse screening, with a response time controlled within 1 second. In the cloud, a finely tuned Qwen2.5-VL-7B-Instruct model is called via API to process fine-grained diagnostic requests, with end-to-end inference latency controlled within 3-5 seconds, meeting the real-time requirements of outpatient services. This architecture ensures both the real-time performance of coarse screening and the accuracy of fine-grained diagnosis, making it suitable for diverse clinical scenarios such as outpatient initial visits, inpatient assessments, and multi-center consultations. The efficient parameter fine-tuning technology reduces the proportion of trainable parameters to only 0.016%, significantly lowering the model update cost and facilitating clinical deployment and maintenance. Attached Figure Description

[0019] Figure 1 This is a block diagram of the intelligent diagnostic system for fever based on a hierarchical cascaded large language model in the embodiments of this application; Figure 2 This is a flowchart of the intelligent diagnostic method for fever based on a hierarchical cascaded large language model in the embodiments of this application; Figure 3 This is an architecture diagram of the intelligent diagnostic system for fever based on a hierarchical cascaded large language model in this application embodiment. Figure 4 A flowchart for using the Qwen2.5-VL-7B-Instruct model generated in the embodiments of this application; Figure 5 This is an AI diagnostic result diagram of the fever diagnosis system based on a hierarchical cascaded large language model used in Embodiment 10 of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] The terms used in this application are explained as follows: Electronic Medical Record (EMR): An electronic medical record is a text data source that digitally stores all medical information, including patient admission records, test results, and medical records, and serves as the original input text for the model in this embodiment.

[0022] Hierarchical Bidirectional Encoder Representations from Transformers (Hierarchical BERT): A hierarchical pre-trained language model that first encodes text sub-words and then aggregates them into blocks to model long medical records, used to complete the binary classification task of fever etiology infection / non-infection.

[0023] Tokenizer: The core module of text preprocessing, which segments continuous electronic medical record text into sub-word units (tokens) that the model can recognize, and simultaneously performs text length truncation and padding.

[0024] Classification Token (CLS): A dedicated vector token placed at the beginning of each text block, used to aggregate the global semantic features of the corresponding text block, serving as the basic input for block-level feature fusion.

[0025] Softmax activation function (Soft Maximum): Converts the original scores of the network output into a probability distribution. In this embodiment, it is used to output the binary probability of infectious fever and non-infectious fever.

[0026] Qwen2.5-VL-7B (Qwen2.5 Vision-Language-7B): A large-scale model with 7 billion parameters based on the Tongyi Qianwen 2.5 visual language, a multimodal large-scale model with a decoder architecture developed by Alibaba; this solution only uses its text capabilities to complete multi-label clinical diagnostic reasoning.

[0027] Retrieval-Augmented Generation (RAG): Integrates knowledge base retrieval results into the input of a large model, reducing model illusion and improving the reliability of diagnostic criteria.

[0028] Low-Rank Adaptation (LoRA): A lightweight fine-tuning scheme that freezes the backbone weights of a large model and trains only a small number of low-rank matrix parameters to complete fine-tuning for clinical scenarios with limited computing power.

[0029] Supervised Fine-Tuning (SFT): Supervised training of a large model using labeled clinical medical record-disease label datasets to align the model output with clinical diagnostic paradigms.

[0030] Root Mean Square Layer Normalization (RMSNorm): A neural network normalization method that stabilizes the feature distribution during training and is more computationally efficient than traditional LayerNorm.

[0031] Fever of Unknown Origin (FUO): Oral temperature ≥38.3℃, fever lasting ≥3 weeks, and fever whose cause cannot be determined after 1 week of inpatient / outpatient systematic examination; the causes cover more than 200 diseases such as infection, autoimmune disease, and tumors, making clinical diagnosis extremely difficult.

[0032] ID (Infectious Disease): Fever caused by pathogens such as bacteria, viruses, fungi, and parasites.

[0033] NID (Non-Infectious Disease): Diseases without pathogen involvement, including autoimmune inflammation, tumors, drug fever, etc.

[0034] NIID (Non-Infectious Inflammatory Disease): The core subclass of non-infectious fever, such as adult Still's disease, systemic lupus erythematosus, vasculitis and other autoimmune diseases.

[0035] AR (Admission Record): The patient's initial medical record, simulating a rapid screening scenario in an outpatient setting.

[0036] INV (Investigation, Laboratory Tests): Results of auxiliary examinations such as complete blood count, inflammatory markers, CT / ultrasound, etc., with two input modes: initial examination and full examination.

[0037] WBC (white blood cells) and N (neutrophils): These are indicators that are significantly elevated during bacterial infection.

[0038] Hb (hemoglobin) and PLT (platelet-rich plasma): used to screen for hematologic malignancies and chronic inflammation.

[0039] CRP (C-reactive protein) and ESR (erythrocyte sedimentation rate): universal markers of chronic inflammation.

[0040] PCT procalcitonin: a specific indicator that distinguishes between bacterial infection and autoimmune inflammation.

[0041] ANA antinuclear antibody, RF rheumatoid factor, and CCP cyclic citrullinated peptide antibody: core indicators for screening autoimmune diseases.

[0042] LLMs (Large Language Models): Generative AI based on Transformer and pre-trained on massive amounts of medical text, capable of understanding medical records and outputting differential diagnoses; the application base is Qwen2.5-VL-7B.

[0043] BERT (Bidirectional Encoder Representations from Transformers): A bidirectional pre-trained classification model that synchronously reads contextual semantics; its derivative version, RoBERTa, is the basis for coarse-grained classification.

[0044] RoBERTa (Robustly Optimized BERT Pretraining Approach): BERT with optimized pretraining strategy, combined with WWM (Whole Word Masking), has a stronger effect on Chinese medical record semantic recognition; WWM refers to completely masking Chinese words during pretraining, rather than individual characters.

[0045] FFN: Feed-Forward Network.

[0046] Bayes' theorem: This application uses the mathematical foundation of cascaded decision fusion, combining coarse-grained binary classification confidence and fine-grained diagnostic probability to calculate the final posterior probability of the disease.

[0047] Temperature scaling and Platt scaling: two types of confidence calibration algorithms that correct overly confident or overly conservative probability outputs from the model, thereby improving the reliability of diagnostics.

[0048] Fused Confidence Score: A final comprehensive score calculated by combining the coarse screening results of Hierarchical BERT with the fine-grained diagnostic probabilities of the large model, used to output a unique recommended diagnosis.

[0049] Chinese-RoBERTa-wwm-ext: This is an open-source pre-trained language model optimized for Chinese scenarios. Its basic architecture is derived from RoBERTa (a robust and optimized version of BERT). wwm represents the Whole Word Masking pre-training strategy, and ext represents the extended training corpus version.

[0050] JSON is an abbreviation for JavaScript Object Notation, a lightweight data-interchange format used to transfer structured data between programs.

[0051] CRF stands for Conditional Random Field Layer, a key component in natural language processing for sequence labeling tasks, often used in conjunction with deep learning models such as LSTM and BERT.

[0052] A JSON Schema is a JSON document that describes the structural rules of another JSON document and can be used to validate whether data conforms to those rules.

[0053] AdamW optimizer, short for Adam with Weight Decay, is one of the most mainstream and recommended optimizers in the field of deep learning.

[0054] The Transformer Encoder is the "encoder" part of the Transformer model, responsible for converting input sequences (such as sentences) into semantically rich vector representations. It is a core component of NLP tasks (such as text classification, named entity recognition, and machine translation).

[0055] HTML tags are the basic building blocks of an HTML document. Keywords enclosed in angle brackets define the structure and content of a webpage and tell the browser how to display the page.

[0056] Dropout regularization is one of the most commonly used techniques to prevent overfitting in deep learning. It was proposed by Geoffrey Hinton's team in 2012. The core idea is to randomly drop some neurons during training, forcing the network to learn more robust feature representations.

[0057] BGE is an abbreviation for BAAI General Embedding, an open-source general text vector model series launched by the Beijing Academy of Artificial Intelligence (BAAI). It is specifically designed to convert text into high-dimensional vector representations (Embedding) and performs well in tasks such as semantic retrieval and RAG.

[0058] text-embedding-3 is the latest generation of text embedding models released by OpenAI in 2024. It includes two versions, text-embedding-3-small and text-embedding-3-large, which are used to convert text into high-dimensional vector representations. It is a core component for building applications such as semantic search, RAG, and text similarity.

[0059] Qwen is a family of large language models launched by Alibaba Group. It has powerful capabilities in language understanding, logical reasoning, code generation, and multilingual support, and can be applied to various scenarios such as dialogue, writing, programming, and data analysis.

[0060] Adult-onset Still's disease (AOSD) is a rare systemic inflammatory disease of unknown cause, but it may be related to genetics, immune abnormalities, and infection triggers.

[0061] Castleman disease (CD) is a rare lymphoproliferative disorder, also known as angiofollicular lymphadenopathy or giant lymphadenopathy. It was included in the first batch of rare disease catalogs by the National Health Commission in 2018.

[0062] Example 1 like Figure 1 , Figure 3 As shown, this embodiment provides a fever diagnosis system based on a hierarchical cascaded large language model, which includes three core components: an information extraction module 1, a hierarchical diagnosis module 2, and a knowledge push module 3.

[0063] Information extraction module 1 is responsible for extracting key medical information in a structured manner from doctor-patient dialogues or electronic medical records, providing standardized input for subsequent diagnosis. This module employs a dual-channel information extraction strategy: Voice-to-text conversion channel: For real-time outpatient consultation scenarios, the iFlytek speech recognition API (Application Programming Interface) is used to convert doctor-patient dialogue into text in real time. Then, the information extraction unit based on the basic model extracts structured information such as the patient's chief complaint, duration of fever, accompanying symptoms, and past medical history.

[0064] Let the input dialogue text be X = {x1, ..., x2}. M The target structured fields are F = {f1, ..., f}. F}, where x1 is the first input dialogue text, and there are a total of M input dialogue texts, x M Let f1, ..., f be the Mth dialogue text. F This represents the first to the Fth target structured fields.

[0065] Sequence labeling: P(t|X) = ∏ m P(t m |t m-1 (X) CRF layer: (1) The symbols and their meanings in the above formulas are shown in Table 1: Table 1

[0066] Text parsing channel: For inpatient medical record scenarios, it directly extracts admission records, progress notes, laboratory test results, imaging results, discharge summaries, etc. from the hospital's electronic medical record system (EMR).

[0067] The extracted medical information, after being structured, is input into the hierarchical diagnosis module in JSON format, including the following fields: patient basic information, fever duration, body temperature curve, accompanying symptoms, history of chronic diseases, current test results (blood routine, inflammatory markers, etc.), and imaging examination summary.

[0068] JSON structured output probability: (2) The symbols and their meanings in equation (2) are shown in Table 2: Table 2

[0069] The hierarchical diagnosis module 2 is the core of this system. It adopts a two-stage cascaded architecture to realize progressive diagnostic reasoning from coarse-grained to fine-grained, including a coarse-grained classification unit 201, a fine-grained diagnosis unit 202, and a cascaded decision fusion unit 203.

[0070] To address the issue that long texts in electronic medical records exceed the length limits of traditional pre-trained models, this application designs a hierarchical long text encoding method, Hierarchical BERT, for binary classification of infectious and non-infectious fever.

[0071] The workflow of coarse-grained classification unit 201 is as follows: (1) Long text segmentation strategy Given an input document D = {w_1, w_2, ..., w_...} N}, where N is the total number of words in the document, w_ N This is the Nth input document. A sliding window overlapping segmentation mechanism is used: c_i = {w_j | j ∈ [i·S, i·S+L]},i=0,1,...,K-1 The symbols and their meanings in the above formulas are shown in Table 3. Table 3

[0072] Total number of blocks K = (NO) / S The upper limit K_max = 32.

[0073] Filling strategy: _i = { c_i ⊕ {[PAD]}_{j=1}^{L-|c_i|}if |c_i| <L;c_iotherwise}(3) The symbols and their meanings in equation (3) are shown in Table 4: Table 4

[0074] (2) Block-level semantic coding Each text block _i encoded using Chinese-RoBERTa-wwm-ext: (4) The symbols and their meanings in equation (4) are shown in Table 5: Table 5

[0075] H_i^(l) = TransformerLayer^(l)(H_i^(l-1)),l=1,...,12(5) The symbols and their meanings in equation (5) are shown in Table 6: Table 6

[0076] Full word mask probability:

[0077] (3) Inter-block dependency modeling Block-level representation matrix H = [h_1; h_2; ...; h_ K ] ∈ R^{K×768}(6) The symbols and their meanings in equation (6) are shown in Table 7: Table 7

[0078] Learnable positional encoding: (7) The symbols and their meanings in equation (7) are shown in Table 8: Table 8

[0079] Global [CLS] token: (8) The symbols and their meanings in equation (8) are shown in Table 9: Table 9

[0080] To capture long-distance semantic dependencies across blocks, a 4-layer Transformer Encoder is introduced for inter-block information fusion: (9) The symbols and their meanings in equation (9) are shown in Table 10: Table 10

[0081] (10) The symbols and their meanings in equation (10) are shown in Table 11: Table 11

[0082] (11) The symbols and their meanings in equation (11) are shown in Table 12: Table 12

[0083] H'^(l) = LayerNorm(FFN(H'^(l)_att) + H'^(l)_att)(12) The symbols and their meanings in equation (12) are shown in Table 13: Table 13

[0084] d k = 768 / 12 = 64, FFN is a feedforward network: FFN(x) = GELU(xW1+ b1)W2+ b2(13) The symbols and their meanings in equation (13) are shown in Table 14: Table 14

[0085] (4) Adaptive document representation Instead of simple average pooling, attention pooling is used, allowing the model to automatically learn the weights for diagnosing key paragraphs. (14) The symbols and their meanings in equation (14) are shown in Table 15: Table 15

[0086] (15) The symbols and their meanings in equation (15) are shown in Table 16: Table 16

[0087] The document states: d = Σ_{i=1}^{K} α_i · h'_i (16) The symbols and their meanings in equation (16) are shown in Table 17: Table 17

[0088] Regularization: = Dropout(d, p=0.3) The symbols and their meanings in the above formula are shown in Table 18: Table 18

[0089] (5) Classification layer and loss function Classification probability: (17)

[0090] The symbols and their meanings in equation (17) are shown in Table 19: Table 19

[0091] P(y|d) = softmax(z) = exp(z_y) / Σ_{j=1}^{2} exp(z_j)(18)

[0092] The symbols and their meanings in equation (18) are shown in Table 20: Table 20

[0093] Cross-entropy loss: (19) The symbols and their meanings in equation (19) are shown in Table 21: Table 21

[0094] Based on coarse-grained classification, the Qwen2.5-VL-7B-Instruct model with efficient parameter fine-tuning is used for fine-grained diagnostic inference.

[0095] The fine-grained diagnostic unit 202 employs LoRA low-rank adaptation technology, as detailed below: (1) LoRA low-rank adaptation To adapt to the task of diagnosing febrile diseases, the pre-trained weight matrix W... o ∈ R d×k The standard has been slightly adjusted and updated as follows: W = W o +ΔW (20) The symbols and their meanings in equation (20) are shown in Table 22: Table 22

[0096] LoRA constrains ΔW to a low-rank decomposition: ΔW = BA (21) The symbols and their meanings in equation (21) are shown in Table 23: Table 23

[0097] Forward propagation: h = W0x + ΔWx = W0x + BAx (22) The symbols and their meanings in equation (22) are shown in Table 24: Table 24

[0098] Scaling factor: (twenty three) The symbols and their meanings in equation (23) are shown in Table 25: Table 25

[0099] Percentage of trainable parameters: Params_LoRA = Σ_{W∈T} 2×d_W×r (24) The symbols and their meanings in equation (24) are shown in Table 26: Table 26

[0100] Ratio = Params_LoRA / Params_total ≈ (2×2×3584×8) / (7×10 9) ≈0.016% (25) The symbols and their meanings in equation (25) are shown in Table 27: Table 27

[0101] (2) Condition generation modeling Input construction: x = [Prompt] ⊕ [Coarse-grained classification results] ⊕ [Medical record text] x = “Based on the diagnosis of infectious fever, the patient's medical record is as follows:”⊕c infectious ⊕D Autoregressive generation: (26) The symbols and their meanings in equation (26) are shown in Table 28: Table 28

[0102] P(y_t | y_{ <t}, x) = softmax(W_lm h_t^(28))(27) The symbols and their meanings in equation (27) are shown in Table 29: Table 29

[0103] (3) Monitor and fine-tune the objectives Negative log-likelihood loss: (28) The symbols and their meanings in equation (28) are shown in Table 30: Table 30

[0104] Rank-5 diagnostic ranking loss: Let the gold standard diagnostic set be The model outputs Top-5 as .

[0105] List likelihood loss: L_List = -Σ_{i=1}^{5} [ I( _i∈Y )logσ(s_i) + I( _i Y )log(1-σ(s_i))](29) The symbols and their meanings in equation (29) are shown in Table 31: Table 31

[0106] Joint training objectives: (30) The symbols and their meanings in equation (30) are shown in Table 32: Table 32

[0107] The workflow of the cascaded decision fusion unit 203 is as follows: (1) Probability decomposition According to Bayes' theorem and the chain rule: P(D_i|medical record) = Σ_{j∈{infected, non-infected}} P(D_i,C_j|medical record) = Σ_j P(D_i|C_j,medical record)·P(C_j|medical record) (31) The symbols and their meanings in equation (31) are shown in Table 33: Table 33

[0108] Because coarse-grained classification unit 201 provides the classification results Simplified to: P(D_i|medical record) ≈ P(D_i| (Medical records)·P( |Medical records) (32) The symbols and their meanings in equation (32) are shown in Table 34: Table 34

[0109] The logarithmic form is: log P(D_i|medical record) = log P(D_i| (, medical records) + log P( |Medical records) (33) The symbols and their meanings in equation (33) are shown in Table 35: Table 35

[0110] (2) Confidence calibration Temperature scaling calibration: P_calibrated(y|d) = softmax(z / T)(34) The symbols and their meanings in equation (34) are shown in Table 36: Table 36

[0111] The temperature parameter T is optimized on the validation set: T = arg min_T NLL(P_calibrated, Y_val)(35) The symbols and their meanings in equation (35) are shown in Table 37: Table 37

[0112] Platt scaling: (36) The symbols and their meanings in equation (36) are shown in Table 38: Table 38

[0113] Knowledge delivery module 3, based on search-enhanced generation technology, provides evidence-based medicine support for diagnostic results. This module includes: The vector index building unit 301 is used to divide medical texts such as electronic medical records and clinical guidelines into blocks and build vector indexes through an embedding model.

[0114] The dynamic retrieval unit 302 is used to dynamically retrieve Rank-K related text fragments as context based on the current medical record characteristics when generating diagnostic criteria.

[0115] The knowledge enhancement generation unit 303 uses the retrieved relevant text fragments as context to guide the large model in generating traceable diagnosis and treatment suggestions.

[0116] The automated medical record generation unit 304 automatically generates outpatient medical records after the doctor completes the consultation. The records are generated by integrating the structured information extracted by the aforementioned modules and using the same RAG+LLMs technology stack. The records include the chief complaint, present illness, past medical history, physical examination, and treatment recommendations.

[0117] This system adopts a collaborative deployment architecture between the edge and the cloud: Edge computing: Deploy the Hierarchical BERT binary classification model to achieve real-time coarse screening; In the cloud: Fine-tuned Qwen2.5-VL-7B-Instruct models are called via the speech recognition API to process fine-grained diagnostic requests.

[0118] This architecture balances response speed and diagnostic accuracy, with an end-to-end inference latency of approximately 3-5 seconds, meeting the real-time requirements of clinical practice.

[0119] Example 2 like Figure 2 As shown, this embodiment provides an intelligent diagnostic method for fever based on a hierarchical cascaded large language model, including the following steps: Step S1: Data Acquisition and Processing S1.1 Obtain electronic medical record data for patients with fever of unknown origin, including admission records, progress notes, laboratory test results, imaging test results, and medical orders; S1.2 De-identify the acquired electronic medical record data; S1.3 The final diagnosis is determined by an expert panel based on complete clinical data and serves as the gold standard label; S1.4 Cases were divided into infectious disease group and non-infectious disease group according to etiology. The infectious disease group was further divided into bacterial infection, fungal infection, viral infection, parasitic infection and other pathogen infection subgroups. The non-infectious disease group was further divided into non-infectious inflammatory disease, tumor disease and other non-infectious factor subgroups.

[0120] Step S2: Information Structure Extraction S2.1 For real-time outpatient consultation scenarios, the doctor-patient dialogue is converted into text through the speech recognition API, and then the structured information is extracted through the information extraction unit; S2.2 For inpatient medical record scenarios, directly extract text information such as admission records, progress notes, laboratory test results, imaging results, and discharge summaries from the electronic medical record system; S2.3 The extracted medical information is organized in a structured manner according to the preset JSON Schema.

[0121] Step S3: Coarse-grained etiological classification S3.1 Input the structured medical record text obtained in step S2.3 into the Hierarchical BERT binary classification model; S3.2 A sliding window overlapping segmentation mechanism is used to segment long documents into multiple text blocks; S3.3 Semantically encode each text block using Chinese-RoBERTa-wwm-ext, and take the [CLS] position as the block-level semantic representation; S3.4 performs inter-block information fusion through a 4-layer Transformer Encoder; S3.5 uses an attention pooling mechanism to generate adaptive document representations; S3.6 Output the binary classification results and confidence scores for infectious and non-infectious fever through the classification layer.

[0122] Step S4: Confidence calibration S4.1 Optimize the temperature parameter T on the validation set and perform temperature scaling calibration on the confidence score output in step S3.6; S4.2 Apply Platt scaling to the calibrated confidence scores to obtain the calibrated classification probabilities.

[0123] Step S5: Fine-grained diagnostic generation S5.1 If step S3.6 determines that the fever is infectious, concatenate the medical record text with the prompt "based on infectious fever determination", and input the fine-tuned Qwen2.5-VL-7B-Instruct model; S5.2 If step S3.6 determines the fever to be non-infectious, concatenate the medical record text with the prompt "based on non-infectious fever determination," and input it into the fine-tuned Qwen2.5-VL-7B-Instruct model, as follows: Figure 4 As shown; The S5.3 model generates a Top-5 differential diagnosis list sorted by confidence level in descending order through autoregression. S5.4 Perform list likelihood calibration on the Top-5 diagnostic results.

[0124] Step S6: Cascaded Decision Fusion S6.1 According to Bayes' theorem, calculate the posterior probability of each candidate diagnosis: P(D i |Medical records) = P(D i |C, medical record)·P(C|medical record); S6.2 Take the diagnosis with the highest posterior probability as the final output.

[0125] Step S7: Knowledge Push and Medical Record Generation S7.1 Use the final diagnosis result output in step S6.2 as a query to retrieve relevant text fragments from the pre-built medical knowledge vector base; S7.2 Using the retrieved relevant text fragments as context, guide the large model to generate traceable diagnostic evidence and treatment recommendations; S7.3 Automatically generates structured outpatient medical records.

[0126] Step S8: Model Training and Update S8.1 uses the internal training set to perform supervised training on the Hierarchical BERT binary classification model, with the loss function being cross-entropy loss plus L2 regularization; S8.2 LoRA fine-tuning of Qwen2.5-VL-7B-Instruct is performed using a weighted joint objective of negative log-likelihood loss, Rank-5 diagnostic ranking loss, and ranking loss; S8.3 Regularly include newly confirmed cases to incrementally update the model.

[0127] Example 3 This embodiment constructs a FUO disease database using a retrospective, multicenter study approach. The specific steps are as follows: Step 1: Case screening: Cases meeting the following inclusion criteria were screened from the electronic medical record systems of participating hospitals: (1) Meeting the classic diagnostic criteria for fever of unknown origin: body temperature ≥38.3℃ (oral temperature), and fever lasting ≥3 weeks; (2) After ≥1 week of systematic examinations (including but not limited to blood routine, blood culture, and imaging examinations), the cause of the disease remains unclear; (3) Age ≥18 years; (4) Complete clinical data, including key information such as chief complaint, present illness, physical examination, laboratory tests, and imaging reports. At the same time, the following were excluded: (1) febrile diseases that were clearly diagnosed before hospitalization; (2) neutropenia (absolute neutrophil count <0.5×10 9 (2) (L) or known immunodeficiency status (such as HIV infection, post-organ transplantation, long-term use of immunosuppressants); (3) pregnant or lactating women; (4) cases with severely missing clinical data or unclear final diagnosis.

[0128] Step 2 Data Extraction: Trained researchers use standardized case report forms to extract all patient case information from the electronic medical record system during the hospitalization period, including: basic information (gender, age, admission date, etc.), written records (admission record, progress notes, consultation records, discharge summary, etc.), laboratory test results, imaging test results, and medical orders.

[0129] Step 3 Data Cleaning: The extracted data is cleaned, including: removing duplicate records, handling missing values ​​(variables with a missing rate >50% are removed), standardizing date formats, standardizing laboratory indicator units, and removing sensitive information (name, ID number, contact information, etc.). After de-identification processing, the data is imported into the FUO disease database.

[0130] Step 4: Gold Standard Diagnostic Criteria: A panel of two infectious disease specialists with over 10 years of clinical experience determines the final diagnosis for each case based on comprehensive clinical, laboratory, imaging, and microbiological data, combined with treatment response. Disagreements are resolved through discussion to reach a consensus. Cases are categorized by etiology as follows: a. Primary classification (secondary classification): Infectious diseases (ID), Non-infectious diseases (NID); b. Secondary classification (infection subclasses): Bacterial infections, fungal infections, viral infections, parasitic infections, and infections caused by other pathogens; c. Secondary classification (non-infectious subclass): Non-infectious inflammatory diseases (NIID), neoplastic diseases, and other non-infectious factors.

[0131] The internal dataset was randomly divided into a training set (for model training), a validation set (for hyperparameter tuning and confidence calibration), and an internal test set (for model selection and preliminary evaluation) in a ratio of 7:1.5:1.5. The external validation set was independent of the training process and was used only for verifying the generalization ability of the final model.

[0132] Example 4 This embodiment provides a training and implementation method for the Hierarchical BERT binary classification model, specifically including: 1. Model architecture parameter settings The core parameter configurations of the Hierarchical BERT binary classification model are shown in Table 39: Table 39

[0133] 2. Training Steps Step 2.1 Text Preprocessing: The electronic medical record text is cleaned, including removing HTML tags, redundant whitespace characters, special symbols, and standardizing number and unit formats. The cleaned document is then segmented into text blocks using a sliding window strategy: window length L = 510 tokens, step size S = 382 tokens, block overlap O = 128 tokens, and maximum total number of blocks Kmax = 32. Blocks shorter than L are padded with [PADs]. The final segmentation matrix C∈R is obtained. K×L .

[0134] Step 2.2 Block-level encoding: Each text block c i The data is sequentially input into the Chinese-RoBERTa-wwm-ext encoder, and after overlaying positional and segmental encodings, 12 layers of Transformer encoding are performed. The 768-dimensional output of the last layer [CLS] position is taken as the semantic representation h_i of the block. The representations of all blocks are stacked into a block-level representation matrix H∈R. K×768 .

[0135] Step 2.3 Inter-block fusion: Learnable block location encoding P is superimposed on H. chunk ∈R K×768 And concatenate the global [CLS] token h global We get H'∈R (K+1)×768 H' is input into a 4-layer Transformer Encoder for inter-block information fusion, enabling the model to capture long-distance semantic dependencies across blocks.

[0136] Step 2.4 Document representation generation: The attention pooling mechanism is used to perform weighted aggregation on the fused block representation. Specifically, the attention score of each block is calculated by equation (14), and the weight is obtained by softmax normalization. Finally, the document representation d is obtained by weighted summation according to equation (16), and dropout regularization (p=0.3) is performed on d.

[0137] Step 2.5 Binary Classification Output: Input the document representation d into the classification layer (Wc∈R) 2×768 The softmax function outputs the binary probabilities of infectious and non-infectious fever.

[0138] Step 2.6 Loss Calculation and Backpropagation: Using the cross-entropy loss function shown in equation (19) plus an L2 regularization term, and employing the AdamW optimizer with a learning rate of 2×10⁻⁶, -5 Backpropagation and parameter updates are performed, and the training is conducted for a total of 10 rounds. After each round, the performance is evaluated on the validation set, and the weights of the model with the best performance on the validation set are saved.

[0139] 3 Input Configuration To simulate different clinical scenarios, this embodiment sets up three progressive data input configurations: AR (Admission Record Only): Provides only the patient's admission record text, simulating the initial outpatient visit scenario; AR+INV1 (Admission Record + Initial Examination Results): Based on the admission record, add the results of the first laboratory test (complete blood count, CRP, PCT, etc.) and imaging examination after admission to simulate the initial assessment scenario in hospitalization; AR+INVall (Admission Record + All Test Results): Provides admission records and all available test results during hospitalization, simulating the final comprehensive assessment scenario.

[0140] Example 5 This embodiment uses Qwen2.5-VL-7B-Instruct as the base model for fine-grained diagnostic generation. Alternatively, open-source large language models such as GLM-4, Baichuan2-13B, ChatGLM3-6B, or LLaMA3-70B can be used, and fine-tuned and adapted using the same method.

[0141] The LoRA fine-tuning parameter configurations are shown in Table 40: Table 40

[0142] The steps for fine-tuning include: Step 1: Constructing the prompt word template: Based on the binary classification results of coarse-grained classification unit 201, construct conditional prompts. If the fever is determined to be infectious, construct the input: x = “[INST] Based on the determination of infectious fever, the patient's medical record is as follows:\n” + medical record text + “\nPlease provide the Top-5 differential diagnoses, diagnostic basis, and treatment suggestions.[ / INST]”; if the fever is determined to be non-infectious, modify the determination result in the prompts accordingly.

[0143] Step 2: LoRA parameter initialization: For the pre-trained weight matrix W o ∈R d×k Randomly initialize a low-rank matrix A∈R r×k (Gaussian distribution) and B∈R d×r (Zero initialization) makes ΔW=BA zero at the start of training, ensuring that the initial output of the model is consistent with the pre-trained model.

[0144] Step 3: Autoregressive Training The constructed input sequence x is input into the model, and the diagnostic output y is generated through autoregression. The training objective is to minimize the weighted joint of the negative log-likelihood loss, the Rank-5 diagnostic ranking loss, and the ranking loss, as shown in Equation (30). The AdamW optimizer is used with a learning rate of 3×10⁻⁶. -4 The parameters are updated, the batch size is 4, the gradient accumulation is 4 steps, and a total of 5 training rounds are performed.

[0145] Step 4: Save the model. After training, only the LoRA adapter weights (matrices A and B) are saved, while the base model weights remain unchanged. The LoRA adapter weight file is approximately 120MB in size, facilitating deployment and version management.

[0146] The steps for reasoning are as follows: Step 1 Input Construction: Construct prompt words and input sequences based on the binary classification results of coarse-grained classification unit 201.

[0147] Step 2: Autoregressive Generation The generation parameters are set as follows: maximum number of new tokens = 1024, temperature = 0.3, top_p = 0.9, and repetition penalty = 1.1. The model generates diagnostic output token by token using an autoregressive approach.

[0148] Step 3: Structured parsing: The generated output text is parsed in a structured manner to extract the Top-5 differential diagnosis list, the confidence score of each diagnosis, diagnostic basis keywords and treatment suggestions, and output according to the preset JSON Schema.

[0149] Example 6 This embodiment optimizes the temperature parameter T on the validation set. Specifically, the validation set is input into the model to obtain the logits vector z for each sample. The probability after temperature scaling is calculated according to equation (34), and the negative log-likelihood shown in equation (35) is used as the optimization objective. A grid search is employed to search within the range T∈[0.5, 2.0] with a step size of 0.1, selecting the T value that minimizes NLL. In a preferred embodiment of this invention, the T value is approximately 1.2 in a binary classification task.

[0150] Based on temperature scaling, the output probability is Platt scaled using equation (36), where s i Let be the original confidence scores, and 'a' and 'b' be learnable parameters. Using validation set data, and with log loss as the optimization objective, the L-BFGS algorithm is used to optimize 'a' and 'b'.

[0151] This embodiment uses the Bayesian probabilistic decomposition method shown in equations (31) and (32) to fuse the outputs of the coarse-grained classification unit 201 and the fine-grained diagnostic unit 202. The fine-grained diagnostic probability is calculated under a given classification condition. The diagnosis with the highest posterior probability is taken as the final output.

[0152] Example 7 This embodiment provides the system deployment architecture of the above system, as follows: Edge deployment: The trained Hierarchical BERT binary classification model (approximately 110M parameters) is deployed on the hospital's local server or outpatient workstation. The edge is responsible for: (1) receiving electronic medical record text input in real time; (2) performing long text segmentation, encoding, and fusion; and (3) outputting binary classification results. The edge inference time is approximately 0.5-1 second, meeting the real-time response requirements of the outpatient department.

[0153] Cloud Deployment: The LoRA-tuned Qwen2.5-VL-7B-Instruct model (base model parameters approximately 7B, LoRA adapter approximately 120MB) is deployed on a cloud GPU server. The cloud is responsible for: (1) receiving binary classification results and medical record text transmitted from the edge; (2) performing fine-grained Top-5 diagnosis generation; and (3) performing RAG knowledge retrieval and evidence push. The cloud-based single-inference takes approximately 2-4 seconds.

[0154] Communication Mechanism: The edge device communicates with the cloud via HTTPS / RESTful API, and data is transmitted using AES-256 encryption to ensure patient privacy and security. The edge device sends the binary classification results and the anonymized medical record text JSON to the cloud API endpoint. After completing the inference, the cloud returns the structured diagnostic results.

[0155] The workflow from edge to cloud is as follows: Step 1 Input: Clinicians input the electronic medical record text into the system (supports text copy and paste or automatic acquisition via EMR interface).

[0156] Step 2 Edge processing: The edge process completes long text encoding and binary classification reasoning within 0.5 to 1 second, and outputs the infected / non-infected judgment and confidence score.

[0157] Step 3: Cloud Processing: If a physician requests a fine-grained diagnosis, the edge device encrypts the binary classification results and the medical record text before sending them to the cloud. The cloud completes the generation of the fine-grained Top-5 diagnosis and RAG knowledge retrieval within 2-4 seconds, returning a structured diagnostic result.

[0158] Step 4 Results Display: The system front end displays the following in 3-5 seconds: (1) Primary etiology classification results and confidence level; (2) Top-5 differential diagnosis list; (3) Main basis for each diagnosis; (4) Traceable evidence-based medicine references or guideline excerpts.

[0159] Example 8 This embodiment provides an example of knowledge push and automated medical record generation based on RAG, as detailed below: I. Knowledge Base Construction Step 1: Knowledge Source Collection: Collect authoritative medical texts such as FUO-related clinical guidelines, expert consensus, medical textbooks, and high-quality review papers, covering the spectrum of FUO-related diseases, including infectious diseases, non-infectious inflammatory diseases, and oncological diseases.

[0160] Step 2: Text Blocking: The collected medical text is divided into text blocks according to semantic units. Each block is no longer than 512 tokens, and appropriate overlap is maintained between blocks.

[0161] Step 3: Vectorization Indexing: Use an embedding model to convert each text block into a vector representation and build the FAISS vector index library. The embedding model can be BGE, text-embedding-3, or Qwen's own embedding model.

[0162] II. Knowledge Retrieval and Push Step 1: Query Construction: Construct a search query based on the system's final diagnosis results and key medical record features (such as "fever + rash + joint pain + adult Still's disease").

[0163] Step 2 Vector Retrieval: After vectorizing the query, retrieve the top-5 most relevant knowledge text blocks in the FAISS index, and use cosine similarity as the similarity measure.

[0164] Step 3 Knowledge Enhancement Generation: The retrieved Top-5 knowledge text blocks are used as context, concatenated with the original query, and then input into the large language model to guide the model to generate diagnostic explanations and treatment suggestions with evidence-based evidence.

[0165] III. Automated Medical Record Generation After the physician completes the consultation, the system automatically generates a standardized outpatient medical record based on the structured data (chief complaint, present illness, past medical history, physical examination, and auxiliary examination results) obtained by the integrated information extraction module and the diagnostic conclusion output by the hierarchical diagnosis module. The medical record template includes: (1) chief complaint; (2) present illness; (3) past medical history; (4) physical examination; (5) auxiliary examination; (6) preliminary diagnosis; and (7) treatment suggestions. The physician can manually review and revise the automatically generated content before confirming and saving it.

[0166] Example 9 This embodiment provides a multicenter randomized controlled simulated case assessment, as detailed below: I. Case Selection Twenty typical cases were selected from the FUO disease database, covering 20 different etiological categories: Five cases of bacterial infection were reported: infective endocarditis, community-acquired pneumonia, pituitary abscess, complicated urinary tract infection, and chronic hematogenous disseminated pulmonary tuberculosis. Two cases of viral infection: infectious mononucleosis and viral encephalitis; One case of fungal infection: Pneumocystis pneumonia; One case of parasitic infection: liver abscess; Seven cases of non-infectious inflammatory diseases: vasculitis, sarcoidosis, polymyalgia rheumatica, Crohn's disease, dermatomyositis, adult-onset Still's disease, and systemic lupus erythematosus; Three cases of neoplastic diseases: Castleman's disease, lymphoma, and myelodysplastic syndrome; One other disease: drug fever.

[0167] All cases were reviewed and confirmed by a panel of senior infectious disease specialists, based on complete clinical data, laboratory tests, imaging examinations, and treatment response data.

[0168] II. Physician Recruitment and Grouping Step 1 Recruitment Criteria: Open to clinical physicians from 95 hospitals nationwide, including departments related to FUO diagnosis and treatment such as infectious diseases, respiratory medicine, rheumatology and immunology, hematology, and emergency medicine.

[0169] Step 2: Stratified randomization: Patients are randomly assigned to the AI-assisted diagnosis group and the independent diagnosis group at a 1:1 ratio. The randomization process is stratified according to hospital level (Grade A, Grade B, others) and years of medical practice (<5 years, 5-10 years, >10 years) to ensure balance between groups.

[0170] Step 3 Sample size: A total of 237 clinicians were recruited. In the end, the AI ​​group completed 73 valid questionnaires and the Non-AI group completed 73 questionnaires, with an effective response rate of 62.6%.

[0171] III. Diagnostic Process Step 1: Case Review. Participating physicians review standardized case materials in sequence, including medical history summary, physical examination, laboratory test results, and imaging reports. The order of all cases is randomly shuffled for each participant to eliminate the order effect. Step 2: Two-level diagnostic decision-making: Primary diagnosis: determined to be infectious fever or non-infectious disease; Secondary diagnosis: If it is determined to be infectious, further differentiation is needed to identify bacteria, fungi, viruses, parasites, or other pathogens; if it is determined to be non-infectious, further differentiation is needed to identify non-infectious inflammatory diseases, neoplastic diseases, or others.

[0172] Step 3: AI Intervention: The AI-assisted diagnosis group includes a "Get AI Hints" button below each case. Clicking this button displays the system's initial diagnostic suggestion. The AI ​​hints have a preset overall accuracy rate of 80% (16 correct and 4 incorrect out of 20 cases). The independent diagnosis group does not provide any AI assistance. Both groups have a 60-minute time limit, with a timer displayed at the top of the page.

[0173] Step 4 Data Recording: The system automatically records the content of the answers, the time taken to answer, and the use of AI prompts (AI group) for each case.

[0174] IV. Calculation of Evaluation Indicators Step 1 Level 1 diagnostic accuracy: Number of cases correctly classified as infected / non-infected / Total number of cases × 100%.

[0175] Step 2 Secondary diagnostic accuracy: assuming the primary diagnosis is correct, the number of cases with correct etiological subclass identification / total number of cases × 100%.

[0176] Step 3: AI Persuasion Index Calculation: For each base model, calculate using the following formula: AI Persuasion Index = (P base +HFDF - P base ) / (P max -P min ), where P base P represents the diagnostic accuracy of the base model. base+HFDF indicates the accuracy of the enhanced model after integrating HFDF, P max With P min These represent the maximum and minimum accuracy values ​​of the two models under the same data input and evaluation dimension, respectively.

[0177] Example 10 like Figure 5 As shown, this embodiment provides a typical case diagnosis example, as follows: Taking a hospitalized case of fever of unknown origin as an example, this paper illustrates the complete diagnostic process of this application.

[0178] Present Illness: In April 2020, the patient developed headache and neck pain, accompanied by neck tension, without limited range of motion. This was accompanied by temporomandibular joint pain, difficulty opening the mouth, and general weakness, without blurred vision, nausea, vomiting, or fever. On April 28th, a cranial MRI at XXX Hospital showed multiple ischemic lesions in the bilateral corona radiata and subcortical frontal lobes. An ultrasound revealed liver cysts and gallstones; the spleen, pancreas, and both kidneys showed no significant abnormalities, and the bilateral carotid arteries were also normal. Treatment with Ginkgo biloba extract did not improve the headache and temporomandibular joint pain.

[0179] On May 2nd, the patient developed a fever with a maximum temperature of 38.3°C (Tmax). There were no chills, rigors, cough, sputum production, rash, or muscle and joint pain. Blood tests showed a white blood cell count of 9.33 × 10^9 / L, normal neutrophils (N%) of 73.3%, hemoglobin (Hb) of 116 g / L, and plaque-releasing hormone (Plt) of 368 × 10^9 / L. The electrocardiogram was normal. The patient was given cefixime 0.1 g twice daily for 6 days for anti-infective treatment. The fever did not subside.

[0180] On May 9th, the patient visited the Department of Stomatology at XXX Hospital due to headache and pain at the base of the tongue. Candida albicans was cultured from the oral secretions, and the patient was treated with oral fluconazole and Kangfuxin.

[0181] On May 19th, a blood routine test at XXX Hospital showed WBC 8.2×10^9 / L, N% 68.7%, Hb 102g / L, Plt 407×10^9 / L, CRP 99.5mg / L. EBV VCA IgG and EBNA IgG were elevated, while cytomegalovirus DNA and EBV-DNA quantification were normal. The patient was treated with 0.5 tablets of Xin Huang Pian twice daily and 1 tablet of Kelepirt once daily. The subsequent fever peak decreased, with a maximum temperature of 37.5°C.

[0182] On June 1st, a follow-up blood test showed: WBC 8.5×10^9 / L, N% 65.9%, Hb 95g / L, Plt 425×10^9 / L, CRP 93.5mg / L, and erythrocyte sedimentation rate 103mm / h. Electroencephalogram (EEG) showed segmental slow waves, prominent in the central frontal region, mixed with a small number of sharp wave discharges. The patient's headache had improved, but a low-grade fever persisted. Recently, the patient experienced stiffness in the lower limb joints and sought medical attention at our outpatient clinic. To further clarify the cause of the fever, the patient has been admitted to our department for hospitalization.

[0183] Since the onset of illness, the patient's mental state has been good, appetite is good, sleep is poor, bowel movements are normal, and weight has decreased significantly by 6 kg. Current medication: None.

[0184] Preliminary diagnosis: Fever of unknown origin; chronic antral gastritis; liver cyst; gallstones The processing flow in this embodiment is as follows: The system extracts structured information from medical history texts, such as duration of fever, body temperature, accompanying symptoms such as joint swelling and pain, significantly elevated CRP (C-reactive protein) and ESR (erythrocyte sedimentation rate), positive ANA (antinuclear antibody), normal PCT, negative blood culture, and no abnormalities found in imaging.

[0185] Based on the Hierarchical BERT binary classifier and the fine-tuned Qwen2.5-VL-7B-Instruct based on the determination results of infectious / non-infectious fever, the Top-5 differential diagnoses are output as follows: (1) Non-infectious - autoinflammatory disease; (2) Infectious - fungal infection; (3) Non-infectious - subacute thyroiditis; (4) Infectious - acute upper respiratory tract infection; (5) Non-infectious - rheumatic fever.

[0186] The system retrieves and recommends supplementary medical history, physical examination, tests, and examinations needed to complete a diagnosis of inflammatory disease based on diagnostic criteria, treatment guidelines, and evidence-based evidence, and provides medication recommendations (such as...). Figure 5 (As shown).

[0187] Final outcome: The final clinical discharge diagnosis for this case was polymyalgia rheumatica. Based on the expert consensus on the diagnosis and treatment of fever of unknown origin, polymyalgia rheumatica falls under the category of non-infectious—non-infectious inflammatory diseases—autoinflammatory diseases. The gold standard diagnosis confirmed by the expert panel was autoinflammatory disease, consistent with the system's recommended primary diagnosis.

[0188] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fever diagnosis system based on a hierarchical cascaded large language model, characterized in that, include: The information extraction module is used to extract key medical information from doctor-patient dialogues or electronic medical records in a structured manner. The key medical information includes the patient's basic information, fever time, body temperature curve, accompanying symptoms, history of chronic diseases, test results and imaging examination summary. A hierarchical diagnostic module, based on the aforementioned key medical information, employs a two-stage cascaded architecture to achieve progressive diagnostic reasoning from coarse-grained to fine-grained. The hierarchical diagnostic module includes: The coarse-grained classification unit uses a Hierarchical BERT binary classification model based on hierarchical long text encoding to perform preliminary classification of structured medical record texts into infectious and non-infectious fever, and outputs the classification results and confidence scores. The fine-grained diagnostic unit uses the Qwen2.5-VL-7B-Instruct model with efficient parameter fine-tuning to perform fine-grained etiological differential diagnosis based on the classification results, generating a candidate diagnosis list arranged in descending order of confidence. The cascaded decision fusion unit fuses the classification results with the candidate diagnosis list according to Bayes' theorem, calculates the posterior probability of each candidate diagnosis, and outputs the final diagnosis suggestion. The knowledge push module, based on search-enhanced generation technology, provides evidence-based medicine support for the final diagnostic recommendations, including relevant medical literature retrieval, diagnostic basis generation, and treatment recommendation push.

2. The intelligent diagnostic system for fever based on a hierarchical cascaded large language model according to claim 1, characterized in that, The coarse-grained classification unit uses a sliding window overlapping segmentation mechanism to process long text medical records, performs block-level semantic encoding through Chinese-RoBERTa-wwm-ext, and uses a multi-layer Transformer encoder to fuse information between blocks. Finally, it generates an adaptive document representation through an attention pooling mechanism.

3. The intelligent diagnostic system for fever based on a hierarchical cascaded large language model according to claim 1, characterized in that, The fine-grained diagnostic unit uses a low-rank adaptation method to perform task-specific fine-tuning on the Qwen2.5-VL-7B-Instruct model, and achieves diagnostic reasoning capabilities for fever-related diseases through conditional generation modeling and supervised fine-tuning objectives.

4. The intelligent diagnostic system for fever based on a hierarchical cascaded large language model according to claim 1, characterized in that, The intelligent diagnostic system for fever diagnosis adopts a collaborative deployment architecture of edge and cloud: The edge terminal is used to deploy the Hierarchical BERT binary classification model to achieve real-time coarse screening; The cloud platform uses the speech recognition API to call the finely tuned Qwen2.5-VL-7B-Instruct model to process fine-grained diagnostic requests.

5. The intelligent diagnostic system for fever based on a hierarchical cascaded large language model according to claim 1, characterized in that, The information extraction module includes: The speech-to-text conversion channel uses a speech recognition API to convert doctor-patient dialogues into text in real time, and extracts structured information through an information extraction unit based on a basic model. The text parsing channel directly extracts admission records, progress notes, laboratory test results, imaging results, and discharge summaries from the electronic medical record system. The extracted medical information is structured according to a preset JSON Schema, including fields such as patient basic information, fever time, body temperature curve, accompanying symptoms, past medical history, and test results.

6. The intelligent diagnostic system for fever based on a hierarchical cascaded large language model according to claim 1, characterized in that, The knowledge delivery module includes: The vector index building unit divides medical text into blocks and builds vector indexes through an embedding model; The dynamic retrieval unit dynamically retrieves Rank-K related text fragments as context based on the current medical record characteristics; The knowledge-enhancing generation unit uses relevant text fragments retrieved as context to guide the large model in generating traceable medical recommendations; The automated medical record generation unit automatically generates outpatient medical records based on integrated structured information, including chief complaint, present illness, past medical history, physical examination, and treatment recommendations.

7. A method for intelligent diagnosis of fever based on a hierarchical cascaded large language model, characterized in that, Includes the following steps: Step S1: Obtain electronic medical record data of patients with fever of unknown origin, de-identify the data, and have the expert group determine the final diagnosis as the gold standard label; Step S2: Extract key medical information from doctor-patient dialogues or electronic medical records in a structured manner. The key medical information includes the patient's basic information, fever time, body temperature curve, accompanying symptoms, history of chronic diseases, laboratory test results, and imaging examination summary. Step S3: Use the Hierarchical BERT binary classification model based on hierarchical long text encoding to perform preliminary classification of infectious and non-infectious fever in structured medical record texts, and output the classification results and confidence scores. Step S4: Perform temperature scaling calibration and Platt scaling calibration on the confidence scores; Step S5: Using the Qwen2.5-VL-7B-Instruct model with efficient parameter fine-tuning, perform fine-grained etiological differential diagnosis based on the classification results, and generate a candidate diagnosis list sorted in descending order of confidence. Step S6: Based on Bayes' theorem, the classification results are cascaded and fused with the candidate diagnosis list to calculate the posterior probability of each candidate diagnosis and output the final diagnosis suggestion; Step S7: Based on search enhancement generation technology, provide evidence-based medical support for the final diagnostic recommendations, including relevant medical literature retrieval, diagnostic basis generation, and treatment recommendation delivery.

8. The intelligent diagnostic method for fever based on a hierarchical cascaded large language model according to claim 7, characterized in that, In step S3, the hierarchical long text encoding method includes: The input medical record document is divided into multiple text blocks using a sliding window overlapping segmentation mechanism. Each text block has a length of 510 tokens, an overlap of 128 tokens, and a step size of 382. The Chinese-RoBERTa-wwm-ext pre-trained model is used to perform semantic encoding on each text block, and the output of the last layer [CLS] position is taken as the block-level semantic representation. Inter-block information fusion is performed on the block-level representation matrix using a 4-layer Transformer encoder; An attention pooling mechanism is used to generate an adaptive document representation; The classification layer outputs the binary probability distribution and confidence score of infectious and non-infectious fever.

9. The intelligent diagnostic method for fever based on a hierarchical cascaded large language model according to claim 7, characterized in that, In step S5, the efficient parameter fine-tuning method includes: A low-rank adaptation method is used to fine-tune the pre-trained large language model. For the pre-trained weight matrix, the parameters are updated by low-rank decomposition.

10. The intelligent diagnostic method for fever based on a hierarchical cascaded large language model according to claim 7, characterized in that, In step S6, the cascade fusion method includes: Calculate the posterior probability of the candidate diagnosis based on Bayes' theorem; Temperature scaling and Platt scaling calibrations are applied to the confidence scores of coarse-grained classification. The diagnosis with the highest posterior probability is taken as the final output.

Images