A small model enhanced medical large model clinical decision reasoning method

Abstract

The present application relates to a small model enhanced medical large model clinical decision reasoning method, belonging to the technical field of medical information processing, solving the problem that the input length of the large model is limited in the prior art, affecting the clinical reasoning performance. The method comprises: obtaining the medical information of a plurality of patients in each department to construct a sample set corresponding to the department; constructing a small model corresponding to each department for dynamic self-adaptive compression of the input data; training the small model of the department based on the sample set of the department to obtain a trained small model; fine-tuning the medical large model based on double-channel optimization; the medical large model is used for key field extraction and clinical decision reasoning; inputting the medical information of the patient to be predicted into the small model to obtain the compressed medical information of the patient to be predicted; inputting the compressed medical information of the patient to be predicted into the fine-tuned medical large model for clinical decision reasoning. The accuracy and reliability of medical text analysis and clinical reasoning are improved.

Description

Technical Field

[0001] This invention relates to the field of medical information processing technology, and in particular to a small-model-enhanced medical large-model clinical decision-making reasoning method. Background Technology

[0002] In the practical application of clinical decision support systems, the core requirement is to accurately extract key medical events such as symptoms and medical history from multiple sources of medical texts, including patients' outpatient medical records and examination reports, and to perform logical reasoning based on these events to provide doctors with diagnostic or treatment suggestions. However, current mainstream technologies have significant limitations and are unable to meet the challenges of complex clinical scenarios.

[0003] Specifically, while small deep learning models like BERT-base offer advantages in computational efficiency and low deployment costs, they perform poorly when handling large-scale, long texts accumulated across departments and time periods. This is particularly true in complex reasoning tasks such as inferring the causes of multiple complications, where they often struggle to fully capture the deep semantic relationships between symptoms, examinations, and diagnoses. On the other hand, large models like GPT, DeepSeek, and Qwen excel in semantic processing, understanding, and complex reasoning, but their limited context length makes them ineffective at handling extremely long comprehensive medical records of chronic disease patients. Manual truncation can easily lead to the loss of crucial information, affecting the accuracy of reasoning. Therefore, effectively integrating the advantages of large and small models to build a system that supports both efficient processing of long texts and powerful clinical reasoning capabilities has become a key direction for overcoming current technological bottlenecks. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a small-model-enhanced medical large-model clinical decision-making reasoning method to solve the problem that the limited input length of existing large models affects the clinical reasoning performance.

[0005] On one hand, embodiments of the present invention provide a method for clinical decision-making reasoning in a large medical model enhanced by a small model, comprising the following steps:

[0006] For each department, medical information from multiple patients in that department is collected to construct a corresponding sample set for that department; Construct a small model corresponding to each department, which is used to dynamically and adaptively compress the input data; train the small model of the department based on the sample set of the department to obtain a trained small model; The medical model is fine-tuned based on dual-channel optimization; the medical model is used for key field extraction and clinical decision reasoning. The medical information of the patient to be predicted is input into the trained small model of the corresponding department to obtain the compressed medical information of the patient to be predicted. The compressed medical information of the patients to be predicted is input into the fine-tuned medical model for clinical decision-making reasoning.

[0007] Further improvements to the above methods include dynamic suggestion optimization and key field weight optimization.

[0008] Further improvements to the above methods involve fine-tuning the large-scale medical model based on dynamic suggestion optimization and key field weight optimization, including: Build a basic prompt word project and reserve dynamic insertion points within it; Build an error knowledge base; During each training round, the parameters of the large medical model are adjusted based on inference loss and key field extraction loss; After each round of training, dynamic insertion information is constructed based on the key fields extracted from the medical big data model and the error knowledge base and inserted into the dynamic insertion point to obtain the prompt word project for the next round of training.

[0009] Based on the above method, a further improvement is made to calculate the key field extraction loss using the following method: ; in, The true value of the i-th key field, This represents the i-th key field extracted from the large medical model. The number of key fields This represents the weight of the i-th key field in the t-th training round. This represents the loss function.

[0010] Based on the above method, the weight of the i-th key field is calculated in the following way: ; in, This represents the weight of the i-th key field during the (t-1)th round of training. Indicates the attenuation factor. This represents the normalized error rate of the i-th key field in round t-1.

[0011] Based on a further improvement of the above method, the small model includes: The adaptive segmentation module is used to adaptively segment the input medical information. The encoding module is used to obtain a feature sequence based on feature extraction from blocks; The decoding module is used to decode the feature sequence to obtain a summary of the input medical information.

[0012] Based on further improvements to the above method, the adaptive block segmentation module includes: Blocking unit; used to block medical information based on similarity; The overlap adjustment unit is used to adjust the block division based on the overlapping area between adjacent blocks determined by multi-factor fusion, so as to obtain the final block division result.

[0013] Based on the above method, the encoding module uses the following method to extract features from the blocks to obtain the feature sequence: For each block, feature extraction is performed to obtain the basic features of each block; For overlapping regions of adjacent blocks, the basic features corresponding to the overlapping regions are fused based on bidirectional attention to obtain the fused features of the overlapping regions; The technical features of the non-overlapping regions of each block and the fusion features of the overlapping regions of adjacent blocks are sequentially spliced ​​together to form a feature sequence.

[0014] Based on the above method, a further improvement is made to obtain the fused features of the overlapping regions by fusing the basic features corresponding to the overlapping regions using bidirectional attention, including: Project the basic features of the overlapping regions of two adjacent blocks onto the query, key, and value spaces, respectively; Bidirectional cross attention is calculated using multi-head attention based on the projected query matrix, key matrix, and value matrix; The fusion features of the overlapping region are obtained by position-weighted fusion of bidirectional cross-attention.

[0015] Based on a further improvement of the above method, the segmentation unit adaptively segments the input medical information in the following manner: S21. Extract sentence sequences of medical information; S22. Set the current sequence number to 1; set the block number of the first sentence to the current sequence number; take the second sentence in the sentence sequence as the current sentence; S23. Calculate the similarity between the current sentence and the block corresponding to the current sequence number; if the similarity is greater than the similarity threshold and the length of the block corresponding to the current sequence number and the total length of the current sentence do not exceed the length threshold, then set the block number of the current sentence to the current sequence number; otherwise, increment the current sequence number by one and set the block number of the current sentence to the current sequence number. S24. If there is a next sentence, then the next sentence is the current sentence, and return to step S23; otherwise, end the block segmentation.

[0016] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: 1. By combining the high efficiency and professionalism of small models with the broad semantic understanding capabilities of large models, the accuracy and reliability of medical text parsing and clinical reasoning are effectively improved.

[0017] 2. It can be applied to various practical clinical scenarios such as outpatient auxiliary diagnosis, inpatient treatment plan generation, and multi-departmental collaborative medical record analysis, significantly enhancing the intelligence level and service efficiency of medical decision-making.

[0018] 3. The output results are directly related to structured medical events, with clear logic and traceability, conforming to doctors' decision-making habits, which helps the system to be implemented and promoted.

[0019] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0020] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a flowchart of the clinical decision-making reasoning method for a large medical model enhanced by a small model, as described in an embodiment of the present invention. Figure 2 This is a block diagram of a small-model-enhanced medical large-model clinical decision-making reasoning system according to an embodiment of the present invention. Detailed Implementation

[0021] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0022] A specific embodiment of the present invention discloses a method for clinical decision-making reasoning in medical systems that enhances small-model approaches, such as... Figure 1 As shown, it includes the following steps: S1. For each department, obtain the medical information of multiple patients in that department to construct the corresponding sample set for that department; S2. Construct a small model corresponding to each department. The small model is used to dynamically and adaptively compress the input data. Train the small model of the department based on the sample set of the department to obtain a trained small model. S3. Fine-tune the large medical model based on dual-channel optimization; the large medical model is used for key field extraction and clinical decision reasoning. S4. Input the medical information of the patient to be predicted into the trained small model of the corresponding department to obtain the compressed medical information of the patient to be predicted. S5. Input the compressed medical information of the patient to be predicted into the fine-tuned medical model for clinical decision-making reasoning.

[0023] Compared to existing technologies, the small-model-enhanced medical large-model clinical decision-making reasoning method provided in this embodiment dynamically and adaptively compresses medical information by training corresponding small models for each department, thereby extracting effective information. This overcomes hardware and configuration limitations in actual deployment, avoids the loss of key information due to manual truncation, and improves the accuracy of key field extraction and clinical decision-making reasoning. Furthermore, it fine-tunes the large-model medical system through dual-channel optimization, further enhancing the model's reasoning capabilities. Through a collaborative mechanism of precise long-text compression and strong logical reasoning, it improves the accuracy, applicability, and interpretability of clinical decisions.

[0024] In practice, patient medical information includes medical text data from multiple sources such as outpatient medical records and examination reports. For each patient, a summary of the medical information, key field information, and doctor's diagnosis information are obtained as tags corresponding to that patient's medical information to construct a sample, thereby building a sample set.

[0025] Key fields include symptom fields, examination fields, medical history fields, and medication fields.

[0026] To address the dual bottlenecks in existing clinical decision support systems—the insufficient ability of small models to understand long texts and perform complex reasoning, and the difficulty of large models to handle extremely long medical texts due to context length limitations—this invention uses department-customized small models to adaptively compress medical information, achieving precise compression of long texts. In turn, the strong logical reasoning of large models improves the accuracy, applicability, and interpretability of decision-making.

[0027] It should be noted that the small model is designed for compressing long medical text information and does not include basic structured data such as the patient's name and age.

[0028] In practice, the characteristics of medical information differ across departments. Therefore, small models are constructed for each clinical department (such as internal medicine, surgery, obstetrics and gynecology) based on their medical text features. It should be noted that the small models can use BERT models. For each department, a small model is trained using samples from that department to obtain that department's small model. That is, the structure of the small models for each department is the same, but the model parameters are different.

[0029] Specifically, the small models constructed include: The adaptive segmentation module is used to adaptively segment the input medical information. The encoding module is used to obtain a feature sequence based on feature extraction from blocks; The decoding module is used to decode the feature sequence to obtain a summary of the input medical information.

[0030] Because medical text information, especially information about chronic diseases, is often very long, exceeding the model's processing capacity, the smaller model uses an adaptive block-splitting module to divide the input medical information into blocks, facilitating subsequent feature extraction.

[0031] Specifically, the adaptive block segmentation module includes: Blocking unit; used to block medical information based on similarity; The overlap adjustment unit is used to adjust the block division based on the overlapping area between adjacent blocks determined by multi-factor fusion, so as to obtain the final block division result.

[0032] During implementation, the segmentation unit adaptively segments the input medical information in the following manner: S21. Extract the sentence sequence of medical information and obtain the embedding vector of each sentence; S22. Set the current sequence number to 1; set the block number of the first sentence to the current sequence number; take the second sentence in the sentence sequence as the current sentence; S23. Calculate the similarity between the current sentence and the block corresponding to the current sequence number; if the similarity is greater than the similarity threshold and the length of the block corresponding to the current sequence number and the total length of the current sentence do not exceed the length threshold, then set the block number of the current sentence to the current sequence number; otherwise, increment the current sequence number by one and set the block number of the current sentence to the current sequence number. S24. If there is a next sentence, then the next sentence is the current sentence, and return to step S23; otherwise, end the block segmentation.

[0033] To ensure the integrity of semantic units, the block division of this invention divides medical information into blocks based on sentences.

[0034] During implementation, the sentence sequence of medical information is extracted first. ,in, Let represent the i-th sentence, and n represent the number of sentences. The embedding vector of the i-th sentence is represented as... In practice, the embedded representation of the sentence can be obtained through an existing sentence encoder (such as BERT's [CLS] token).

[0035] During implementation, initially, the current sequence number is one. (First sentence) The block number is the current block number, that is, the block number is one. The current sentence is the second sentence. ,calculate Similarity to the first block (currently there is only one sentence) If the similarity is greater than the similarity threshold and the length of the block corresponding to the current sequence number is... If the total length does not exceed the length threshold, then... Set the block number to the current number one; otherwise, increment the current number by one before proceeding. The block number is set to the current number, that is... This marks the beginning of a new block. Then, with... Return to step S13 for the current sentence, and so on, until the block segmentation ends.

[0036] In practice, when calculating the similarity between the current sentence and the block corresponding to the current sequence number, the embedding vector of the block corresponding to the current sequence number is defined as the average pooling of the embedding vectors of the sentences within the block corresponding to the current sequence number. Then, the similarity between the embedding vector of the current sentence and the embedding vector of the block corresponding to the current sequence number is calculated. For example, cosine similarity can be calculated.

[0037] During implementation, the similarity threshold can be set according to the similarity calculation method used. The length threshold can be set according to the processable length of the model.

[0038] To maintain semantic continuity, the overlapping areas between adjacent blocks are determined by the overlap adjustment unit based on multi-factor fusion, and the block segmentation is adjusted to obtain the final segmentation result.

[0039] Specifically, the block division is adjusted based on the overlapping areas between adjacent blocks determined by multi-factor fusion, resulting in the final block division result, including: ; in, This represents the size of the overlapping region between the i-th block and the (i+1)-th block determined based on semantic units. This represents the size of the overlapping region between the i-th block and the (i+1)-th block, determined based on the block length. This represents the size of the overlapping region between the i-th block and the (i+1)-th block, determined based on entity density. , and All represent weighting coefficients. This represents the length of the overlapping area between the final determined i-th block and the (i+1)-th block.

[0040] get Then, search backwards from the end of the i-th block. The position of the length is used as the starting position of the (i+1)th block. In order to maintain the continuity of the statement, if the starting position of the search is in the middle of the statement, then the starting position of the statement is used as the starting position of the (i+1)th block.

[0041] Specifically, the following formula is used for calculation. : ; in, Indicates the minimum overlap length. Indicates the maximum overlap length. Let represent the set of the last k sentences of the i-th block. `s` represents the length of sentence `s`. `k` represents the minimum number of overlapping sentences, for example, 2 overlapping sentences, 3 overlapping sentences. In implementation, and It can be set according to factors such as the length of medical text and computational efficiency requirements.

[0042] Specifically, the following formula is used for calculation. : ; in, This represents the basic ratio, for example, it can be 0.2. This represents the length of the i-th block. This represents the length of the (i+1)th block.

[0043] Specifically, the following formula is used for calculation. : ; ; ; ; in, This represents the number of entities within the last k statements of the i-th block. Indicates the minimum overlap length. This represents the length of the last k statements in the i-th block. This represents the number of entities in the i-th block. This represents the density adjustment function.

[0044] During implementation, entities can be identified using existing NER models.

[0045] In implementation, the density adjustment function can be a piecewise function. 。

[0046] By considering the semantic units, length, and entity density of adjacent blocks, the semantics of overlapping areas are made smoother and the connections are more reasonable.

[0047] After the medical information is divided into blocks, the encoding module extracts features from the blocks to obtain feature sequences.

[0048] Specifically, the encoding module extracts features from the blocks to obtain feature sequences in the following way: For each block, feature extraction is performed to obtain the basic features of each block; For overlapping regions of adjacent blocks, the basic features corresponding to the overlapping regions are fused based on bidirectional attention to obtain the fused features of the overlapping regions; The technical features of the non-overlapping regions of each block and the fusion features of the overlapping regions of adjacent blocks are sequentially spliced ​​together to form a feature sequence.

[0049] During implementation, the encoding module can use existing text feature extraction models to extract the basic features of each block, such as a lightweight Albert model.

[0050] For overlapping regions of adjacent blocks, feature fusion is performed based on bidirectional attention.

[0051] Specifically, based on bidirectional attention, the following formula is used to fuse the basic features corresponding to overlapping regions. This includes: Project the basic features of the overlapping regions of two adjacent blocks onto the query, key, and value spaces, respectively: ; ; ; ; ; ; in, , and Let these represent the query projection matrix, key projection matrix, and value projection matrix of the j-th attention head, respectively. This represents the basic features of the overlapping portion of the i-th block with the (i+1)-th block. This represents the basic features of the part that overlaps with the i-th block in the (i+1)-th block. , and The query matrix, key matrix, and value matrix represent the query matrix, key matrix, and value matrix of the j-th attention head corresponding to the basic features of the part overlapping with the (i+1)-th block in the i-th block; , and This represents the query matrix, key matrix, and value matrix of the j-th attention head corresponding to the basic features of the part overlapping with the i-th block in the (i+1)-th block.

[0052] Bidirectional cross attention is calculated using multi-head attention based on the projected query matrix, key matrix, and value matrix; ; ; ; ; in, This represents the attention calculated by the j-th attention head; and This represents the attention mask matrix, used to control which locations can pay attention to each other. The superscript T indicates transpose. The dimension of the attention head is represented by h. h represents the number of attention heads. Indicates the output projection matrix. Indicates splicing, This represents the softmax function.

[0053] Position-weighted fusion of bidirectional cross-attention yields fusion features of overlapping regions: ; ; ; ; in, This refers to the dropout function. The layer normalization function is represented. This represents a multilayer perceptron. This represents the sigmoid function. This represents the fusion feature of the overlapping region between the i-th block and the (i+1)-th block; This represents the fusion feature of the region overlapping with the ith block in the (i+1)th block. express The fusion feature at position p in the middle, express The fusion feature at position p in the middle, This represents the weight of the p-th position. This represents the fused feature at the p-th position in the final fused feature set. This represents the final fusion feature obtained.

[0054] Since calculations in two directions may produce different results, adaptive weighting is required during the final fusion process. This involves calculating the weights for each direction at each location to improve feature accuracy and maintain global coherence.

[0055] Then, the technical features of the non-overlapping regions of each block and the fusion features of the overlapping regions of adjacent blocks are sequentially spliced ​​together to form a feature sequence.

[0056] For example, the block sequence obtained by the adaptive block segmentation module is: .

[0057] In practice, during the stitching process, only the right overlapping region of the previous block needs to be extracted (because the left overlapping region of the subsequent block and the right overlapping region of the previous block are the same region and have already been merged into the same features). For example, in the segmentation... Middle and Block The overlapping part is represented as The other parts are represented as Segmentation Middle and Block The overlapping part is represented as The other parts are represented as Segmentation Middle and Block The overlapping part is represented as The other parts are represented as The feature sequence formed by sequentially splicing together is .

[0058] During implementation, the obtained feature sequences are decoded to output a summary of medical information. The decoding module can employ a single-layer Transformer decoder.

[0059] In implementation, for each department's mini-model, a trained mini-model is obtained by training it on the department's sample set. The training loss for the mini-model can use existing loss functions, such as cross-entropy loss or mean squared error loss. For example, the uncompressed text may contain irrelevant or potentially misleading information such as precautions, differential diagnoses, and suggestions provided during diagnosis and treatment. The compressed text will remove this information to avoid misleading the large model's reasoning, resulting in a shorter text length for the compressed medical information compared to the uncompressed version.

[0060] For large-scale medical models used for key field extraction and clinical decision reasoning, this invention employs dual-channel optimization for fine-tuning. In implementation, large-scale medical models such as Qwen and GPT can be used.

[0061] Specifically, dual-channel optimization includes dynamic prompt optimization and key field weight optimization, which involves fine-tuning the large medical model based on dynamic prompt optimization and key field weight optimization.

[0062] Specifically, fine-tuning the large-scale medical model includes: Build a basic prompt word project and reserve dynamic insertion points within it; Build an error knowledge base; During each training round, the parameters of the large medical model are adjusted based on inference loss and key field extraction loss; After each round of training, dynamic insertion information is constructed based on the key fields extracted from the medical big data model and the error knowledge base and inserted into the dynamic insertion point to obtain the prompt word project for the next round of training.

[0063] When training the large model, the input data includes basic structured data of patients and prompts containing medical information summaries. The output consists of extracted key fields and clinical decision reasoning results. During implementation, existing large model fine-tuning techniques (such as LoRA, QLoRA, adapter techniques, etc.) are used to fine-tune the parameters of the medical large model.

[0064] During implementation, first design the basic prompt word project and reserve dynamic insertion points, for example: """Document Medical Records: {input_text} JSON used to describe the surgery: [ { "Surgery time": "", "Combined surgical procedure name": "", "Sub-operation": [ { "Surgery Name": "", "Surgical Site": "", "Surgery Status": "", Surgical procedure: "", "Surgical approach": "", "Extended surgical resection area": ​​"", "Assemblage method": "", "Nature of surgery": "" } ], Intraoperative blood loss: "", "Intraoperative blood loss in units": "", "Transfusion Status": "", "Details of intraoperative blood transfusion": [ { "Intraoperative blood transfusion components": "", Intraoperative blood transfusion volume: "", "Intraoperative blood transfusion volume (units): "" } ], "Anesthesia method": "", ASA Score: "", "Intraoperative / postoperative complications": [ { Symptoms / Diagnosis: "", "Symptoms / Diagnostic Status": "", "Symptoms / Diagnosis Time": "" } ], "Intraoperative medication": [ { "Drug Name": "", "Method of administration": "", Dosage: " } ] } ] Field description: Please note that multiple surgery times may occur, and all of them must be output.

[0065] Combined surgical procedure name: Sometimes a single surgery involves multiple procedures. The medical record will provide a combined name for this surgery, extracted from the original medical record's 'Surgery / Procedure Name'. For example: AAA+BBB+CCC, etc.

[0066] Surgical Name: Take the names corresponding to the surgical steps included in the 'Combined Surgical Name', one by one, in sequence. For example, if the combined surgical name is 'Distal Gastrectomy, D2 Lymph Node Dissection, Billroth II + Braun Anastomosis', the sub-surgical names are 'Distal Gastrectomy', 'D2 Lymph Node Dissection', and 'Billroth II + Braun Anastomosis', respectively. Surgical site: Only take the value from the name of the surgery, such as: the surgical site of total gastrectomy is the whole stomach.

[0067] Surgical status: Value range: confirmed / suspected / denied Surgical procedure: Examples include open surgery, laparoscopic surgery, endoscopic surgery, robotic surgery, etc. If no specific procedure is mentioned, output a null value.

[0068] Surgical nature: Exploration, diagnosis, radical treatment, modified radical treatment, palliative. If the specific surgical nature is not explicitly mentioned, output a null value.

[0069] Surgical approach: refers to the path through which surgery enters the surgical area. For example, the surgical approach for laparoscopy is the abdomen, the surgical approach for thoracoscopic surgery is the chest, the surgical approach for open surgery is the abdomen, and the surgical approach for thoracolaparoscopy is the chest and abdomen.

[0070] Anastomosis method: refers to the reconstruction method that connects the severed ends after surgery. Examples include: Billroth I, Billroth II, Billroth II + Braun, Roux-en-Y, Uncut Roux-en-Y, and esophageal-gastric remnant anastomosis.

[0071] Surgical resection scope: Only the value is taken from the name of the surgery, such as the surgical site of total gastrectomy is the whole stomach.

[0072] ### [Dynamic Insertion Point - START] Dynamic content includes: Error correction rules based on an error knowledge base - Field weight adjustment instructions ### [Dynamic Insertion Point - END] Read the surgical names in the above medical records and describe the other surgeries mentioned in the records, excluding those mentioned in the postoperative status description, using the given JSON format. If no surgeries meet the criteria, output an empty list.

[0073] """ During implementation, an error knowledge base is built based on experience. The error knowledge base stores the error types that may occur for each key field, as well as the corresponding prompts for that error.

[0074] During implementation, unrecorded error types are clustered to generate candidate knowledge entries. Human-machine collaborative verification is then implemented, automatically recommending new error patterns, which are then confirmed by domain experts to update the error knowledge base.

[0075] During each round of training of the medical big data model, the parameters of the medical big data model are adjusted based on inference loss and key field extraction loss.

[0076] Specifically, the inference loss can use existing loss functions, such as the cross-entropy loss function and the mean squared error loss function.

[0077] Specifically, the loss for extracting key fields is calculated using the following method: ; in, The true value of the i-th key field, This represents the i-th key field extracted from the large medical model. The number of key fields This represents the weight of the i-th key field in the t-th training round. This represents the loss function.

[0078] Specifically, the weight of the i-th key field is calculated using the following method: ; in, This represents the weight of the i-th key field during the (t-1)th round of training. Indicates the attenuation factor. This represents the normalized error rate of the i-th key field in round t-1.

[0079] By using a key field weighted loss function, the weights of each key field are calculated based on the error rate of the key fields in the previous round, so that the model training resources are tilted towards fields with high error rates, thereby improving the accuracy of key field extraction.

[0080] The total loss is obtained by summing the inference loss and the key field extraction loss, and the parameters of the large medical model are adjusted based on the total loss.

[0081] After each round of training, dynamic insertion information is constructed based on the key fields extracted from the medical big data model and the error knowledge base and inserted into the dynamic insertion point to obtain the prompt word project for the next round of training.

[0082] During implementation, after each training round, the key field information of errors extracted in that round is extracted. It is then determined whether this erroneous key field information exists in the error knowledge base. If it does, the corresponding prompt content is inserted into the dynamically reserved insertion point in the prompt word project. For example, if an error in the format of the surgical time key field is found after the current training round, the corresponding prompt content is inserted into the aforementioned prompt word project. The prompt word project then becomes: """Document Medical Records: {input_text} JSON used to describe the surgery: [ { "Surgery time": "", "Combined surgical procedure name": "", "Sub-operation": [ { "Surgery Name": "", "Surgical Site": "", "Surgery Status": "", Surgical procedure: "", "Surgical approach": "", "Extended surgical resection area": ​​"", "Assemblage method": "", "Nature of surgery": "" } ], Intraoperative blood loss: "", "Intraoperative blood loss in units": "", "Transfusion Status": "", "Details of intraoperative blood transfusion": [ { "Intraoperative blood transfusion components": "", Intraoperative blood transfusion volume: "", "Intraoperative blood transfusion volume (units): "" } ], "Anesthesia method": "", ASA Score: "", "Intraoperative / postoperative complications": [ { Symptoms / Diagnosis: "", "Symptoms / Diagnostic Status": "", "Symptoms / Diagnosis Time": "" } ], "Intraoperative medication": [ { "Drug Name": "", "Method of administration": "", Dosage: " } ] } ] Field description: Please note that multiple surgery times may occur, and all of them must be output.

[0083] Combined surgical procedure name: Sometimes a single surgery involves multiple procedures. The medical record will provide a combined name for this surgery, extracted from the original medical record's 'Surgery / Procedure Name'. For example: AAA+BBB+CCC, etc.

[0084] Surgical Name: Take the names corresponding to the surgical steps included in the 'Combined Surgical Name', one by one, in sequence. For example, if the combined surgical name is 'Distal Gastrectomy, D2 Lymph Node Dissection, Billroth II + Braun Anastomosis', the sub-surgical names are 'Distal Gastrectomy', 'D2 Lymph Node Dissection', and 'Billroth II + Braun Anastomosis', respectively. Surgical site: Only take the value from the name of the surgery, such as: the surgical site of total gastrectomy is the whole stomach.

[0085] Surgical status: Value range: confirmed / suspected / denied Surgical procedure: Examples include open surgery, laparoscopic surgery, endoscopic surgery, robotic surgery, etc. If no specific procedure is mentioned, output a null value.

[0086] Surgical nature: Exploration, diagnosis, radical treatment, modified radical treatment, palliative. If the specific surgical nature is not explicitly mentioned, output a null value.

[0087] Surgical approach: refers to the path through which surgery enters the surgical area. For example, the surgical approach for laparoscopy is the abdomen, the surgical approach for thoracoscopic surgery is the chest, the surgical approach for open surgery is the abdomen, and the surgical approach for thoracolaparoscopy is the chest and abdomen.

[0088] Anastomosis method: refers to the reconstruction method that connects the severed ends after surgery. Examples include: Billroth I, Billroth II, Billroth II + Braun, Roux-en-Y, Uncut Roux-en-Y, and esophageal-gastric remnant anastomosis.

[0089] Surgical resection scope: Only the value is taken from the name of the surgery, such as the surgical site of total gastrectomy is the whole stomach.

[0090] ### [Dynamic Insertion Point - START] Dynamic content includes: Surgical time: The output format for surgical time is yyyy-mm-dd ### [Dynamic Insertion Point - END] Read the surgical names in the above medical records and describe the other surgeries mentioned in the records, excluding those mentioned in the postoperative status description, using the given JSON format. If no surgeries meet the criteria, output an empty list.

[0091] """ By combining precise compression of long texts with a strong logical reasoning collaboration mechanism, the accuracy, applicability, and interpretability of clinical decision-making are improved, significantly enhancing the intelligence level and service efficiency of medical decision-making.

[0092] A specific embodiment of the present invention discloses a small-model-enhanced medical large-model clinical decision-making reasoning system, such as... Figure 2 As shown, it includes: The sample set construction module is used to obtain the medical information of multiple patients in each department and construct the corresponding sample set for that department. The small model training module is used to construct a small model corresponding to each department. The small model is used to dynamically and adaptively compress the input data. The small model of the department is trained based on the sample set of the department to obtain the trained small model. The large model fine-tuning module is used to fine-tune the large medical model based on dual-channel optimization; the large medical model is used for key field extraction and clinical decision reasoning. The reasoning module is used to input the medical information of the patient to be predicted into the trained small model of the corresponding department to obtain the compressed medical information of the patient to be predicted; and to input the compressed medical information of the patient to be predicted into the fine-tuned large medical model for clinical decision reasoning.

[0093] The above-described method and system embodiments are based on the same principles, and their related aspects can be referenced from each other to achieve the same technical effects. For specific implementation processes, please refer to the foregoing embodiments, which will not be repeated here.

[0094] Electronic device example: One specific embodiment of this application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the small-model-enhanced medical large-model clinical decision reasoning method in the method embodiment.

[0095] Examples of readable storage media: One specific embodiment of this application discloses a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the small-model-enhanced medical large-model clinical decision-making reasoning method in the method embodiment.

[0096] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0097] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for clinical decision-making reasoning in healthcare using a small-model-enhanced approach, characterized in that: Includes the following steps: For each department, medical information from multiple patients in that department is collected to construct a corresponding sample set for that department; Construct a small model corresponding to each department, which is used to dynamically and adaptively compress the input data; train the small model of the department based on the sample set of the department to obtain a trained small model; The medical model is fine-tuned based on dual-channel optimization; the medical model is used for key field extraction and clinical decision reasoning. The medical information of the patient to be predicted is input into the trained small model of the corresponding department to obtain the compressed medical information of the patient to be predicted. The compressed medical information of the patients to be predicted is input into the fine-tuned medical model for clinical decision-making reasoning.

2. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 1, characterized in that, The dual-channel optimization includes dynamic prompt optimization and key field weight optimization.

3. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 2, characterized in that, Fine-tuning of the large medical model based on dynamic suggestion optimization and key field weight optimization includes: Build a basic prompt word project and reserve dynamic insertion points within it; Build an error knowledge base; During each training round, the parameters of the large medical model are adjusted based on inference loss and key field extraction loss; After each round of training, dynamic insertion information is constructed based on the key fields extracted from the medical big data model and the error knowledge base and inserted into the dynamic insertion point to obtain the prompt word project for the next round of training.

4. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 3, characterized in that, The loss for extracting key fields is calculated using the following method: ； in, The true value of the i-th key field, This represents the i-th key field extracted from the large medical model. The number of key fields This represents the weight of the i-th key field in the t-th training round. This represents the loss function.

5. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 4, characterized in that, The weight of the i-th key field is calculated using the following method: ； in, This represents the weight of the i-th key field during the (t-1)th round of training. Indicates the attenuation factor. This represents the normalized error rate of the i-th key field in round t-1.

6. The method for clinical decision-making reasoning in a large-scale medical model enhanced by a small model according to claim 1, characterized in that, The small model includes: The adaptive segmentation module is used to adaptively segment the input medical information. The encoding module is used to obtain a feature sequence based on feature extraction from blocks; The decoding module is used to decode the feature sequence to obtain a summary of the input medical information.

7. The method for clinical decision-making reasoning in a large-scale medical model enhanced by a small model according to claim 6, characterized in that, The adaptive block partitioning module includes: Blocking unit; used to block medical information based on similarity; The overlap adjustment unit is used to adjust the block division based on the overlapping area between adjacent blocks determined by multi-factor fusion, so as to obtain the final block division result.

8. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 7, characterized in that, The encoding module extracts features from the blocks in the following way to obtain the feature sequence: For each block, feature extraction is performed to obtain the basic features of each block; For overlapping regions of adjacent blocks, the basic features corresponding to the overlapping regions are fused based on bidirectional attention to obtain the fused features of the overlapping regions; The technical features of the non-overlapping regions of each block and the fusion features of the overlapping regions of adjacent blocks are sequentially spliced ​​together to form a feature sequence.

9. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 8, characterized in that, The fused features of the overlapping regions are obtained by fusing the basic features corresponding to the overlapping regions based on bidirectional attention, including: Project the basic features of the overlapping regions of two adjacent blocks onto the query, key, and value spaces, respectively; Bidirectional cross attention is calculated using multi-head attention based on the projected query matrix, key matrix, and value matrix; The fusion features of the overlapping region are obtained by position-weighted fusion of bidirectional cross-attention.

10. The method for clinical decision-making reasoning in a large medical model enhanced by a small model according to claim 7, characterized in that, The segmentation unit adaptively segments the input medical information in the following manner: S21. Extract sentence sequences of medical information; S22. Set the current sequence number to 1; set the block number of the first sentence to the current sequence number; take the second sentence in the sentence sequence as the current sentence; S23. Calculate the similarity between the current sentence and the block corresponding to the current sequence number; If the similarity is greater than the similarity threshold and the length of the block corresponding to the current sequence number does not exceed the length threshold with the total length of the current sentence, then the block sequence number of the current sentence is set to the current sequence number; otherwise, the current sequence number is incremented by one, and the block sequence number of the current sentence is set to the current sequence number. S24. If there is a next sentence, then the next sentence is the current sentence, and return to step S23; otherwise, end the block segmentation.

Images