A medical auxiliary reasoning method and system based on a small language model and monte carlo tree search

Abstract

This invention relates to the field of medical informatics, providing a medical-assisted reasoning method and system based on a small language model and Monte Carlo tree search. The method includes: standardizing raw patient medical data to obtain an initial training dataset; expanding candidate diagnostic reasoning paths within a Monte Carlo tree search framework based on a small language model to obtain multiple candidate reasoning paths; inputting these multiple candidate reasoning paths into a process reward model, scoring the intermediate reasoning steps of each path to obtain step scores; ranking and filtering the candidate reasoning paths based on the step scores to obtain a set of high-quality reasoning paths; jointly training the policy model and the process reward model to obtain an optimized reasoning model; and generating reasoning results from the optimized reasoning model based on the optimal reasoning path to obtain a comprehensive suggestion set. This invention improves the interpretability of medical-assisted reasoning and enhances the robustness of the diagnostic system.

Description

Technical Field

[0001] This invention belongs to the field of medical informatics technology, and in particular relates to a medical auxiliary reasoning method and system based on small language models and Monte Carlo tree search. Background Technology

[0002] With the rapid development of deep learning technology, AI-based medical auxiliary diagnostic methods are widely used in disease detection, image analysis, and clinical decision support. Monte Carlo Tree Search (MCTS), a heuristic search algorithm based on random sampling, was initially applied to game theory reasoning and has been gradually introduced into complex reasoning tasks in recent years, providing an effective path search framework for multi-step decision problems. Meanwhile, Small Language Models (SLMs), due to their small parameter count, fast inference speed, and low deployment cost, have strong application potential in resource-constrained medical scenarios. Combining Small Language Models with Monte Carlo Tree Search provides a new technical approach for constructing lightweight and interpretable medical auxiliary diagnostic methods.

[0003] Most existing medical assistance methods rely on large language models for single-step reasoning, which involves inputting the patient's symptoms, examination results, and other medical information into the model all at once to directly generate a final diagnosis. However, this single-step reasoning approach cannot assess the quality and dynamically adjust intermediate steps in the reasoning process when dealing with complex cases or multi-system diseases. As a result, the model lacks the ability to identify and eliminate low-quality reasoning steps when generating the reasoning path, making it difficult to guarantee the accuracy and reliability of the final diagnosis. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of the invention is to provide a medical auxiliary reasoning method and system based on small language models and Monte Carlo tree search.

[0005] This invention provides a medical assisted reasoning method based on a small language model and Monte Carlo tree search, comprising: S1: Standardize the patient's raw medical data to obtain the initial training dataset; S2: Input the initial training dataset into the small language model, and expand the candidate diagnostic reasoning path under the Monte Carlo tree search framework to obtain multiple candidate reasoning paths; S3: Input multiple candidate reasoning paths into the process reward model, score the intermediate reasoning steps of the multiple candidate reasoning paths, and obtain the step scores of the multiple candidate reasoning paths; S4: Based on the step scores, sort and filter multiple candidate reasoning paths to obtain a high-quality reasoning path set; S5: Input the set of high-quality reasoning paths into the policy model and the process reward model for joint training to obtain an optimized reasoning model; S6: The optimized reasoning model generates reasoning results based on the optimal reasoning path to obtain a comprehensive set of suggestions.

[0006] According to the medical auxiliary reasoning method based on small language model and Monte Carlo tree search provided by the present invention, step S1 further includes: S11: Extract raw medical data from the patient's symptom records, physical signs data, laboratory test results, imaging data and medical history information, and preprocess the raw medical data by missing value imputation and outlier filtering to obtain cleaned medical data; S12: The cleaned medical data is structured and mapped according to a unified medical coding standard to obtain a structured medical feature vector; S13: Based on the structured medical feature vector, construct labeled samples containing the correspondence between patient symptoms and diagnoses, and filter to obtain the initial training dataset.

[0007] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, step S2 further includes: S21: Represent the patient state in the initial training dataset as the root node in the Monte Carlo tree search framework, and sample the scalable action space of the current node through a small language model to obtain a set of candidate inference steps; S22: Based on the set of candidate inference steps, the confidence upper bound of multiple nodes is calculated using the UCT algorithm to obtain the node selection score corresponding to each step; S23: Expand the Monte Carlo tree search tree in descending order of node selection scores to obtain multiple candidate reasoning paths.

[0008] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, in step S21, when the small language model samples the scalable action space of the current node, it embeds clinical decision rules to verify the compliance of the sampling results, and assigns negative penalty scores to candidate reasoning steps that do not conform to the clinical decision rules.

[0009] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, step S3 further includes: S31: Decompose multiple candidate reasoning paths according to reasoning steps. For each reasoning state, calculate the Q value of the current step through the process reward model to obtain an instant scoring sequence of multiple steps. S32: Based on the instant scoring sequence, calculate the terminal reward value of each candidate inference path through an accumulation operation to obtain the path score of each candidate inference path; S33: Combine the path score with the immediate score sequence to obtain the step scores of multiple candidate reasoning paths.

[0010] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, step S4 further includes: S41: Based on the step score, extract the inference path with the highest Q value from multiple candidate inference paths as the positive sample path, and the inference path with the lowest Q value as the negative sample path, and construct a preference pair dataset. S42: Input the preference pair dataset into the initial process reward model, and optimize the parameters of the process reward model through the pairwise ranking loss function to obtain the optimized process reward model; S43: The optimization process reward model re-scores and sorts multiple candidate inference paths to obtain a set of high-quality inference paths.

[0011] According to the medical assisted reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, in step S42, the expression of the pairwise ranking loss function is: in, This is the set of all trainable parameters for the process reward model. For the process reward model in parameters The pairwise ranking loss function value is as follows. Indexed by the sequence number of the preference pair. It is the Sigmoid activation function. To input medical data, For positive sample paths, For negative sample paths, For parameters The output rating value.

[0012] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, step S5 further includes: S51: Using the set of high-quality inference paths as training data, the policy model is supervised and fine-tuned so that the policy model can generate new inference trajectories for similar medical data under the Monte Carlo tree search framework, thus obtaining an extended set of inference trajectories. S52: Based on the extended inference trajectory set, and according to the preference pair construction method, select high-scoring inference trajectories from the extended inference trajectory set as new training samples to obtain an updated training dataset; S53: Input the updated training dataset into the policy model and the process reward model respectively, and continuously optimize the model parameters of the policy model and the process reward model through backpropagation to obtain the optimized inference model.

[0013] According to the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided by the present invention, step S6 further includes: S61: Input the patient's current medical data into the optimized inference model, and use Monte Carlo tree search to locate the inference path with the highest terminal reward value in the inference tree to obtain the optimal inference path; S62: Based on the optimal reasoning path, extract the reasoning basis and confidence score of each reasoning node step by step to obtain the complete reasoning chain; S63: Combine the terminal diagnosis node corresponding to the optimal reasoning path with the complete reasoning chain and output it to obtain a comprehensive suggestion set that includes disease diagnosis, differential diagnosis and reasoning chain.

[0014] This invention also provides a medical auxiliary reasoning system based on a small language model and Monte Carlo tree search, used in a medical auxiliary reasoning method based on a small language model and Monte Carlo tree search as described in any of the above claims, comprising: Standardization module: Used to standardize the raw patient medical data to obtain the initial training dataset; An extension module, configured as a small language model, is used to receive the initial training dataset and search and expand the candidate diagnostic inference paths under the Monte Carlo tree search framework to obtain multiple candidate inference paths. The scoring module is configured as a process reward model to score the intermediate inference steps of multiple candidate inference paths, thereby obtaining step scores for multiple candidate inference paths. The filtering module is used to sort and filter multiple candidate reasoning paths based on step scores to obtain a high-quality set of reasoning paths. Training module: Used to input the set of high-quality inference paths into the policy model and the process reward model for joint training to obtain an optimized inference model; The reasoning module is configured with the optimized reasoning model obtained by the training module, and is used to generate reasoning results based on the optimal reasoning path to obtain a comprehensive set of suggestions.

[0015] This invention standardizes raw patient medical data and constructs an initial training dataset, ensuring that the input data relied upon by subsequent inference processes has a unified structured expression, thereby effectively reducing inference bias caused by inconsistent data formats. Secondly, within the Monte Carlo tree search framework, this invention utilizes the UCT algorithm to comprehensively calculate the cumulative reward value and access frequency of each inference node, enabling the search tree to dynamically balance exploration and utilization. The expansion direction of the inference path continuously converges towards high-confidence diagnostic conclusions, rather than relying on a single forward propagation to generate results. This results in stronger path diversity and fault tolerance in the inference process for complex cases and multi-system diseases. Furthermore, the process reward model of this invention refines the evaluation granularity of inference quality to each intermediate step through the real-time scoring of each inference step and the cumulative calculation of terminal rewards. Combined with the optimization mechanism of preference pair construction and pairwise ranking loss function, the model can clearly distinguish between high-quality and low-quality inference paths from the comparison of positive and negative samples, making the selection of inference steps more discriminative. Secondly, the joint training mechanism of the strategy model and the process reward model shares underlying feature encoding, reducing redundant computational overhead. Furthermore, the continuous optimization design of the training process allows the system to continuously expand high-quality training samples without relying on large amounts of manually labeled data, and the model's generalization ability gradually increases with each iteration. The final comprehensive diagnostic recommendation output includes a complete reasoning chain and confidence scores for each step, enabling clinicians to clearly trace the model's complete decision-making process from symptoms to diagnostic conclusions, significantly enhancing the interpretability and clinical credibility of the diagnostic results. Attached Figure Description

[0016] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0017] Figure 1 A schematic diagram of a medical auxiliary reasoning method based on a small language model and Monte Carlo tree search provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a medical auxiliary reasoning system based on a small language model and Monte Carlo tree search, provided as an embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0019] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.

[0020] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.

[0022] The embodiments of the present invention are described below with reference to the figures.

[0023] like Figure 1 As shown, this invention provides a medical assisted reasoning method based on a small language model and Monte Carlo tree search, comprising: S1: Standardize the patient's raw medical data to obtain the initial training dataset.

[0024] Step S1 further includes: S11: Extract raw medical data from the patient's symptom records, physical signs, laboratory test results, imaging data, and medical history information. Preprocess the raw medical data by filling in missing values ​​and filtering outliers to obtain cleaned medical data.

[0025] This invention first extracts raw medical data from patients' symptom records, physical signs, laboratory test results, imaging data, and medical history. Since this data originates from different clinical recording systems and has varying formats, this invention preprocesses the collected data in step S11. Specifically, for missing fields in the raw medical data, this invention uses mean imputation or interpolation based on similar patient data to fill in missing values; for numerical records that significantly deviate from the normal medical range, outlier filtering is performed by setting a threshold range, removing data exceeding the threshold, ultimately obtaining cleaned medical data.

[0026] S12: The cleaned medical data is structured and mapped according to a unified medical coding standard to obtain a structured medical feature vector.

[0027] In step S12, the present invention aims to perform structured mapping of the cleaned medical data according to a unified medical coding standard. Specifically, symptom descriptions are mapped to standardized disease codes according to the ICD coding system, laboratory test results are mapped to standardized test item codes according to the LOINC coding system, and descriptive text of imaging data is extracted with key lesion terms and mapped to corresponding imaging diagnostic labels. After the above mapping process, each patient record is transformed into a structured medical feature vector composed of multi-dimensional standardized codes, where each dimension corresponds to a specific type of clinical information.

[0028] S13: Based on the structured medical feature vector, construct labeled samples containing the correspondence between patient symptoms and diagnoses, and filter to obtain the initial training dataset.

[0029] After obtaining the structured medical feature vectors, this invention pairs the structured medical feature vectors with the corresponding diagnostic labels based on the final clinical diagnosis result corresponding to each patient record, constructing labeled samples containing the correspondence between patient symptoms and diagnoses. For samples with ambiguous labels or incomplete diagnostic records, this invention manually reviews and removes unqualified samples one by one, thereby obtaining the initial training dataset.

[0030] S2: Input the initial training dataset into the small language model, and expand the candidate diagnostic reasoning path under the Monte Carlo tree search framework to obtain multiple candidate reasoning paths.

[0031] Step S2 further includes: S21: Represent the patient states in the initial training dataset as the root node in the Monte Carlo tree search framework, and sample the scalable action space of the current node through a small language model to obtain a set of candidate inference steps.

[0032] In step S21, when the small language model samples the scalable action space of the current node, it embeds clinical decision rules to verify the compliance of the sampling results and assigns negative penalty scores to candidate reasoning steps that do not conform to the clinical decision rules.

[0033] In step S21, the present invention uses the structured medical feature vector of each patient record in the initial training dataset as the initial input, and represents the current state of the corresponding patient as the root node in the Monte Carlo tree search framework. The obtained root node carries the complete medical feature information of the patient. The Monte Carlo tree search starts from the root node and gradually expands the inference steps downward.

[0034] Furthermore, at each current node, this invention invokes a small language model to sample the scalable action space corresponding to that node. The scalable action space refers to the set of next logically feasible reasoning actions that the model deems feasible in the current patient state, such as further inquiring about a certain physical sign, inferring an intermediate diagnostic conclusion, or excluding a certain disease category. Subsequently, based on the patient state characteristics of the current node, the small language model samples from the aforementioned action space to generate several candidate reasoning steps, forming a candidate reasoning step set.

[0035] During the sampling process, this invention simultaneously embeds clinical decision rules to verify the compliance of the sampling results. Clinical decision rules are a predefined set of medical constraints, such as the types of diseases that must be investigated under specific symptom combinations, and logical constraints between examination results and diagnoses. For candidate reasoning steps that do not conform to the aforementioned clinical decision rules, this invention directly assigns them negative penalty scores, placing them at a disadvantage in subsequent node selections.

[0036] S22: Based on the set of candidate inference steps, the upper confidence bound of multiple nodes is calculated using the UCT algorithm to obtain the node selection score corresponding to each step.

[0037] Specifically, the expression for the node selection branch calculated in step S22 is: in, For the inference step nodes in the Monte Carlo tree search framework, For nodes Choose the confidence upper bound score. For nodes The cumulative reward value, For nodes Total number of times accessed during historical search processes For nodes The total number of times the parent node was visited during the historical search process. To balance the hyperparameters of exploration and utilization, a larger c value indicates that the search process tends to visit nodes with fewer historical visits, while a smaller c value indicates that the search process tends to visit nodes with higher historical cumulative reward values.

[0038] S23: Expand the Monte Carlo tree search tree in descending order of node selection scores to obtain multiple candidate reasoning paths.

[0039] In steps S22 to S23, the present invention calculates the node selection score for each node in the candidate reasoning step set using the UCT algorithm. The UCT algorithm comprehensively considers the historical cumulative reward value of each node and the frequency of the node being visited, and gives higher selection weight to nodes with fewer visits but higher historical scores, thereby achieving a balance between utilizing known high-quality paths and exploring unknown paths.

[0040] After calculating the score, the present invention expands the Monte Carlo tree in descending order of node selection scores, selecting the node with the highest score as the starting point for the next reasoning step, and continues to expand until the preset reasoning depth is reached or the termination condition is triggered, thus obtaining multiple candidate reasoning paths extending from the root node to the terminal diagnostic node.

[0041] S3: Input multiple candidate reasoning paths into the process reward model, score the intermediate reasoning steps of the multiple candidate reasoning paths, and obtain the step scores of the multiple candidate reasoning paths.

[0042] Step S3 further includes: S31: Divide multiple candidate reasoning paths into reasoning steps. For each reasoning state, calculate the Q value of the current step using the process reward model to obtain an instant scoring sequence for multiple steps.

[0043] In step S31, the present invention breaks down each candidate reasoning path sequentially according to the order of reasoning steps, obtaining a state sequence with each reasoning step as a unit. For each reasoning state, the present invention inputs the patient feature information corresponding to the state along with the record of executed reasoning steps into the process reward model. The process reward model calculates the Q value of the reasoning step, which reflects the contribution of the current reasoning step to the final correct diagnosis in the entire reasoning path. A higher Q value indicates that the reasoning step is closer to the correct diagnostic direction. After calculating each reasoning step, the instant scoring sequence corresponding to each candidate reasoning path is finally obtained.

[0044] S32: Calculate the terminal reward value of each candidate inference path by accumulating the instant scoring sequence to obtain the path score of each candidate inference path; S33: Combine the path score with the instant scoring sequence to obtain the step score of multiple candidate inference paths.

[0045] Specifically, the calculated path score expression is as follows: in, This is the index of the reasoning steps within a single candidate reasoning path, where n represents the total number of reasoning steps in that path. For the j-th reasoning step node, For the first Each reasoning step node The instantaneous Q value, The overall path score for this candidate reasoning path is calculated from the initial patient state node to the final diagnosis node.

[0046] In steps S32 and S33, this invention accumulates the Q-values ​​of each step in the instant scoring sequence of each candidate inference path to obtain the terminal reward value of that path, i.e., the path score. The path score represents the overall quality of the entire inference path from the initial patient state to the terminal diagnostic conclusion. After obtaining the path score, this invention merges the path score with the corresponding instant scoring sequence to form a complete step score covering the overall path quality and the quality of each intermediate step, for use in subsequent path selection and model training.

[0047] S4: Based on the step score, sort and filter multiple candidate reasoning paths to obtain a high-quality reasoning path set.

[0048] Step S4 further includes: S41: Based on the step score, extract the inference path with the highest Q value from multiple candidate inference paths as the positive sample path, and the inference path with the lowest Q value as the negative sample path, and construct a preference pair dataset.

[0049] In step S41, after obtaining the step scores of multiple candidate inference paths, the present invention sorts all candidate inference paths according to the Q-value of each inference path. Specifically, the present invention extracts the two inference paths with the highest Q-values ​​from the sorting results as positive sample paths, and extracts the two inference paths with the lowest Q-values ​​as negative sample paths. The positive sample paths and negative sample paths are paired up to construct a preference pair dataset.

[0050] For the obtained preference pair dataset, each record consists of a positive sample path and a negative sample path, which share the same input medical data and differ only in the quality of inference. The pairing structure allows the model to directly obtain supervision signals from the quality difference between positive and negative samples during subsequent training.

[0051] S42: Input the preference pair dataset into the initial process reward model, and optimize the parameters of the process reward model through the pairwise ranking loss function to obtain the optimized process reward model.

[0052] In step S42, the expression for the pairwise ranking loss function is: in, This is the set of all trainable parameters for the process reward model. For the process reward model in parameters The pairwise ranking loss function value is as follows. Indexed by the sequence number of the preference pair. It is the Sigmoid activation function. To input medical data, For positive sample paths, For negative sample paths, For parameters The output rating value.

[0053] In step S42, the present invention inputs the preference dataset into the initial process reward model. The initial process reward model receives an inference path and the corresponding input medical data, and outputs a scalar score value, which reflects the overall quality of the inference path.

[0054] For the initial process reward model, this invention optimizes all trainable parameters of the initial process reward model using a pairwise ranking loss function. The core logic of the pairwise ranking loss function is that, for each pair of positive and negative samples in the preference pair dataset, the difference between the score of the initial process reward model for the positive sample path and the score of the negative sample path is calculated. This difference is then input into the Sigmoid activation function, which maps the difference to a probability value between 0 and 1. The larger the difference, the closer the probability value is to 1, indicating that the model distinguishes between positive and negative samples more accurately.

[0055] When training the initial process reward model, this invention takes the logarithm of the above probability values ​​for all preference pairs and sums them, then inverts the negative sign to use as the loss function value. The parameters of the initial process reward model are then updated by backpropagation through gradient descent until the loss function converges, thus obtaining the optimized process reward model.

[0056] S43: The optimization process reward model re-scores and sorts multiple candidate inference paths to obtain a set of high-quality inference paths.

[0057] After obtaining the optimization process reward model in step S42, the present invention re-inputs all multiple candidate inference paths into the optimization process reward model. The optimization process reward model outputs a new score value for each candidate inference path and re-sorts them according to the score value from high to low. The inference paths with the highest ranking are selected to obtain a set of high-quality inference paths.

[0058] S5: Input the set of high-quality reasoning paths into the policy model and the process reward model for joint training to obtain an optimized reasoning model.

[0059] Step S5 further includes: S51: Using the set of high-quality inference paths as training data, the policy model is supervised and fine-tuned so that the policy model can generate new inference trajectories for similar medical data under the Monte Carlo tree search framework, thus obtaining an extended set of inference trajectories.

[0060] Furthermore, in step S51, the present invention uses a set of high-quality reasoning paths as supervised training data and inputs it into the policy model for supervised fine-tuning. The policy model is a small language model that takes on the role of generating reasoning steps within the framework of the present invention. It receives patient state features as input and outputs the probability distribution of the next reasoning action.

[0061] Specifically, when supervising and fine-tuning the strategy model, this invention uses each inference step of each path in the set of high-quality inference paths as the target output and the corresponding patient state features as the input. It calculates the deviation between the current output and the target output of the strategy model through cross-entropy loss and updates the parameters of the strategy model through backpropagation, so that the strategy model tends to generate inference trajectories similar to the structure of high-quality inference paths when facing similar medical data.

[0062] After completing the supervised fine-tuning, the driving strategy model of this invention re-executes multiple search expansions on similar medical data under the Monte Carlo tree search framework, generating a batch of new inference trajectories and obtaining an expanded inference trajectory set.

[0063] S52: Based on the extended inference trajectory set, and according to the preference pair construction method, select high-scoring inference trajectories from the extended inference trajectory set as new training samples to obtain an updated training dataset.

[0064] Subsequently, the present invention scores and filters each inference trajectory in the extended inference trajectory set according to the preference construction method described in step S4. That is, the optimization process reward model scores each trajectory in the extended inference trajectory set, extracts high-scoring inference trajectories as new training samples, and merges them with the original high-quality inference path set to obtain an updated training dataset.

[0065] S53: Input the updated training dataset into the policy model and the process reward model respectively, and continuously optimize the model parameters of the policy model and the process reward model through backpropagation to obtain the optimized inference model.

[0066] In step S53, the present invention inputs the updated training dataset into the policy model and the process reward model respectively, and performs backpropagation updates on the parameters of both models simultaneously. The policy model uses the updated inference step sequence in the training dataset as the supervision objective, while the process reward model uses the reconstructed preference pairs in the updated training dataset as the training input. The parameter updates of both are performed alternately within the same training round. The present invention continuously optimizes the above joint parameter update process. After each round of optimization, it checks whether the average score of the inference path generated by the policy model on the validation set has improved. If the improvement is lower than a preset threshold, the optimization is terminated, and an optimized inference model is obtained.

[0067] S6: The optimized reasoning model generates reasoning results based on the optimal reasoning path to obtain a comprehensive set of suggestions.

[0068] Step S6 further includes: S61: Input the patient's current medical data into the optimized inference model, and use Monte Carlo tree search to locate the inference path with the highest terminal reward value in the inference tree to obtain the optimal inference path.

[0069] In the inference application phase, this invention first inputs the patient's current medical data into an optimized inference model. The optimized inference model then drives a Monte Carlo tree search to traverse the inference tree. Specifically, each leaf node in the inference tree corresponds to a complete inference path extending from the patient's initial state to a specific diagnostic conclusion, and each leaf node stores the terminal reward value for that path. After traversing all leaf nodes, this invention locates the inference path corresponding to the leaf node with the highest terminal reward value, thus obtaining the optimal inference path.

[0070] S62: Based on the optimal reasoning path, extract the reasoning basis and confidence score of each reasoning node step by step to obtain the complete reasoning chain.

[0071] In step S62, the present invention extracts the reasoning basis and confidence score node by node along the optimal reasoning path from the root node to the terminal diagnosis node. Each reasoning node stores the description of the reasoning action corresponding to the node, the patient feature information on which the action is triggered, and the score value output by the optimization process reward model for that node. This score value is the confidence score for that step. Finally, the present invention concatenates the reasoning basis and confidence score of each node in the order from the root node to the terminal node to obtain a complete reasoning chain.

[0072] S63: Combine the terminal diagnosis node corresponding to the optimal reasoning path with the complete reasoning chain and output it to obtain a comprehensive suggestion set that includes disease diagnosis, differential diagnosis and reasoning chain.

[0073] After obtaining the optimal reasoning path and the complete reasoning chain, this invention extracts the main diagnostic conclusions corresponding to the terminal diagnostic nodes of the optimal reasoning path, and combines them with the diagnostic conclusions pointed to by other reasoning paths with the second highest scores in the reasoning tree as differential diagnoses. The main diagnostic conclusions, differential diagnoses and the complete reasoning chain are merged and output to obtain a comprehensive set of suggestions that includes disease diagnosis, differential diagnosis and reasoning chain.

[0074] like Figure 2 As shown, the present invention also provides a medical auxiliary reasoning system based on a small language model and Monte Carlo tree search, comprising: Standardization Module 100: Used to standardize the patient's raw medical data to obtain the initial training dataset; The extension module 200 is configured as a small language model to receive the initial training dataset and search and expand the candidate diagnostic reasoning paths under the Monte Carlo tree search framework to obtain multiple candidate reasoning paths. The scoring module 300 is configured as a process reward model to score the intermediate reasoning steps of multiple candidate reasoning paths and obtain the step scores of multiple candidate reasoning paths. Filtering module 400: Used to sort and filter multiple candidate reasoning paths based on step scores to obtain a high-quality set of reasoning paths; Training module 500: Used to input the set of high-quality inference paths into the policy model and the process reward model for joint training to obtain an optimized inference model; The reasoning module 600 is configured with the optimized reasoning model obtained by the training module 500, and is used to generate reasoning results based on the optimal reasoning path to obtain a comprehensive suggestion set.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0077] In practice, this invention is applicable to clinical scenarios requiring multi-step reasoning analysis of patient medical data to output interpretable diagnostic suggestions. In the field of internal medicine diagnosis, this invention receives patient symptom records, physical signs, laboratory test results, and medical history information, and uses a Monte Carlo tree search framework for step-by-step reasoning. It is suitable for the auxiliary diagnosis of multi-system diseases such as respiratory diseases, cardiovascular diseases, endocrine and metabolic diseases, and nervous system diseases. It is particularly suitable for cases with overlapping symptoms and complex differential diagnoses, where single-step reasoning methods are prone to diagnostic bias. The multi-step reasoning mechanism of this invention can narrow down the diagnostic scope layer by layer.

[0078] In the field of image diagnosis, this invention receives descriptive text or image feature extraction results of image data as input, and combines them with the patient's clinical symptoms and examination data to generate a reasoning path from image features to lesion type under the Monte Carlo tree search framework. It is suitable for assisting radiologists in identifying and qualitatively analyzing lesion areas in X-ray, CT and MRI images.

[0079] In the field of emergency decision support, this invention is applicable to rapid risk assessment scenarios for sudden acute illnesses such as acute myocardial infarction, stroke, and acute respiratory failure. When applied to emergency decision support, this invention receives real-time vital sign data and chief complaints from emergency patients, completes multi-step reasoning in a short time through an optimized reasoning model, and outputs diagnostic suggestions and a complete reasoning chain including confidence scores, providing decision-making references for emergency physicians.

[0080] In the field of telemedicine, this invention is based on a small language model, which has a small number of model parameters and low computational resource requirements. It is suitable for local deployment in primary hospitals, community health service centers and medical institutions in remote areas. It can complete inference tasks without relying on high-performance servers, providing intelligent auxiliary diagnosis support for the above-mentioned medical institutions.

[0081] In the field of medical education and training, the complete reasoning chain output by this invention fully records the reasoning basis and confidence score of each step from patient symptoms to final diagnosis. It can be used as a reference case for medical students and resident physicians to learn clinical reasoning thinking and to demonstrate a standardized multi-step diagnostic reasoning process.

[0082] It should be noted that this invention is a data processing method based on a small language model and Monte Carlo tree search. Its operation targets collected medical data such as patient symptom records, physical signs, laboratory test results, imaging data, and medical history information. All processing steps of this invention are executed within a computer system on the aforementioned medical data, including standardization of the raw medical data, search expansion of candidate inference paths within the Monte Carlo tree search framework, scoring calculation of inference steps using a process reward model, and joint training and parameter updating of the policy model and the process reward model. All of these steps fall within the scope of computer data processing. The comprehensive suggestion set ultimately output by this invention is provided to clinicians as supplementary reference information. The final diagnostic conclusion is made by the clinician based on the actual situation; this invention itself does not directly make a legally significant clinical diagnostic conclusion.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A medical auxiliary reasoning method based on a small language model and Monte Carlo tree search, characterized in that, include: S1: Standardize the patient's raw medical data to obtain the initial training dataset; S2: Input the initial training dataset into the small language model, and expand the candidate diagnostic reasoning path under the Monte Carlo tree search framework to obtain multiple candidate reasoning paths; S3: Input multiple candidate reasoning paths into the process reward model, score the intermediate reasoning steps of the multiple candidate reasoning paths, and obtain the step scores of the multiple candidate reasoning paths; S4: Based on the step scores, sort and filter multiple candidate reasoning paths to obtain a high-quality reasoning path set; S5: Input the set of high-quality reasoning paths into the policy model and the process reward model for joint training to obtain an optimized reasoning model; S6: The optimized reasoning model generates reasoning results based on the optimal reasoning path to obtain a comprehensive set of suggestions.

2. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 1, characterized in that, Step S1 further includes: S11: Extract raw medical data from the patient's symptom records, physical signs data, laboratory test results, imaging data and medical history information, and preprocess the raw medical data by missing value imputation and outlier filtering to obtain cleaned medical data; S12: The cleaned medical data is structured and mapped according to a unified medical coding standard to obtain a structured medical feature vector; S13: Based on the structured medical feature vector, construct labeled samples containing the correspondence between patient symptoms and diagnoses, and filter to obtain the initial training dataset.

3. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 1, characterized in that, Step S2 further includes: S21: Represent the patient state in the initial training dataset as the root node in the Monte Carlo tree search framework, and sample the scalable action space of the current node through a small language model to obtain a set of candidate inference steps; S22: Based on the set of candidate inference steps, the confidence upper bound of multiple nodes is calculated using the UCT algorithm to obtain the node selection score corresponding to each step; S23: Expand the Monte Carlo tree search tree in descending order of node selection scores to obtain multiple candidate reasoning paths.

4. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 3, characterized in that, In step S21, when the small language model samples the scalable action space of the current node, it embeds clinical decision rules to verify the compliance of the sampling results and assigns negative penalty scores to candidate reasoning steps that do not conform to the clinical decision rules.

5. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 1, characterized in that, Step S3 further includes: S31: Decompose multiple candidate reasoning paths according to reasoning steps. For each reasoning state, calculate the Q value of the current step through the process reward model to obtain an instant scoring sequence of multiple steps. S32: Based on the instant scoring sequence, calculate the terminal reward value of each candidate inference path through an accumulation operation to obtain the path score of each candidate inference path; S33: Combine the path score with the immediate score sequence to obtain the step scores of multiple candidate reasoning paths.

6. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 1, characterized in that, Step S4 further includes: S41: Based on the step score, extract the inference path with the highest Q value from multiple candidate inference paths as the positive sample path, and the inference path with the lowest Q value as the negative sample path, and construct a preference pair dataset. S42: Input the preference pair dataset into the initial process reward model, and optimize the parameters of the process reward model through the pairwise ranking loss function to obtain the optimized process reward model; S43: The optimization process reward model re-scores and sorts multiple candidate inference paths to obtain a set of high-quality inference paths.

7. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 6, characterized in that, In step S42, the expression for the pairwise ranking loss function is: in, This is the set of all trainable parameters for the process reward model. For the process reward model in parameters The pairwise ranking loss function value is below. Indexed by the sequence number of the preference pair. It is the Sigmoid activation function. To input medical data, For positive sample paths, For negative sample paths, For parameters The output rating value.

8. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 6, characterized in that, Step S5 further includes: S51: Using the set of high-quality inference paths as training data, the policy model is supervised and fine-tuned so that the policy model can generate new inference trajectories for similar medical data under the Monte Carlo tree search framework, thus obtaining an extended set of inference trajectories. S52: Based on the extended inference trajectory set, and according to the preference pair construction method, select high-scoring inference trajectories from the extended inference trajectory set as new training samples to obtain an updated training dataset; S53: Input the updated training dataset into the policy model and the process reward model respectively, and continuously optimize the model parameters of the policy model and the process reward model through backpropagation to obtain the optimized inference model.

9. The medical auxiliary reasoning method based on a small language model and Monte Carlo tree search according to claim 1, characterized in that, Step S6 further includes: S61: Input the patient's current medical data into the optimized inference model, and use Monte Carlo tree search to locate the inference path with the highest terminal reward value in the inference tree to obtain the optimal inference path; S62: Based on the optimal reasoning path, extract the reasoning basis and confidence score of each reasoning node step by step to obtain the complete reasoning chain; S63: Combine the terminal diagnosis node corresponding to the optimal reasoning path with the complete reasoning chain and output it to obtain a comprehensive suggestion set that includes disease diagnosis, differential diagnosis and reasoning chain.

10. A medical auxiliary reasoning system based on a small language model and Monte Carlo tree search, used in the medical auxiliary reasoning method based on a small language model and Monte Carlo tree search as described in any one of claims 1 to 9, characterized in that, include: Standardization module: Used to standardize the raw patient medical data to obtain the initial training dataset; An extension module, configured as a small language model, is used to receive the initial training dataset and search and expand the candidate diagnostic inference paths under the Monte Carlo tree search framework to obtain multiple candidate inference paths. The scoring module is configured as a process reward model to score the intermediate inference steps of multiple candidate inference paths, thereby obtaining step scores for multiple candidate inference paths. The filtering module is used to sort and filter multiple candidate reasoning paths based on step scores to obtain a high-quality set of reasoning paths. Training module: Used to input the set of high-quality inference paths into the policy model and the process reward model for joint training to obtain an optimized inference model; The reasoning module is configured with the optimized reasoning model obtained by the training module, and is used to generate reasoning results based on the optimal reasoning path to obtain a comprehensive set of suggestions.

Images