A medical document-oriented personalized collaborative input method based on probability correction

Abstract

The present application relates to the technical field of natural language processing and artificial intelligence, in particular to a personalized collaborative input method based on probability correction for medical documents, the process is as follows, S1, offline vector database construction; S2, online inference feature extraction; S3, double function and retrieval of hidden layer state; S4, retrieval distribution stabilization strategy; S5, mixed probability generation. In the present application, the posterior probability of the adaptive model on the mixed data set can be effectively approximated by correcting the output probability of the pre-training model during inference without retraining, and accordingly the correction algorithm and various distribution stabilization strategies (ranking calibration, time momentum, frequency scaling) are realized to ensure the continuity and fluency of the generation process, and finally a set of fine-grained guided generation method without training is obtained, which has significant effect in various scenes such as medical document writing, and can realize word / sentence accurate association recommendation during large medical record writing, greatly reducing the burden of physician document writing.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing (NLP) and artificial intelligence, and in particular to a probability-based personalized collaborative data entry method for medical documents. Background Technology

[0002] In recent years, large-scale language models such as GPT, Llama, and Qwen have demonstrated powerful general-purpose generation capabilities in zero-shot and few-shot tasks. However, applying these general-purpose models to specific professional fields (such as medical record writing, legal document drafting, and code generation) or to meet users' personalized writing styles still faces significant challenges. Especially in clinical medical scenarios, physicians routinely need to complete a large number of standardized medical documents, such as large medical records, initial progress notes, specialist examination records, and discharge summaries. Existing auxiliary input tools can only achieve fixed phrase association and whole-sentence text generation, which cannot conform to the specialist terminology and diagnostic logic of different departments, nor can they adapt to the personalized writing habits of different physicians. Furthermore, they cannot provide continuous and accurate fine-grained guidance during the word-by-word writing process, resulting in physicians having to spend a lot of non-clinical time on text deletion and correction, leading to a heavy clinical workload.

[0003] In order to overcome the problems of applying large language models in the medical field, existing technologies have been improved. The mainstream technical routes include the following: (1) Fine-tuning Tuning: Gradient updates of the model using domain data can achieve excellent domain adaptive performance; however, this method requires a large amount of high-quality labeled data and expensive GPU computing resources, and each update requires complete training, making it difficult to quickly adapt to dynamically changing personal writing habits or newly emerging domain knowledge, such as updates to clinical diagnosis and treatment guidelines and changes in physicians' medical record writing habits. In addition, the fine-tuning method can only create a model in general and cannot customize a model for each department's doctors or patients. (2) Retrieval Enhancement Generation (RAG): Relevant document fragments are spliced ​​into the input prompts to guide the model to generate. RAG does not require training, but this method only provides a one-time context injection at the beginning of generation, which is a coarse-grained initial condition adjustment; however, as the generation length increases, the influence of the initial prompts gradually decays (i.e., "prompt decay" or "context drift"), and it is impossible to continuously and accurately constrain the generation of each subsequent token, causing the generated content to deviate from the user's intention, especially in the writing of large medical records, which is prone to deviating from clinical norms and still requires a lot of manual editing. (3) Token-level methods (such as kNN) LM) achieves token granularity control by fusing the retrieved token distribution through linear interpolation; however, recent studies have shown that this type of method only achieves probability improvement on sparse high-frequency tokens, distorts the overall generation distribution, and is prone to generating repetitive and non-fluent text, which is not suitable for generating long texts in open domains and cannot meet the professional requirements of medical documents. (4) Stream Collaborative Writing based Co (Writing): In the coding domain, this method has achieved success with tools like Copilot, providing real-time line-by-line completion and greatly improving development efficiency. However, in non-coding domains such as medicine, it still primarily generates entire paragraphs, which cannot adapt to the word-by-word input habits of physicians in large medical records, resulting in low efficiency. In summary, existing technologies lack a large-scale model inference technology that requires no training, can provide fine-grained and stable guidance for each token throughout the generation process, and is suitable for personalized collaborative writing methods in medical document writing scenarios. Summary of the Invention

[0004] To overcome the shortcomings of existing large-scale language models and their improvements in the medical field due to technological limitations, as described in the background section, this invention provides a method for personalized collaborative input of medical documents based on probability correction. This method, based on a large-scale language model and with the combined effect of related technical solutions, perfectly integrates the general language capabilities of the large-scale model with specific domain / personal habits. It can be adapted to clinical writing scenarios. When physicians input single characters / words, it can generate content recommendations that fit personal habits and specialty standards in real time, greatly reducing the distance of modification and editing and the number of keyboard clicks. This method enables efficient collaborative writing of medical documents.

[0005] The technical solution adopted by this invention to solve its technical problem is: A probability-based personalized collaborative data entry method for medical documents includes the following steps: S1, offline vector database construction: S1. Supplementary personalized datasets are input into the basic large language model for offline forward computation, extracting the hidden state vector corresponding to each token position. This vector is used as the feature representation of the context and directly stored in an external vector database. S2. Online inference feature extraction: During the online inference process of streaming collaborative writing, in each autoregressive token generation step, the current complete input sequence is sent to the core processing layer of the large language model. The model outputs the context hidden layer state of the current generation step through forward computation. S3. Dual function and retrieval of hidden layer states: For data generated during the typical autoregressive generation process, only one copy of the calculation result is needed; no additional computation is required. Similarity calculation uses the corresponding norm, i.e. S4. Retrieval Distribution Stabilization Strategy: Perform a three-step joint refinement optimization on the retrieved candidates, including ranking-based calibration, historical momentum integration, and frequency scaling. S5. Mixed Probability Generation: According to the derived Bayesian formula, merge the external probability with the original posterior probability of the base model to form a mixed posterior distribution, i.e. Then, use greedy decoding to sample the next token.

[0006] Furthermore, in step S1, the specific calculation process is as follows: traverse the newly added dataset. ,for any sequence of documents It is then input into the basic large language model for forward computation, for each time step The model will output the hidden layer state vector corresponding to that position. Build key-value pairs For direct storage into an external vector database, the entire process only requires traversing the dataset once, without needing to perform any backpropagation or gradient updates on the model.

[0007] Furthermore, in step S3, the hidden layer state has the following functions: (1) transforming the model to the lexical space through the linear projection layer of the model to form the original posterior probability distribution of the basic model; (2) using the similarity function to retrieve the semantically most relevant context from the constructed external vector database as a query vector.

[0008] Furthermore, in step S3, the model's own calculation results are reused, and the basic generation probability and retrieval suggestions for specific domains are obtained simultaneously without the need to introduce an additional encoder network, thus achieving fine-grained control at the token level throughout the generation process.

[0009] Furthermore, in step S4, the ranking-based calibration process is as follows: candidate items are sorted in descending order of similarity and mapped to a preset sparse template distribution, directly discarding long-tail irrelevant terms. Specifically, a monotonically non-decreasing template formula is introduced. This will retrieve all similarities from the previous step. Sort in descending order and assign the corresponding values ​​to the template. The historical momentum integration process is as follows: when calculating the weight of the current step, the smoothed historical weight of the previous time step is introduced to maintain the inertia of sequence generation. ,in The momentum decay coefficient, This represents the preceding state in the historical trajectory. The frequency scaling process is as follows: a penalty term is introduced to explicitly reduce the probability weight of globally high-frequency words. Specifically, the probability weight of each word is scaled using the following formula. ,in For this word in the dataset The global occurrence count in The severity of the punishment.

[0010] Furthermore, in step S4, historical momentum integration is preferably performed in practice. In frequency scaling, the preferred method is... ; Furthermore, in step S4, by aggregating using a method inherited from kernel density estimation, the probability density can be estimated, thereby generating the final target-specific guided distribution.

[0011] Compared with existing technologies, the beneficial effects of this invention are as follows: The posterior probability of the adaptive model on a mixed dataset (pre-trained data + personalized data) can be effectively approximated by correcting the output probability of the pre-trained model during inference, without retraining. Based on this, a correction algorithm and various distribution stabilization strategies (ranking calibration, time momentum, frequency scaling) are implemented to ensure the continuity and smoothness of the generation process. This results in a fine-grained guided generation method that requires no training, demonstrating significant effectiveness in various scenarios such as medical document writing. It enables precise word-by-word / sentence-by-sentence association recommendations during the writing of large medical records, significantly reducing the burden of document writing for physicians. In summary, this invention has promising application prospects. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the process method of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] The core theoretical foundation of the algorithm for achieving the invention's objective, based on a large-scale language model, is as follows: Given a vocabulary... Qwen3 series pre-trained models Let the pre-training dataset behind this model be denoted as . Add a new dataset The pre-trained dataset As a core asset of pre-trained dataset manufacturers, it is not publicly available due to data privacy and copyright restrictions, and is usually unavailable; new datasets are added. To provide personalized supplementary datasets for medical scenarios, such as anonymized clinical medical records and examination reports, the hybrid dataset would be... The posterior probability on it can be expressed as p represents probability, the subscript indicates the order in which tokens are generated in the sequence, and the separator is... This formula, used to represent conditional probability, has a given condition on its right side. It expresses the true probability of generating the next target token given a historical sequence, under ideal conditions that integrate basic general knowledge and personalized domain knowledge. This is also the theoretical optimization objective of this invention. To simplify the notation and make the subsequent derivation of the formula based on Bayes' theorem more intuitive and clear, let's denote it as... To further simplify, the given sequence is: The target token is , According to Bayes' theorem, it can be expressed by the following formula. ,in, can be Approximate calculation, Normalization can be used to eliminate it. include and It can be calculated based on the autoregressive principle, and After autoregressive expansion, it can be used This means that we have obtained the result, so we only need to calculate... To this end, the empirical distribution is used for calculation, i.e. ,in The frequency is estimated empirically. For vocabulary The k-th token in the array, This is the indicator function; furthermore, it is processed using a method similar to kernel density estimation, i.e. ,in It is a similarity function that measures the current input context. Historical context in external datasets The semantic distance between them is used as a weighting term in probability estimation, thereby effectively alleviating the data sparsity problem in the high-dimensional token sequence space. Norm, i.e. , This indicates the use of the L2 norm. Specifically, compared to the commonly used cosine similarity, the L2 norm distance can capture both the magnitude and direction features of a vector. This section aims to explain the overall technical framework and principles, not the independent execution steps. The specific implementation process is described below.

[0015] Figure 1As shown, a personalized collaborative data entry method based on probability correction for medical documents includes the following steps. Step 1: Construction of an offline vector database. Specifically, supplementary personalized datasets (such as compliant anonymized medical records) are input into the basic large language model for offline forward computation (pre-fill stage). The hidden state vector corresponding to each token position is extracted and then used as the feature representation of the context, directly stored in the external vector database. The specific calculation process is as follows: traverse the newly added dataset... ,for any sequence of documents It is then input into the basic large language model for forward computation, for each time step The model will output the hidden layer state vector corresponding to that position. Build key-value pairs The data is directly stored in an external vector database; the entire process only requires traversing the dataset once, without any backpropagation or gradient updates to the model. This step efficiently transforms external text into a vector form that is completely and naturally aligned with the internal representation space of the base model, greatly reducing computational costs and supporting low-cost, rapid updates to knowledge databases in fields such as medicine.

[0016] Figure 1 As shown, Step 2: Online inference feature extraction. Specifically, in the online inference process of streaming collaborative writing, in each autoregressive token generation step, the current complete input sequence is sent to the core processing layer of the large language model. The model outputs the hidden state of the current generation step through forward computation (forward computation is the existing technology). Step 3: Dual function of hidden state and retrieval step. The hidden state has dual functions: (1) Transforms the model to the lexical space through the linear projection layer to form the original posterior probability distribution of the basic model (shown as a 1 distribution in the figure); (2) At the same time, it is used as a query vector to retrieve the most semantically relevant context from the external vector database constructed in Step 2 using the similarity function. This step also needs to be calculated in the usual model autoregressive generation process, so only one copy of the calculation result is needed, and no additional calculation is required. The similarity calculation uses the aforementioned L2 norm, i.e. This step fully reuses the model's own calculation results, and obtains the basic generation probability and retrieval suggestions for specific domains without introducing an additional encoder network. This breaks through the limitation that traditional RAGs can only be guided in the initial prompt, and realizes fine-grained control at the token level throughout the generation process.

[0017] Figure 1As shown, step four: retrieval distribution stabilization strategy. Specifically, the original retrieval results are prone to noise and are insufficient to achieve stable guidance. Therefore, the retrieved candidate items need to be jointly refined and optimized in the following three steps. (1) Ranking-based calibration: Arrange the candidate items in descending order of similarity and map them to a preset sparse template distribution. Directly discard long-tail irrelevant items. Specifically, introduce a monotonically non-decreasing template. This will retrieve all similarities from the previous step. Sort in descending order and assign the corresponding values ​​to the template. (2) Historical momentum integration: When calculating the weights of the current step, smoothed historical weights from the previous time step are introduced to maintain the inertia of sequence generation, i.e. ,in, The momentum decay coefficient, As a previous state in the historical trajectory, it is recommended to set it in actual implementation. (3) Frequency scaling: Introduce a penalty term to explicitly reduce the probability weight of high-frequency words (such as meaningless common function words or punctuation marks). Specifically, for words, their probability weights are scaled, and the formula is as follows: ,in, For this word in the dataset The global occurrence count in To determine the severity of the penalty, it is recommended to set the appropriate level in practice. The refined signals mentioned above are aggregated using a method inherited from kernel density estimation (KDE) to estimate the probability density, thereby generating the final target-specific guided distribution (shown as a binary distribution in the figure). This step significantly eliminates the variance noise caused by sparse retrieval data, breaks the degradation phenomenon that external retrieval often leads to the model falling into a "local repetitive dead loop," and prompts the model to focus on richer professional patterns in the text, ensuring the coherence and fluency of the generation of professional fields such as long medical documents. Step 5: Hybrid probability generation, specifically, according to the Bayesian formula derived above, the external probability is merged with the original posterior probability of the base model to form a hybrid posterior distribution, i.e. Then, greedy decoding is used to sample the next token. The entire technology is designed so that it shares the same external interface as a typical language model during operation, allowing for seamless replacement.

[0018] Figure 1As shown, through all the above technical solutions, this invention provides the posterior probability of an ideal domain-adaptive model on a hybrid dataset (pre-trained data + personalized data). This probability can be effectively approximated by correcting the output probability of the pre-trained model during inference, without retraining. Based on this, a correction algorithm and multiple distribution stabilization strategies (ranking calibration, time momentum, frequency scaling) are implemented to ensure the continuity and fluency of the generation process. Ultimately, a fine-grained guided generation method that requires no training is obtained, achieving a perfect fusion between the language capabilities of a general large model and specific domain / personal habits, adaptable to clinical writing scenarios. As physicians input single characters / words, content recommendations tailored to personal habits and specialty standards can be generated in real time, significantly reducing the distance of editing and the number of keyboard keystrokes, enabling efficient collaborative writing of medical documents.

[0019] The foregoing has shown and described the basic principles and main features of the present invention, as well as its advantages. It will be apparent to those skilled in the art that the present invention is limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0020] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in the embodiments can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A personalized collaborative data entry method for medical documents based on probability correction, characterized in that, The process includes the following steps: S1, Offline vector database construction: Input the supplementary personalized dataset into the basic large language model for offline forward computation, extract the hidden state vector corresponding to each token position, use it as the feature representation of the context, and directly store it into the external vector database; S2. Online reasoning feature extraction: In the online reasoning process of streaming collaborative writing, in each autoregressive token generation step, the current complete input sequence is sent to the core processing layer of the large language model. The model outputs the context hidden layer state of the current generation step through forward computation. S3. Dual Functionality and Retrieval of Hidden Layer States: Calculations are performed on data generated during the typical autoregressive model generation process. Only a copy of the calculation results is needed; no additional calculations are required. Similarity calculations use the corresponding norm, i.e. ; S4. Retrieval Distribution Stabilization Strategy: Perform a three-step joint refinement and optimization on the retrieved candidates, including ranking-based calibration, historical momentum integration, and frequency scaling. S5. Mixed Probability Generation: Following the derived Bayes' theorem, the external probability is combined with the original posterior probability of the base model to form a mixed posterior distribution, i.e. Then, use greedy decoding to sample the next token.

2. The personalized collaborative data entry method for medical documents based on probability correction according to claim 1, characterized in that, In step S1, the specific calculation process is as follows: traverse the newly added dataset. ,for any sequence of documents It is then input into the basic large language model for forward computation, for each time step The model will output the hidden layer state vector corresponding to that position. Build key-value pairs For direct storage into an external vector database, the entire process only requires traversing the dataset once, without needing to perform any backpropagation or gradient updates on the model.

3. The personalized collaborative data entry method for medical documents based on probability correction according to claim 1, characterized in that, In step S3, the hidden layer state has the following functions: (1) transforming the model to the lexical space through the linear projection layer to form the original posterior probability distribution of the basic model; (2) using the similarity function to retrieve the semantically most relevant context from the constructed external vector database as a query vector.

4. The personalized collaborative data entry method for medical documents based on probability correction according to claim 1, characterized in that, In step S3, the model's own calculation results are reused, and the basic generation probability and retrieval suggestions for specific domains are obtained simultaneously without the need to introduce an additional encoder network, thus achieving fine-grained control at the token level throughout the generation process.

5. A personalized collaborative data entry method for medical documents based on probability correction according to claim 1, characterized in that, In step S4, the ranking-based calibration process is as follows: candidate items are sorted in descending order of similarity and mapped to a preset sparse template distribution, directly discarding long-tail irrelevant terms. Specifically, a monotonically non-decreasing template formula is introduced. This will retrieve all similarities from the previous step. Sort in descending order and assign the corresponding values ​​to the template. The historical momentum integration process is as follows: when calculating the weight of the current step, the smoothed historical weight of the previous time step is introduced to maintain the inertia of sequence generation. ,in The momentum decay coefficient, This represents the preceding state in the historical trajectory. The frequency scaling process is as follows: a penalty term is introduced to explicitly reduce the probability weight of globally high-frequency words. Specifically, the probability weight of each word is scaled using the following formula. ,in For this word in the dataset The global occurrence count in The severity of the punishment.

6. A personalized collaborative data entry method for medical documents based on probability correction according to claim 5, characterized in that, Step S4, Historical Momentum Integration: In actual implementation, the preferred method is... In frequency scaling, the preferred method is... .

7. A personalized collaborative data entry method for medical documents based on probability correction according to claim 5, characterized in that, In step S4, the probability density is estimated by aggregating the data using a method inherited from kernel density estimation, thereby generating the final target-specific guided distribution.

Images