A method for accelerating the question-answering process using language models in medical consultation scenarios

Abstract

This invention relates to a method for accelerating the question-and-answer process using a language model in medical consultation scenarios, belonging to the field of prompting engineering. The method includes the following steps: S1: Start a time window timer, initialize a buffer, and receive user questions through an event-driven framework; S2: Determine if the time window has timed out. If not, the buffer receives consecutive user questions, refreshes the time window, and asynchronously executes step S4; otherwise, execute step S3; S3: Integrate all questions in the buffer and pass them to the language model, which generates corresponding responses, then return to step S1; S4: Use a medical entity dictionary to perform entity recognition and classification on the question set in the buffer from step S2; S5: Generate several predicted questions based on the medical entities identified and classified in step S4, according to rules. If the predicted question is confirmed by the user, it is submitted to the language model to generate a corresponding response; otherwise, return to step S2. This invention solves the problem of lengthy and inefficient question-and-answer processes caused by the current "one question, one answer" chain pattern of language models.

Description

Technical Field

[0001] This invention belongs to the field of prompting engineering in edge environments, and relates to a method for accelerating the question-and-answer process of a language model for medical consultation scenarios. Background Technology

[0002] Large Language Models (LLMs) have garnered significant attention due to their superior performance across various tasks. In the field of cue learning, researchers focus on designing effective cue strategies to wrap the original input, maximizing the potential of LLMs. For example, In-Context Learning (ICL) encourages models to leverage analogical learning to improve performance by adding multiple relevant examples to the cue. Chain of Thought (CoT) methods suggest incorporating intermediate inference steps into the context to assist LLMs in complex reasoning. These strategies often lead to increased cue length, which poses challenges to the inference efficiency and edge-side deployment of language models, given that the computational cost and memory requirements of LLMs are quadratic with cue length.

[0003] To address this challenge, current efforts to optimize inference efficiency primarily focus on three levels: data, model, and system. These methods are mainly dedicated to improving the efficiency of LLM in question-and-answer scenarios, and experimental results demonstrate that they do indeed achieve significant results.

[0004] However, as LLM is increasingly applied to more scenarios, the traditional "question-and-answer" model is revealing its shortcomings. In this model, LLM needs to respond to each user input sequentially. This results in a lengthy and inefficient question-and-answer process, making it unsuitable for medical consultation scenarios that require multiple rounds of dialogue to gradually clarify needs. Furthermore, this chain-like interaction often consumes significant computing resources and memory, increasing the difficulty of deploying LLM at the edge. Therefore, optimizing the question-and-answer efficiency of LLM in medical consultation scenarios can not only improve the fluency of the dialogue and the efficiency of LLM usage, but also reduce the difficulty of scenario deployment and the complexity of user operations, enhancing the inclusiveness and ease of use of intelligent services. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for accelerating the question-and-answer process of a language model in medical consultation scenarios. This method, within a set time window, batch-integrates user questions through a cache, reducing the response frequency of the LLM (Language Modeling) system. Simultaneously, it performs entity recognition and classification on the questions in the cache based on a medical entity dictionary, and generates predicted questions according to rule templates based on the classification results, helping users quickly formulate new questions. Therefore, this method can solve the problem of lengthy and inefficient question-and-answer processes caused by the current chained "question-and-answer" model of language models.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for accelerating the question-answering process using language models in medical consultation scenarios includes the following steps:

[0008] S1: Start the time window timer, initialize the buffer, and receive user queries through the event-driven framework;

[0009] S2: Determine if the time window has timed out. If it has not timed out, the buffer receives continuous user queries, refreshes the time window, and asynchronously executes step S4; otherwise, execute step S3.

[0010] S3: Integrate all the questions in the cache and pass them to the language model, which will generate the corresponding response. Then, proceed to step S1.

[0011] S4: Use a medical entity dictionary to perform entity recognition and classification on the query set in the cache area of ​​step S2;

[0012] S5: For the medical entities identified and classified in step S4, generate several predicted questions according to the rules. If the user clicks to confirm the predicted question, submit it to the language model to generate the corresponding response; otherwise, proceed to step S2.

[0013] Furthermore, step S1 specifically includes the following steps:

[0014] S11: Set the time window timer status to enabled, set the time window parameter value, and clear the buffer.

[0015] S12: Initialize the event-driven framework and start the asynchronous callback process to continuously listen for and capture user input signals;

[0016] Furthermore, step S2 specifically includes the following steps:

[0017] S21: After the user's input question is captured by the event-driven framework, it is determined whether the time window has expired. If it has not expired, steps S22 and S23 are executed; otherwise, step S24 is executed.

[0018] S22: Immediately write the user's question to the cache;

[0019] S23: Refresh the time window to ensure the user has sufficient time for the next input;

[0020] S24: Pause accepting new questions and set the time window to timeout status;

[0021] Furthermore, step S3 specifically includes the following steps:

[0022] S31: The event-driven framework detects timeout events in the time window and integrates the questions in the cache into a unified question input format;

[0023] S32: Call the language model to process the integrated question input, generate and return the answer, and jump to step S1;

[0024] Furthermore, step S4 specifically includes the following steps:

[0025] S41: Use query statements to extract medical entity sets from the medical knowledge graph to construct an entity classification dictionary;

[0026] S42: Use a string search algorithm to match the user queries in the cache in step S2 with the entity classification dictionary to identify and classify the medical entities in the user queries;

[0027] Furthermore, step S5 specifically includes the following steps:

[0028] S51: Using the classification results of step S4 as clues, obtain coarse-grained or fine-grained user intent types by connecting these clues;

[0029] S52: Based on the intent type, first use a rule-based method to generate the corresponding predictive question;

[0030] S53: The user makes a selection based on the predicted question obtained in step S52. If the user selects to confirm, the predicted question is passed to the language model to generate a corresponding response; otherwise, the process jumps to step S2.

[0031] The beneficial effects of this invention are as follows:

[0032] 1) The time window mechanism effectively integrates users' continuous questions, freeing the language model from frequently responding to user questions, greatly shortening the time required for the question-and-answer process, and improving process efficiency.

[0033] 2) The intent prediction mechanism can quickly and easily generate predictive questions for users to refer to, increasing the possibility that user questions will be captured by the cache and further processed, thus improving the efficiency of process interaction.

[0034] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. Attached Figure Description

[0035] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0036] Figure 1 For; Input template diagram;

[0037] Figure 2 This is a time-consuming flowchart for a scheduled window.

[0038] Figure 3 For example: Schematic diagram of entity recognition and classification;

[0039] Figure 4 For example: a diagram illustrating the generation of predicted questions;

[0040] Figure 5 For: Summary diagram of the instruction manual; Detailed Implementation

[0041] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0042] Figure 1 This is a schematic diagram of the input template. Figure 2 This is a diagram illustrating the time consumption of the timed window process. Figure 3 This is a diagram illustrating entity recognition and classification. Figure 4 A diagram is generated to predict the question.

[0043] This invention provides a method for accelerating the language model question-answering process in medical consultation scenarios. The method mainly includes the following steps:

[0044] Step 1: Start the time window timer, initialize the buffer, and receive user queries through the event-driven framework;

[0045] Step 2: Determine if the time window has timed out. If it has not timed out, the buffer receives continuous user queries, refreshes the time window, and asynchronously executes step S4; otherwise, execute step S3.

[0046] Step 3: Integrate all the questions in the cache and pass them to the language model, which will generate the corresponding response. Then, proceed to step S1.

[0047] Step 4: Use a medical entity dictionary to perform entity recognition and classification on the query set in the cache area from step S2;

[0048] Step 5: For the medical entities identified and classified in Step S4, generate several predicted questions according to the rules. If the user clicks to confirm the predicted question, submit it to the language model to generate the corresponding response; otherwise, go to Step S2.

[0049] Step 1 specifically includes:

[0050] (1) Initialize data structures. Create a buffer (Buffer) and an input time queue (Time). inputTo store consecutive user queries Q = [q1, q2, ..., q n [Time] and the time taken for the user to input the question. input =[t input1 ,t input2 ,...,t inputk ];

[0051] (2) Initialize parameters. Initialize parameters such as time window length w and maximum input time queue capacity k. Among them, k is used to dynamically adjust the time window length w to ensure that the time window matches the user's input speed.

[0052] (3) Bind an input event listener to the user input device (such as a keyboard or input box). Each time an input event is captured, record the start and end times of the input and calculate the duration of a single input.

[0053] Step 2 specifically includes:

[0054] (1) Capturing user input. The system listens for user input signals through an event-driven mechanism. Whenever the user enters a question q... i At that time, the event-driven framework will capture the input and trigger the callback function, with the input time t... input It will be recorded in the Time queue. input =[t input1 ,t input2 ,...,t inputk ].

[0055] (2) Determine if the time window has timed out. When the user inputs time t... input If the time window value is less than or equal to w, execute (3); otherwise execute (4).

[0056] (3) The buffer receives user questions and refreshes the time window. First, the question q is displayed. i Write to the buffer, and then the system immediately refreshes the time window. If dynamic adjustment is enabled, it is based on Time. input The median of the most recent k records is reset by w = median(t) input1 ,t input2 ,...,t inputk ).

[0057] (4) Time window timeout handling. When the user inputs a time t... input When the value exceeds the time window value w, the cache will prohibit further writes and force a time window timeout.

[0058] Step 3 specifically includes:

[0059] (1) Question Integration. When a time window timeout signal is detected, the questions in the buffer are integrated into a unified prompt template in chronological order. The prompt template consists of two parts: instructions and questions. The instruction part clarifies the model's role and answer structure, ensuring that the model-generated answer is logically clear and comprehensive. The question part connects the questions already entered by the user, providing the necessary information for the model to generate an answer, as detailed below. Figure 1 As shown.

[0060] (2) Processing by calling the language model. The integrated question is passed to the language model for processing. After analyzing the context of the question, the language model returns the generated overall response to the user.

[0061] Specifically, the time consumed in the question-and-answer process can be roughly divided into three parts: the time spent by the user asking and inputting the question (t). input Language model response time t model And the waiting time t of the time window window .like Figure 2 As shown, without a timed window, each question entered by the user is sent to the language model, requiring multiple responses to provide a satisfactory answer. With a timed window, if the user completes each question within its designated window without triggering a timeout, a satisfactory answer can be obtained in one go. Conversely, if a timeout is triggered, previously entered questions are sent to the language model, providing an intermediate answer. The user then needs to continue entering the remaining questions to complete the information required by the language model and obtain the final answer. When a timeout is triggered, unfinished questions are not cleared, allowing the user to continue entering, but only after the language model returns an intermediate answer.

[0062] like Figure 3 As shown, step 4 is as follows:

[0063] (1) Constructing a medical entity classification dictionary. Using the Cypher query `MATCH(m:{EntityType})`, medical entity sets are retrieved from the open-source medical knowledge graph by category to construct a classification dictionary D. There are five categories: diseases, symptoms, medicines, food, and examinations, i.e., D = {E}. disease E symptom E medicine E food E check}, where E disease E represents a set of medical entities related to diseases. symptom E medicine E food E check And so on.

[0064] (2) Constructing a medical entity prefix tree. Initialize an empty root node, then traverse the classification dictionary D. For each entity, insert it character by character into the Trie tree. If a character does not exist in the child node set of the current node, create a new node and add it to the child node set of the current node. After all characters of an entity have been inserted, mark the last character node of the entity as the word end node. The constructed medical entity prefix tree can complete the insertion operation of a single entity in O(L) time complexity (where L is the average length of the entity) and form a hierarchical structure in the tree, effectively supporting subsequent matching tasks.

[0065] (3) Matching medical entities. Using the constructed Trie tree, we write the user query Q = [q1, q2, ..., q] into the buffer in step 2. n The medical entities in the dataset are matched character-by-character against a known set of entities. Starting with the initial character input by the user, the system searches down the Trie tree level by level, attempting to find a matching path. If a match is successful and the current node is marked as the end of a word, a complete entity is identified. For each identified medical entity, its category can be determined using the classification dictionary D. If a match cannot be continued at a certain node, the system jumps to the node with the longest suffix path based on the failure pointer and continues to attempt a match, thus avoiding repeated scanning of already processed characters.

[0066] The failure pointer is calculated using a breadth-first search. It provides a backtracking path for each node, pointing to a node where a match might exist. During the matching process, when a path is broken, the failure pointer allows for a quick jump. Combined with the character-by-character search, unnecessary paths can be effectively skipped. Furthermore, to support simultaneous recognition of multiple entities, when a character matches multiple termination nodes, we record all possible entities and continue searching downwards to ensure coverage of all valid paths.

[0067] Step 5 is as follows:

[0068] (1) Predicting user intent type. In this process, each result identified and categorized in the user's question is first treated as a clue. i Each clue includes the medical entity e and its type y, i.e., Clue i = {e,y}, where e∈{E} disease ∪E symptom ∪E medicine ∪E food ∪E checkLet y ∈ {disease, symptom, medicine, food, check}. Then, these clues are linked together. If the number of clue types extracted from the user description is 1, coarse-grained intent inference is performed to identify the user's basic needs; if the number of clue types extracted is greater than or equal to 2, fine-grained intent inference is performed to more accurately analyze the user's complex needs. For example, in Figure 4 In the first dialogue, step 4 extracts the clue Clue1 = {gallstones, disease}, and we coarsely infer the user's intent as a "disease-related" intent. In the second dialogue, step 4 extracts two clues Clue1 = {rhinitis, disease} and Clue2 = {amoxicillin, medicine}, and we finely infer the user's intent as a mixed "medicine and disease" intent.

[0069] (2) Generating Predictive Questions Based on Rules. Based on the inferred user intent type, we use a rule-based approach to generate common medical-related questions. Question templates typically consist of two parts: entities and fixed template statements. Entities refer to specific medical concepts, such as… Figure 4 The template includes options like gallstones, rhinitis, and amoxicillin; while the fixed template statements are tailored to the user's intent. Specifically, the predicate in the template typically expresses the action or state of the core question (e.g., "what is it?", "what are some examples"), while the object supplements the question with specific information (e.g., "symptoms", "treatment methods"). It's important to note that predictive questions are usually generated in multiple ways, allowing the user to choose, thus increasing the prediction success rate.

Claims

1. A method for accelerating the question-and-answer process using a language model in medical consultation scenarios, characterized in that, The method includes the following steps: S1: Start the time window timer, initialize the buffer, and receive user queries through the event-driven framework; S2: Determine if the time window has timed out. If it has not timed out, the buffer receives continuous user queries, refreshes the time window, and asynchronously executes step S4; otherwise, execute step S3. S3: Integrate all the questions in the cache and pass them to the language model, which will generate the corresponding response. Then, proceed to step S1. S4: Use a medical entity dictionary to perform entity recognition and classification on the query set in the cache area of ​​step S2; S5: For the medical entities identified and classified in step S4, generate several predicted questions according to the rules; if the predicted question is confirmed by the user, submit it to the language model to generate the corresponding response; otherwise, go to step S2.

2. The method for accelerating the question-and-answer process of a language model for medical consultation scenarios according to claim 1, characterized in that, Step S1 specifically includes: S11: Set the time window timer status to enabled, set the time window parameter value, and clear the buffer. S12: Initialize the event-driven framework and start an asynchronous callback process to continuously listen for and capture user input signals.

3. The method for accelerating the question-and-answer process of a language model for medical consultation scenarios according to claim 1, characterized in that, Step S2 specifically includes: S21: After the user's input question is captured by the event-driven framework, it is determined whether the time window has expired. If it has not expired, steps S22 and S23 are executed; otherwise, step S24 is executed. S22: Immediately write the user's question to the cache; S23: Refresh the time window to ensure the user has sufficient time for the next input; S24: Pause accepting new questions and set the time window to timeout status.

4. The method for accelerating the question-and-answer process of a language model for medical consultation scenarios according to claim 1, characterized in that, Step S3 specifically includes: S31: The event-driven framework detects timeout events in the time window and integrates the questions in the cache into a unified question input format; S32: Call the language model to process the integrated question input, generate and return the answer, and jump to step S1.

5. The method for accelerating the question-and-answer process of a language model for medical consultation scenarios according to claim 1, characterized in that, Step S4 specifically includes: S41: Use query statements to extract medical entity sets from the medical knowledge graph to construct an entity classification dictionary; S42: Use a string search algorithm to match the user queries in the cache from step S2 with the entity classification dictionary to identify and classify the medical entities in the user queries.

6. The method for accelerating the question-answering process of a language model in a medical consultation scenario according to claim 1, characterized in that, Step S5 specifically includes: S51: Using the classification results of step S4 as clues, obtain coarse-grained or fine-grained user intent types by connecting these clues; S52: Based on the intent type, first use a rule-based method to generate the corresponding predictive question; S53: The user makes a selection based on the predicted question obtained in step S52. If the user selects to confirm, the predicted question is passed to the language model to generate a corresponding response; otherwise, the process jumps to step S2.

Images