Medical conversation-oriented time-aware enhanced reply generation method and device

Abstract

The application provides a time-aware enhanced reply generation method and device for medical dialogues, which has the following characteristics: S1, constructing a time-aware multi-turn dialogue dataset according to existing medical data; S2, training an existing medical large language model according to the time-aware multi-turn dialogue dataset to obtain a medical dialogue model; S3, encoding a dialogue sequence to obtain an encoded state representation; S4, encoding a historical dialogue to obtain an encoded memory vector; S5, combining the encoded state representation and the encoded memory vector to obtain a fused state representation; S6, enhancing the fused state representation according to a time attention mechanism to obtain an enhanced state representation; and S7, inputting a current question and the enhanced state representation into the medical dialogue model to obtain a medical dialogue reply. In summary, the method can process time-related medical dialogues and generate replies with time awareness.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical dialogue, and particularly relates to a time-aware enhanced reply generation method and device for medical dialogue. BACKGROUND

[0002] Medical dialogue systems have great potential in providing timely and effective medical information and consultation, especially in emergency or resource-limited situations. However, current large language models (LLMs) often overlook the time-awareness factor when dealing with medical dialogue. In emergency medical situations or chronic disease management, the importance of time is self-evident, and any delay may affect treatment effectiveness or even endanger lives. Therefore, it is very important to enhance the time-awareness of LLMs in medical dialogue systems.

[0003] In recent years, LLMs have made significant progress in the field of medical dialogue. Some research focuses on simulating real-world interactions between doctors and patients and providing subtle medical advice. These systems emphasize interactivity and context awareness, and even attempt to transform LLMs into virtual medical consultants. At the same time, the medical research community is also working to organize comprehensive medical dialogue datasets, and some research has enriched the model's real clinical dialogue and knowledge retrieval capabilities. In particular, Yun et al. fine-tuned the ChatDoctor model using 100,000 real-world patient-doctor dialogues from an online medical consultation website, improving the model's autonomous knowledge retrieval capabilities. In addition, some research focuses on the development of medical dialogue basic knowledge coding systems, aiming to enhance the medical knowledge base of LLMs. For example, Toma et al. integrated a conversational knowledge coding system into LLMs to enhance their knowledge reserves in the medical field.

[0004] In summary, existing technologies mainly focus on improving interactivity, context awareness, and the accuracy of medical knowledge, and have made significant progress in dataset construction and transfer learning. These works collectively highlight the evolving pattern of LLMs in medical dialogue, pushing them towards a more interactive and context-aware direction. However, they often overlook the time factor, which is particularly crucial in medical scenarios. Without time information, even accurate medical advice may lack specific execution timing, affecting treatment effectiveness and patient satisfaction. SUMMARY

[0005] The present application is to solve the above problems, and aims to provide a time-aware enhanced reply generation method and device for medical dialogue.

[0006] The application provides a time-aware enhanced reply generation method for medical dialogue, which is used for generating a corresponding medical dialogue reply according to a dialogue sequence containing a history dialogue and a current question, and has the following characteristics, comprising the following steps: step S1, constructing a time-aware multi-turn dialogue dataset according to existing medical data; step S2, training an existing medical large language model according to the time-aware multi-turn dialogue dataset to obtain a medical dialogue model; step S3, encoding the dialogue sequence to obtain an encoded state representation s (1：t) ; step S4, encoding the history dialogue to obtain an encoded memory vector m h ; step S5, combining the encoded state representation s (1：t) and the encoded memory vector m h to obtain a fused state representation Step S6, enhancing the fused state representation according to a time attention mechanism to obtain an enhanced state representation Step S7, inputting the current question and the enhanced state representation into the medical dialogue model to obtain the medical dialogue reply, wherein step S2 comprises the following sub-steps: step S2-1, performing supervised fine-tuning on the medical large language model according to the time-aware multi-turn dialogue dataset to obtain a fine-tuned medical dialogue model; step S2-2, obtaining a fine-tuned dialogue reply according to the time-aware multi-turn dialogue dataset and the fine-tuned medical dialogue model; step S2-3, constructing a reward model according to the fine-tuned dialogue reply; and step S2-4, performing dynamic time reinforcement learning on the fine-tuned medical dialogue model according to the reward model to obtain the medical dialogue model.

[0007] In the time-aware enhanced reply generation method for medical dialogue provided by the application, the process expression for generating the encoded state representation s (1：t) in step S3 can be e (1：t) =Emb(d (1：t) )+PE((1:t)),s (1：t) =TE(e (1：t) ), wherein Emb, PE and TE represent embedding operation, position encoding operation and processing via a Transformer encoder in sequence, d (1：t) is the dialogue sequence, e (1：t) is the embedding representation of the dialogue sequence, and PE((1:t)) is a position encoding function that provides a unique representation of each time in the dialogue sequence.

[0008] In the time-aware enhanced reply generation method for medical dialogue provided by the application, the calculation expression of the encoded memory vector m h in step S4 can be m hTE(d h , where d h is the history dialogue, TE is the processing via the Transformer encoder, and the enhanced state representation is obtained in step S6.

[0009] In the medical dialogue-oriented time-aware enhanced reply generation method provided by the application, the expression of the enhanced state representation obtained in step S6 can have the following features: where α i is the attention weight of the i-th historical event, W a is the weight matrix for attention calculation, b a is the bias term for attention calculation, and v i is the state representation of the i-th historical event.

[0010] In the medical dialogue-oriented time-aware enhanced reply generation method provided by the application, the step S1 can include the following sub-steps: step S1-1, preprocessing the medical data to obtain preprocessed medical data; step S1-2, dividing the preprocessed medical data into different dialogue records according to patients, and sorting the data in each dialogue record in chronological order to obtain the corresponding multi-round dialogue record; step S1-3, inserting a timestamp into each multi-round dialogue record according to a large language model to obtain a corresponding time-aware multi-round dialogue record; and step S1-4, taking all time-aware multi-round dialogue records as a time-aware multi-round dialogue dataset.

[0011] In the medical dialogue-oriented time-aware enhanced reply generation method provided by the application, the expression of the loss function of the supervised fine-tuning in step S2-1 can have the following features: where is the supervised fine-tuning loss, CrossEntropy is the cross-entropy loss function, is the prediction reply obtained by the medical large language model according to the question D j of the j-th dialogue in the time-aware multi-round dialogue record in the input time-aware multi-round dialogue dataset and the corresponding enhanced state representation , Res is the reply corresponding to the j-th dialogue in the time-aware multi-round dialogue record, and ∑ j is the training of the medical large language model using all time-aware multi-round dialogue records in the time-aware multi-round dialogue dataset.

[0012] The time-aware enhanced response generation method for medical dialogue provided by this invention may also have the following features: In step S2-2, the fine-tuned dialogue response is a sequence of responses generated by the fine-tuned medical dialogue model based on all responses from time-aware multi-turn dialogue records in the input time-aware multi-turn dialogue dataset, after refinement and sorting by medical experts. In step S2-3, the reward model is trained by calculating the loss through fine-tuning the dialogue responses. The expression for the loss calculation is: In the formula, x represents a response in the fine-tuning dialogue, D is the set of all ranked pairs constructed from pairwise responses in the fine-tuning dialogue, and y... w and y l For the two responses in the ranked pair, and y w For the higher-ranked responses, y l For lower-ranked responses, r θ The scalar output of the reward model is given, where k is the total number of replies in the fine-tuned dialogue.

[0013] The time-aware enhanced response generation method for medical dialogue provided by this invention may also have the following feature: wherein, in steps S2-4, dynamic time reinforcement learning includes Q-value update, action selection, timeliness decay, and policy optimization, and the expression for Q-value update is: In the formula Q(s) t (a; θ) is the action value function, used to estimate the action value at time step t in state s. t Let α be the expected reward when action 'a' is taken and the policy parameter is θ, where α is the learning rate and γ is the discount factor. Let R be the maximum Q value that can be obtained in the next time step t+1 and all possible actions a′. t The expression for action selection is: (The reward obtained at time t according to the reward model) In the formula a t+1 To determine the optimal action at the next time step t+1, To assess the current state The relative value of taking action 'a', Argmax a To find the relative value among all possible actions a The expression for the maximum timeliness decay of an action is: In the formula, β is the decay factor, and Δt is the time difference between now and the initial event. The raw reward given to the model's response at a specific time step t. To take into account the time elapsed after the event, the original reward... The new reward after time decay adjustment, the policy optimization expression is: In the formula The advantage function of the previously deployed strategy, clip is the clipping function, ∈ is the clipping parameter, π θ (a|s) represents the probability of taking action a given state s under the current policy parameter θ. For the previous policy parameter θ old Given the same state s, the probability of taking the same action a.

[0014] This invention also provides a time-aware enhanced response generation device for medical dialogues, used to generate corresponding medical dialogue responses based on a dialogue sequence including historical dialogues and the current question. It features the following components: a data input module for user input of the dialogue sequence; and a first encoding module for encoding the dialogue sequence to obtain an encoded state representation s. (1：t) The second encoding module is used to encode historical dialogues, obtaining an encoded memory vector m. h The fusion module is used to convert the encoded state representation s (1：t) and the encoded memory vector m h By combining them, we obtain the fusion state representation. Enhancement modules are used to represent the fusion state based on a temporal attention mechanism. Enhancement is performed to obtain the enhanced state representation. The response generation module includes a medical dialogue model, used to generate responses based on the current question and enhanced state representation. The medical dialogue response is obtained by training an existing large-scale medical language model on a time-aware multi-turn dialogue dataset constructed from existing medical data. The training process includes the following sub-steps: Step S2-1, supervised fine-tuning of the large-scale medical language model based on the time-aware multi-turn dialogue dataset to obtain a fine-tuned medical dialogue model; Step S2-2, obtaining a fine-tuned dialogue response based on the time-aware multi-turn dialogue dataset and the fine-tuned medical dialogue model; Step S2-3, constructing a reward model based on the fine-tuned dialogue response; Step S2-4, performing dynamic temporal reinforcement learning on the fine-tuned medical dialogue model based on the reward model to obtain the medical dialogue model.

[0015] The role and effect of invention

[0016] According to the time-aware enhanced response generation method and apparatus for medical dialogues of the present invention, on the one hand, by constructing a time-aware multi-turn dialogue dataset with timestamp information and using it as the training dataset for the model, the trained medical dialogue model can effectively process time-sensitive information; on the other hand, by training an existing large language model based on the time-aware multi-turn dialogue dataset through supervised fine-tuning and dynamic temporal reinforcement learning, the trained medical dialogue model can achieve better time-aware performance. Therefore, the time-aware enhanced response generation method and apparatus for medical dialogues of the present invention can handle time-related medical dialogues and generate time-aware responses. Attached Figure Description

[0017] Figure 1 This is a block diagram of a time-aware enhanced response generation device for medical dialogue in an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram illustrating the principle of constructing a time-aware multi-turn dialogue dataset in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the process of constructing a time-aware multi-turn dialogue dataset in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the process of training the medical dialogue model in an embodiment of the present invention;

[0021] Figure 5 This is a flowchart illustrating the time-aware enhanced response generation method for medical dialogue in an embodiment of the present invention. Detailed Implementation

[0022] To make the technical means, creative features, objectives and effects of the present invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, provide a detailed description of the time-awareness-enhanced response generation method and apparatus for medical dialogue.

[0023] This embodiment provides a time-aware enhanced response generation device for medical dialogues, which generates corresponding medical dialogue responses based on a dialogue sequence that includes historical dialogues and the current question.

[0024] Figure 1 This is a block diagram of a time-aware enhanced response generation device for medical dialogue in an embodiment of the present invention.

[0025] like Figure 1 As shown, the time-aware enhanced response generation device 100 includes a data input module 10, a first encoding module 20, a second encoding module 30, a fusion module 40, an enhancement module 50, a response generation module 60, and a data output module 70.

[0026] The data input module 10 is used for user input of dialogue sequences.

[0027] The first encoding module 20 is used to encode the dialogue sequence to obtain the encoded state representation s. (1:t) .

[0028] In medical dialogues, context and temporal ordering are crucial. Therefore, in this embodiment, the first encoding module 20 first performs an embedding process on the dialogue sequence to ensure that each part of the dialogue represents not only its content but also its position in the sequence. Then, a Transformer encoder is used to encode this representation. This encoder enables the proposal to capture the content of the dialogue and its relationship with previous and subsequent dialogues, thereby obtaining the encoded state representation s. (1:t) The process expression for the above process is:

[0029] e (1：t) =Emb(d (1：t) )+PE((1:t)),

[0030] s (1：t) =TE(e (1：t) ),

[0031] In the formula, Emb, PE, and TE represent the embedding operation, the positional encoding operation, and the processing via the Transformer encoder, respectively, and d (1：t) For the dialogue sequence, e (1：t) For the embedded representation of the dialogue sequence, PE((1:t)) is the positional encoding function that provides a unique representation of each time in the dialogue sequence.

[0032] The second encoding module 30 is used to encode historical dialogues to obtain an encoded memory vector m. h .

[0033] Wherein, the encoding memory vector m h The calculation expression is:

[0034] m h =TE(d) h ),

[0035] In the formula d h For historical dialogues, TE is processed via the Transformer encoder.

[0036] The fusion module 40 is used to convert the encoded state representation s (1：t) and the encoded memory vector m h By combining them, we obtain the fusion state representation.

[0037] Wherein, the fusion state represents The calculation expression is:

[0038]

[0039] In this embodiment, the long-term historical dialogue is encoded into a memory vector, namely the encoded memory vector m, by the second encoding module 30. h Then, the fusion module 40 represents the real-time time state, i.e., the encoded state, as s. (1：t) This is seamlessly combined with the long-term memory vector to generate a fused state representation that integrates current and historical interactions. This addresses the issue that capturing only recent interactions, given the temporal depth of medical conversations, might miss crucial historical information.

[0040] Enhancement module 50 is used to represent the fusion state according to the temporal attention mechanism. Enhancement is performed to obtain the enhanced state representation.

[0041] In this embodiment, a temporal attention mechanism is introduced through enhancement module 50 to assign variable attention weights to different historical events. Key events that may have greater weight include early disease indicators, past surgeries, drug responses, and major health changes. By emphasizing specific historical events, the state representation is further enhanced, resulting in an enhanced state representation. Then, an enhanced state representation is generated. The expression is:

[0042]

[0043]

[0044] In the formula α i W represents the attention weight for the i-th historical event. a b is the weight matrix used for attention calculation. a v is the bias term used for attention calculation. i Let i represent the state of the i-th historical event.

[0045] The response generation module 60 includes a medical dialogue model for use based on the current question and enhanced state representation. Received a response from the medical dialogue.

[0046] The medical dialogue model is trained on an existing large-scale medical language model based on a time-aware multi-turn dialogue dataset constructed from existing medical data. In this embodiment, the existing medical data is data from the MIMIC III database, which includes patient medical records, diagnoses, laboratory tests, treatments, and times.

[0047] Figure 2This is a schematic diagram illustrating the principle of constructing a time-aware multi-turn dialogue dataset in an embodiment of the present invention.

[0048] like Figure 2 As shown, in this embodiment, the existing medical data is preprocessed, patient records are sorted, multi-turn dialogues are generated, and timestamps are added in sequence to construct a time-aware multi-turn dialogue dataset.

[0049] Figure 3 This is a schematic diagram of the process for constructing a time-aware multi-turn dialogue dataset in an embodiment of the present invention.

[0050] like Figure 3 As shown, constructing a time-aware multi-turn dialogue dataset includes the following steps:

[0051] Step S1-1: Preprocess the medical data to obtain preprocessed medical data.

[0052] In this embodiment, step S1-1 is to perform data preprocessing, which includes filling missing data with the average value, performing rule checks on inconsistent records, such as conflicts between a patient's birth date and admission date, and deleting outliers, such as an adult's weight being recorded as 5 kg.

[0053] Step S1-2 involves dividing the preprocessed medical data into different records according to the patients, and sorting the data in each record in chronological order to obtain the corresponding sorted records. In this embodiment, step S1-2 is the operation of sorting patient records.

[0054] Steps S1-3 generate corresponding multi-turn dialogue records based on each sorted record.

[0055] In this embodiment, steps S1-3 involve performing the multi-turn dialogue generation operation, generating sorted multi-turn dialogue records based on timestamps of patient records using GPT-4. The timestamps are constructed based on the time of important events such as "one day before surgery" and "three days after surgery". These events, such as "symptom onset", "medication", "surgery" and "examination", are extracted by UIE.

[0056] Steps S1-4: For each multi-turn dialogue record, timestamps are inserted according to the large language model to obtain the corresponding time-aware multi-turn dialogue record.

[0057] In this embodiment, steps S1-4 involve adding timestamps to address biases in manually constructed multi-turn conversation logs with timestamps. These timestamps are extracted from time-related information in the MIMIC III database. When a conversation mentions a specific event, such as surgery or an examination, these events are also recorded in the MIMIC III database. First, a random timestamp is inserted at the start of the conversation. Subsequently, additional timestamps corresponding to these specific events in the database are added throughout the conversation. The inserted timestamps are validated by a medical professional before being integrated into the conversation. For example, phrases like "removed catheter three days post-surgery" can be inserted to ensure that the timestamps accurately correspond to the actual timing of the event.

[0058] Steps S1-5: Collect all time-aware multi-turn dialogue records as a time-aware multi-turn dialogue dataset.

[0059] Figure 4 This is a schematic diagram of the process of training a medical dialogue model in an embodiment of the present invention.

[0060] like Figure 4 As shown, the process of training a large-scale medical language model based on a time-aware multi-turn dialogue dataset to obtain a medical dialogue model includes the following sub-steps:

[0061] Step S2-1: Supervised fine-tuning of the large-scale medical language model based on the time-aware multi-turn dialogue dataset to obtain the fine-tuned medical dialogue model.

[0062] The expression for the supervised fine-tuning loss function is as follows:

[0063]

[0064] In the formula For supervised fine-tuning loss, CrossEntropy is the cross-entropy loss function. For a large-scale medical language model, the question D of the j-th dialogue in the time-aware multi-turn dialogue record in the input time-aware multi-turn dialogue dataset is generated. j and the corresponding enhanced state representation The predicted responses are obtained, where Res is the response corresponding to the j-th dialogue in the time-aware multi-turn dialogue record, ∑ j To train a large-scale medical language model using all time-aware multi-turn dialogue records in a time-aware multi-turn dialogue dataset.

[0065] Step S2-2: Based on the time-aware multi-turn dialogue dataset and the fine-tuned medical dialogue model, obtain the fine-tuned dialogue response.

[0066] Among them, the fine-tuned dialogue response is the response sequence generated by the fine-tuned medical dialogue model based on all responses generated from the time-aware multi-turn dialogue records in the input time-aware multi-turn dialogue dataset, after being refined and sorted by medical experts. In this embodiment, all responses before refinement and sorting are sorted according to the output probability. When refining and sorting, not only the content is considered, but also the time relevance of the responses.

[0067] Steps S2-3: Construct a reward model based on the fine-tuned dialogue responses.

[0068] The reward model is trained by fine-tuning dialogue responses and calculating the loss. The expression for the loss calculation is as follows:

[0069]

[0070] In the formula, x represents a response in the fine-tuning dialogue, D is the set of all ranked pairs constructed from pairwise responses in the fine-tuning dialogue, and y... w and y l For the two responses in the ranked pair, and y w For the higher-ranked responses, y l For lower-ranked responses, r θ The scalar output of the reward model is given, where k is the total number of replies in the fine-tuned dialogue.

[0071] In this embodiment, the input to the reward model is the response, and the output is the response quality score, i.e., the score. Furthermore, because fine-tuning dialogue responses is based on response sequences according to human preferences, the trained reward model can mimic human preferences, assigning higher scores (rewards) to better responses and lower scores (rewards) to poorer responses.

[0072] Step S2-4: Perform dynamic temporal reinforcement learning on the fine-tuned medical dialogue model based on the reward model to obtain the medical dialogue model.

[0073] Dynamic temporal reinforcement learning includes Q-score updates, action selection, timeliness decay, and policy optimization. In this embodiment, Q-score updates and action selection are directly related to how to react based on the current dialogue state and historical information. Timeliness decay and policy optimization help the model adjust its actions and policies over a long period to ensure that the provided medical advice, i.e., medical dialogue responses, remains relevant over time.

[0074] In this embodiment, the Q-value represents the patient's response to the clarity or usefulness of the suggestions given by the medical dialogue model. The expression for updating the Q-value is:

[0075]

[0076] In the formula Q(s) t(a; θ) is the action value function, used to estimate the action value at time step t in state s. t Let α be the expected reward when action 'a' is taken and the policy parameter is θ, α be the learning rate (which determines how quickly new information overwrites old information), and γ be the discount factor (which balances the importance of immediate and future rewards). A value close to 1 will make the model pay more attention to future rewards, while a value close to 0 will make the model focus almost exclusively on immediate rewards. To determine the maximum expected Q-value among all possible actions a′ at the next time step t+1, this part of the calculation is based on the same policy parameter θ, and greedily selects the action that maximizes the expected reward by choosing the action with the maximum Q-value. t Let be the reward obtained according to the reward model at time t.

[0077] In this embodiment, considering the immediate context of the dialogue is important, but its temporal details are also crucial. For example, if a patient mentions they "just had surgery," the medical dialogue model must be able to retain this key event and provide relevant medical advice in subsequent interactions, such as suggesting "remove the tube in three days." Therefore, the expression for action selection is:

[0078]

[0079] In the formula a t+1 To determine the optimal action at the next time step t+1, To assess the current state The relative value of taking action 'a', Argmax a To find the relative value among all possible actions a The biggest action.

[0080] In this embodiment, the expression for action selection ensures that every decision made by the medical dialogue model is consistent with the ongoing dialogue and maximizes the positive outcomes expected by the patient.

[0081] Because the relevance of medical advice often depends on its timeliness, the length of time since the event occurred needs to be considered when adjusting the rewards for certain historical behaviors. Therefore, the importance of medical advice may decrease over time. In this embodiment, a decay factor β and a time interval Δt are used to adjust the reward value to ensure that medical advice remains consistent with the latest medical conditions and patient feedback. The expression for timeliness decay is:

[0082]

[0083] In the formula, β is the decay factor, and Δt is the time difference between now and the initial event. The raw reward given to the model's response at a specific time step t. To take into account the time elapsed after the event, the original reward... The new reward after time decay adjustment.

[0084] This embodiment uses a clip function to limit the pace of policy updates to ensure stability. This function minimizes the expected value of the smaller of the probability ratio between the current and old policies multiplied by the Advantage value, and the probability ratio of the clipped policy multiplied by the Advantage value. For example, if a patient receives advice about postoperative care, the relevance of those advice and the resulting reward may decrease as the days following surgery progress. This ensures that the proposed recommendations are always consistent with the latest medical information and patient feedback. Therefore, the expression for policy optimization in this embodiment is:

[0085]

[0086] In the formula The advantage function of the previously deployed strategy, clip is the clipping function, ∈ is the clipping parameter, π θ (a|s) represents the probability of taking action a given state s under the current policy parameters θ, indicating the likelihood of the model taking a specific action under the latest optimized policy. For the previous policy parameter θ old The probability of taking the same action a given state s represents the likelihood that the model would take that action before the last policy update.

[0087] The data output module 70 is used to display medical dialogue responses to the user.

[0088] The following description, in conjunction with the accompanying drawings, illustrates the process of generating a time-aware enhanced response for medical dialogue using the time-aware enhanced response generation device of this embodiment.

[0089] Figure 5 This is a flowchart illustrating the time-aware enhanced response generation method for medical dialogue in an embodiment of the present invention.

[0090] like Figure 5 As shown, the time-aware enhanced response generation method for medical dialogue in this embodiment includes the following steps:

[0091] Step S1: Construct a time-aware multi-turn dialogue dataset based on existing medical data.

[0092] Step S2: Train the existing large-scale medical language model based on the time-aware multi-turn dialogue dataset to obtain a medical dialogue model.

[0093] Step S3: The dialogue sequence received by the data input module 10 is encoded by the first encoding module 20 to obtain the encoded state representation s. (1：t) .

[0094] Step S4: Encode the historical dialogue using the second encoding module 30 to obtain the encoded memory vector m. h .

[0095] Step S5, the encoded state representation s is processed by the fusion module 40. (1：t) and the encoded memory vector m h By combining them, we obtain the fusion state representation.

[0096] Step S6, the enhancement module 50 adjusts the fusion state representation according to the time attention mechanism. Enhancement is performed to obtain the enhanced state representation.

[0097] Step S7, represent the current question and enhancement state. Input the response generation module 60, which contains a medical dialogue model, and obtain the medical dialogue response.

[0098] Step S8: Display the medical dialogue response to the user through the data output module 70.

[0099] The role and effect of the embodiments

[0100] According to the time-aware enhanced response generation method and apparatus for medical dialogue involved in this embodiment, on the one hand, by constructing a time-aware multi-turn dialogue dataset with timestamp information and using it as the training dataset for the model, the trained medical dialogue model can effectively process time-sensitive information; on the other hand, based on the time-aware multi-turn dialogue dataset, an existing large language model is trained through supervised fine-tuning and dynamic temporal reinforcement learning, enabling the trained medical dialogue model to have better time-aware performance. In summary, this method can handle time-related medical dialogues and generate time-aware responses.

[0101] The above embodiments are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention.

Claims

1. A time-aware enhanced response generation method for medical dialogues, used to generate corresponding medical dialogue responses based on a dialogue sequence including historical dialogues and the current question, characterized in that, Includes the following steps: Step S1: Construct a time-aware multi-turn dialogue dataset based on existing medical data; Step S2: Train the existing large-scale medical language model based on the time-aware multi-turn dialogue dataset to obtain a medical dialogue model; Step S3: Encode the dialogue sequence to obtain the encoded state representation s. (1:t) ; Step S4: Encode the historical dialogue to obtain the encoded memory vector m. h ; Step S5, represent the encoded state s (1:t) and the encoded memory vector m h By combining them, we obtain the fusion state representation. Step S6: Represent the fusion state according to the time attention mechanism. Enhancement is performed to obtain the enhanced state representation. Step S7, represent the current question and the enhanced state. Input the medical dialogue model to obtain the medical dialogue response. Step S2 includes the following sub-steps: Step S2-1: Supervised fine-tuning of the large medical language model based on the time-aware multi-turn dialogue dataset to obtain a fine-tuned medical dialogue model. Step S2-2: Based on the time-aware multi-turn dialogue dataset and the fine-tuned medical dialogue model, obtain the fine-tuned dialogue response; Steps S2-3: Construct a reward model based on the fine-tuned dialogue responses; Steps S2-4: Perform dynamic temporal reinforcement learning on the fine-tuned medical dialogue model based on the reward model to obtain the medical dialogue model. Step S1 includes the following sub-steps: Step S1-1: Preprocess the medical data to obtain preprocessed medical data; Step S1-2: Divide the preprocessed medical data into different records according to the patients, and sort the data in each record according to the time order to obtain the corresponding sorted records; Steps S1-3: Generate corresponding multi-turn dialogue records based on each of the sorted records; Steps S1-4: For each of the multi-turn dialogue records, timestamps are inserted according to the large language model to obtain the corresponding time-aware multi-turn dialogue records; Steps S1-5: Collect all the time-aware multi-turn dialogue records as the time-aware multi-turn dialogue dataset. In step S2-2, the fine-tuned dialogue response is a sequence of responses generated by the fine-tuned medical dialogue model based on the dialogue records of the time-aware multi-turn dialogue in the input time-aware multi-turn dialogue dataset, after being refined and sorted by medical experts. In steps S2-3, the reward model is trained by calculating the loss through the fine-tuned dialogue responses. The expression for the loss calculation is: In the formula, x is a response in the fine-tuned dialogue response, D is the set of all ranked pairs constructed from pairwise responses in the fine-tuned dialogue response, and y w and y l For the two responses in the ranked pair, and y w For the higher-ranked responses, y l For lower-ranked responses, r θ Let k be the scalar output of the reward model, and k be the total number of replies in the fine-tuning dialogue. In steps S2-4, the dynamic temporal reinforcement learning includes Q-value update, action selection, timeliness decay, and policy optimization. The expression for updating the Q value is: In the formula Q(s) t (a; θ) is the action value function, used to estimate the action value at time step t in state s. t The expected reward when action 'a' is taken and the policy parameter is θ, where α is the learning rate and γ is the discount factor. For the next time step t+1 and all possible actions a ′ In the mean, the maximum Q value that can be obtained, R t The reward obtained at time t according to the reward model. The expression for the action selection is: In the formula a t+1 To determine the optimal action at the next time step t+1, To assess the current state The relative value of taking action 'a', Argmax a To find the relative value among all possible actions a The biggest operation The expression for the time-related decay is: In the formula, β is the decay factor, and Δt is the time difference between now and the initial event. The raw reward given to the model's response at a specific time step t. To take into account the time elapsed after the event, the original reward... The new reward after time decay adjustment. The expression for the strategy optimization is: In the formula The advantage function of the previously deployed strategy, clip is the clipping function, ∈ is the clipping parameter, π θ (a|s) represents the probability of taking action a given state s under the current policy parameter θ. For the previous policy parameter θ old Given the same state s, the probability of taking the same action a.

2. The time-aware enhanced response generation method for medical dialogue according to claim 1, characterized in that: in, In step S3, the encoded state representation s is generated. (1:t) The process expression is: it is (1:t) =Emb(d (1:t) )+PE((1:t)), S (1:t) =TE(e (1:t) ), In the formula, Emb, PE, and TE represent the embedding operation, the positional encoding operation, and the processing via the Transformer encoder, respectively, and d (1:t) For the dialogue sequence, e (1:t) For the embedded representation of the dialogue sequence, PE((1:t)) is a positional encoding function that provides a unique representation of each time in the dialogue sequence.

3. The time-aware enhanced response generation method for medical dialogue according to claim 1, characterized in that: in, In step S4, the encoded memory vector m h The calculation expression is: m h =TE(d h ), In the formula d h For the aforementioned historical dialogue, TE is processed via the Transformer encoder. In step S5, the fusion state represents The calculation expression is:

4. The time-aware enhanced response generation method for medical dialogue according to claim 1, characterized in that: in, In step S6, the enhanced state representation is generated. The expression is: In the formula α i W represents the attention weight for the i-th historical event. a b is the weight matrix used for attention calculation. a v is the bias term used for attention calculation. i Let i represent the state of the i-th historical event.

5. The time-aware enhanced response generation method for medical dialogue according to claim 1, characterized in that: in, In step S2-1, the expression for the supervised fine-tuning loss function is: In the formula For supervised fine-tuning loss, CrossEntropy is the cross-entropy loss function. The large-scale medical language model is based on the question D of the j-th dialogue in the time-aware multi-turn dialogue records in the input time-aware multi-turn dialogue dataset. j and the corresponding enhanced state representation The predicted response obtained, Res, is the response corresponding to the j-th dialogue in the time-aware multi-turn dialogue record, ∑ j The medical large-scale language model is trained using all the time-aware multi-turn dialogue records in the time-aware multi-turn dialogue dataset.

6. A time-aware enhanced response generation device for medical dialogue, based on the time-aware enhanced response generation method for medical dialogue according to any one of claims 1-5, used to generate corresponding medical dialogue responses based on a dialogue sequence including historical dialogues and the current question, characterized in that, include: A data input module is used for the user to input the dialogue sequence; The first encoding module is used to encode the dialogue sequence to obtain an encoded state representation s. (1:t) ; The second encoding module is used to encode the historical dialogue to obtain an encoded memory vector m. h ; The fusion module is used to convert the encoded state representation s (1:t) and the encoded memory vector m h By combining them, we obtain the fusion state representation. An enhancement module is used to represent the fusion state according to a temporal attention mechanism. Enhancement is performed to obtain the enhanced state representation. The response generation module includes a medical dialogue model, used to generate responses based on the current question and the enhanced state representation. Received the medical dialogue response, The medical dialogue model is trained on an existing large-scale medical language model using a time-aware multi-turn dialogue dataset constructed from existing medical data. The training process includes the following sub-steps: Step S2-1: Supervised fine-tuning of the large medical language model based on the time-aware multi-turn dialogue dataset to obtain a fine-tuned medical dialogue model. Step S2-2: Based on the time-aware multi-turn dialogue dataset and the fine-tuned medical dialogue model, obtain the fine-tuned dialogue response; Steps S2-3: Construct a reward model based on the fine-tuned dialogue responses; Step S2-4: Perform dynamic temporal reinforcement learning on the fine-tuned medical dialogue model according to the reward model to obtain the medical dialogue model.

Images