Method and apparatus for medical question summarization based on reorderer

Abstract

The application discloses a medical question abstract generation method and device based on a reorderer, a storage medium and an electronic device, and belongs to the fields of artificial intelligence and natural language processing.The technical problem to be solved by the application is to automatically generate a short abstract for a long patient health problem.The technical scheme adopted is as follows: ① a medical question abstract generation method based on a reorderer, the method comprising the following steps: S1, constructing a medical question abstract generation dataset;S2, constructing and training a medical question abstract generation model;S3, constructing and training a reorderer model;S4, using the medical question abstract generation model and the reorderer model for reasoning.② a medical question abstract generation device based on a reorderer, the device comprising: a medical question abstract generation dataset construction unit, a medical question abstract generation model construction and training unit, a reorderer model construction and training unit, and a medical question abstract generation model and reorderer model reasoning unit.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, specifically to a method and apparatus for generating medical problem summaries based on a reorderer. Background Technology

[0002] With the rapid development of the internet, the healthcare industry has actively integrated with it, resulting in a large number of online applications that benefit both patients and reduce the workload of doctors. Among these, online medical Q&A communities have developed very rapidly. Patients can post their medical-related questions in these online communities, and doctors can answer them online at their convenience. This model significantly reduces the steps involved in direct consultation between patients and doctors, avoiding the traditional hospital visits, registration, and queuing processes, lowering the barrier to medical consultation, and improving doctors' work efficiency. However, as the number of users asking questions online increases, the number of patients asking questions far exceeds the number of doctors answering them, and the rate of increase in questions far outpaces the rate of response. This leads to a rapid decline in the patient experience of online medical Q&A. To address this, many researchers have proposed various automated medical question-answering systems based on natural language processing technology. Most of these automated medical question-answering systems are based on text matching technology, that is, selecting matching answers from existing question-and-answer databases for patient questions and recommending them to patients; this can quickly meet patient needs while reducing the workload of doctors. However, the medical and health questions raised by patients often contain a lot of useless peripheral information, such as everyday language or patient medical history; moreover, patients often lack medical expertise, and their descriptions of medical problems may contain non-professional terms that differ from those used by doctors. These factors make it difficult for text-matching-based automated medical question-answering systems to find the correct answers to such questions in the database, severely limiting the development of automated medical question-answering systems. In this context, automated medical question-answering systems urgently need a technology that can summarize lengthy patient medical questions into concise, refined, and easily searchable common questions—that is, medical question summary generation technology.

[0003] In recent years, many pre-trained sequence-to-sequence language models and methods have been applied to medical question summarization. Common sequence-to-sequence models for medical question summarization include T5, PEGASUS, ProphetNet, and BART, with the BART-based method showing the best performance. Currently, a trend in the development of medical question summarization technology is to enhance sequence-to-sequence models using various specific methods. Common methods include external knowledge-based reinforcement, transfer learning-based methods, multi-task learning-based methods, reinforcement learning-based methods, and contrastive learning-based methods. These methods have achieved good performance in medical question summarization tasks, but they all ignore the inconsistency between the objective function and evaluation metrics of sequence-to-sequence models. This means that the unique summary generated by these methods may not be the best summary in the decoding search space. To bridge this gap, it is necessary to expand the decoding search space of sequence-to-sequence models to obtain more candidate summaries, then evaluate the quality of these candidate summaries using appropriate methods, and re-rank the candidate summaries to select the best summary. Summary of the Invention

[0004] To address the shortcomings of current medical question summary generation methods, this invention aims to provide a method and apparatus for generating medical question summaries based on a reorderer. It addresses how to use natural language processing technology to extract concise and clear question summaries from lengthy patient medical questions, facilitating retrieval and matching in medical question-and-answer systems. This overcomes the limitation of existing medical question-and-answer systems in answering lengthy questions, thereby improving the processing capabilities of medical question-and-answer systems. The method and apparatus propose a reorderer-based medical question summary generation model architecture, which mainly consists of a sequence-to-sequence model, a candidate summary generation module, and a reorderer. The sequence-to-sequence model is responsible for encoding and decoding the input medical question; the candidate summary generation module generates candidate summaries for the medical question; and the reorderer evaluates the quality of the candidate summaries and reorders them.

[0005] The technical objective of this invention is achieved as follows: a method for generating medical problem summaries based on a reorderer, comprising the following steps:

[0006] Step S1, constructing a medical question summary generation dataset: First, it is necessary to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to form a medical question summary generation dataset, and then split it into a training dataset, a validation dataset and a test dataset;

[0007] Step S2 involves building and training a medical problem summary generation model. The main operations include: building a sequence-to-sequence model, building a cross-entropy loss function, optimizing the training of the sequence-to-sequence model, and generating candidate summaries for medical problems.

[0008] Step S3: Construct and train the reorderer model. The main operations include: constructing the reorderer model, constructing the ranking loss function, and optimizing the training of the reorderer model.

[0009] Step S4 involves reasoning using a medical question summary generation model and a reorderer model. The main operations include: generating candidate summaries for the medical question, encoding the medical question and its candidate summaries, calculating text semantic similarity and reordering them, and selecting the final summary through a confidence gating mechanism.

[0010] Preferably, the construction and training of the medical problem summary generation model is as follows:

[0011] The main operations for building and training a medical problem summary generation model include: building a sequence-to-sequence model, building a cross-entropy loss function, optimizing the training of the sequence-to-sequence model, and generating candidate summaries for medical problems.

[0012] The construction of the sequence-to-sequence model specifically involves: first, constructing the structure of the sequence-to-sequence model, and then loading the pre-trained sequence-to-sequence model weights and parameter configuration file.

[0013] The construction of the cross-entropy loss function is specifically as follows: For the constructed sequence-to-sequence model, its standard training framework is maximum likelihood estimation; for a specific training sample {CHQ,FAQ} in the loaded training dataset, where CHQ represents a medical question, denoted as Q, and FAQ represents a reference summary, denoted as S; maximum likelihood estimation is equivalent to minimizing l tokens {s1, ..., s1} in the reference summary S. j ,...,s l The sum of the negative log-likelihoods of} is used to optimize the following cross-entropy loss function:

[0014]

[0015] Among them, s * This refers to the tokens currently generated by the model; S ＜j This refers to a predefined starting word s0 and a current word s j The lexical table formed by the previous lexical units, i.e. {s0,s1,…,s j-1};

[0016] p true This represents a one-hot encoded distribution within the standard maximum likelihood estimation framework; θ refers to the parameters of f. The probability distribution is caused by the parameters of f; f is a model function. Given a medical problem Q, the goal of the medical problem summarization task is to learn a function f to generate a summary S of the medical problem Q.* The specific formula is as follows:

[0017] S * ←f(Q)

[0018] The optimized sequence-to-sequence model training specifically involves using Adam as the optimization algorithm, setting the learning rate of the Adam optimizer to 1e-5, and using the constructed cross-entropy loss function on the loaded training dataset to optimize and train the constructed sequence-to-sequence model.

[0019] The process of generating candidate summaries for medical questions involves the following steps: The trained sequence is fed into a sequence model, denoted as g(·); for a specific sample {CHQ,FAQ} in the loaded training dataset, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S); g(·) is used to generate a probability distribution D for Q, using the following formula:

[0020] D = g(Q)

[0021] Use a beam search decoding algorithm to generate a set of candidate summaries for a given medical problem Q.

[0022]

[0023] Where BeamSearch(·) refers to the beam search algorithm, and n represents the number of candidate summaries; the ROUGE-L F1 score from the ROUGE scoring method is used to calculate the candidate summary set. Candidate summary C i The score between the candidate abstract and the reference abstract S, and the scored candidate abstract set are represented as follows:

[0024]

[0025] in, They are sorted in descending order based on their scores; samples with scores are used as supervision signals for optimizing the constructed reorderer model; it is important to note that these scores are only used in the training process and not in the inference process, because the labels, i.e., the reference summary S, cannot be used during inference.

[0026] More preferably, the construction and training of the reorderer model is specifically as follows: The main operations of constructing and training the reorderer model include: constructing the reorderer model, constructing the ranking loss function, and optimizing the training of the reorderer model.

[0027] The construction of the reorderer model specifically involves: for a specific training sample {CHQ,FAQ} in the loaded training dataset, where CHQ represents a medical question (Q) and FAQ represents a reference summary (S); using the trained sequence-to-sequence model to generate a set of candidate summaries for the medical question Q and scoring each candidate summary to obtain a scored set of candidate summaries. The core idea of ​​the text semantic similarity-based reorderer model is to calculate Q and The cosine similarity between them is then used to measure the similarity. The quality; using the pre-trained language model RoBERTa as the encoder of the reorderer model for Q, S and Encode them to obtain their embedding representation E Q E S and

[0028] E Q =embedding(Q)

[0029] E S =embedding(S)

[0030]

[0031] Where i = 1, 2, ..., n, n is the number of candidate summaries generated; embedding(·) is a function that uses the encoder to obtain the text embedding representation.

[0032] The construction of the ranking loss function specifically involves: for the constructed reorderer model, constructing a ranking loss function to enhance its effect on evaluating the quality of candidate summaries; for a specific training sample {CHQ,FAQ} in the loaded training dataset, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S); using the trained sequence-to-sequence model to generate a set of candidate summaries for the medical question Q and scoring each candidate summary to obtain a scored set of candidate summaries. The encoder using the reorderer model obtains Q, S, and The embedding representation of E Q E S and The formula for the constructed ranking loss function is as follows:

[0033]

[0034] Where i = 1, 2, ..., n, and n is the number of candidate summaries; λ ij= (ji)*λ is the sorting interval, where λ is a hyperparameter controlling the size of the interval; max(x,y) returns the maximum value between x and y; sim(·) refers to the calculation of cosine similarity, where sim(E Q E S For example, the specific calculation formula is as follows:

[0035]

[0036] The calculation method is similar and will not be repeated here.

[0037] The optimized reorderer model training is specifically as follows: Adam is used as the optimization algorithm, the initial learning rate of the Adam optimizer is set to 2e-7, the maximum learning rate is set to 2e-3, and the constructed reorderer model is optimized and trained on the loaded training dataset using the constructed ranking loss function.

[0038] More preferably, the reasoning using the medical problem summary generation model and the reorderer model is specifically as follows:

[0039] Inference is performed using a medical question summary generation model and a reorderer model. The main operations include: generating candidate summaries for medical questions, encoding medical questions and their candidate summaries, calculating text semantic similarity and reordering, and selecting the final summary through a confidence gating mechanism.

[0040] The process of generating candidate summaries for medical questions involves the following steps: The trained sequence is fed into a sequence model, denoted as g(·); for a specific sample {CHQ,FAQ} in the loaded validation or test dataset, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S); g(·) is used to generate a probability distribution D for Q, using the following formula:

[0041] D = g(Q)

[0042] Use a beam search decoding algorithm to generate a set of candidate summaries for a given medical problem Q.

[0043]

[0044] Where BeamSearch(·) refers to the beam search algorithm, and n represents the number of candidate summaries, which is set to 16.

[0045] The encoding of the medical question and its candidate summary specifically involves: for a specific sample {CHQ,FAQ} in the loaded validation or test dataset, where CHQ represents the medical question, denoted as Q; and for the candidate summary of Q, denoted as... The encoder of the trained reorderer model is used to pair Q and The candidate summaries in the dataset are encoded to obtain the medical question and its candidate summary embedding representation E. Q and

[0046] E Q =embedding(Q)

[0047]

[0048] Where i = 1, 2, ..., n, n is the number of candidate summaries generated; embedding(·) is a function that uses the encoder to obtain the text embedding representation.

[0049] The calculation of text semantic similarity and reordering specifically involves: using the embedded representation E of the obtained medical question and its candidate summary. Q and Calculate the textual semantic similarity between each candidate abstract and its medical question, and then select the candidate abstract C with the highest score based on the textual semantic similarity. max The specific formula is as follows:

[0050]

[0051] in, It is a calculation and E Q The cosine similarity between them is calculated using the following formula:

[0052]

[0053] argmax(·) is a function that selects the candidate summary with the highest text semantic similarity score.

[0054] The selection of the final summary through the confidence gating mechanism specifically involves: selecting the candidate summary C with the highest score. max and the generated candidate summary set The C1 value is fed into the confidence gating to obtain the final summary C. final The specific formula for confidence gating is as follows:

[0055]

[0056] in, It is to calculate C max Embedded representation The embedding representation of E and Q Q The cosine similarity between them is calculated using the following formula:

[0057]

[0058] and The calculation method is similar and will not be repeated here; η is the confidence threshold, which is a hyperparameter, and the specific formula for its value is as follows:

[0059] η = [λ × 10 / 2, λ × 10]

[0060] Here, λ is a hyperparameter in the ranking loss function.

[0061] More preferably, the specific steps for constructing the medical problem summary to generate the dataset are as follows:

[0062] First, we need to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to form a medical question summary generation dataset. Then, we need to split it into training dataset, validation dataset and test dataset.

[0063] A medical problem summary generation apparatus based on a reorderer, the apparatus comprising:

[0064] The medical problem summary generation dataset building unit is responsible for preprocessing the original dataset and splitting it into training dataset, validation dataset, and test dataset.

[0065] The medical problem summary generation model building and training unit is responsible for building a sequence-to-sequence model, training the model's summary generation capability using the cross-entropy loss function, and generating candidate summaries for the reorderer model.

[0066] The reorderer model building and training unit is responsible for building the reorderer model and using the ranking loss to train the model to evaluate the ability of candidate summary quality.

[0067] The medical problem summary generation model and reorderer model inference unit is used to combine the trained medical problem summary generation model and reorderer model to build a complete inference process for medical problem summary generation.

[0068] An electronic device includes a storage medium and a processor; the storage medium stores a plurality of instructions, which are loaded by the processor to execute the steps of the above-described method for generating medical problem summaries based on a reorderer; the processor is used to execute the instructions in the storage medium.

[0069] The medical problem summary generation method and apparatus based on reorderer of the present invention have the following advantages:

[0070] (i) This invention uses a high-performance sequence-to-sequence model as the basic model and applies it to the generation of medical problem summaries, providing a basic guarantee for the generation of medical problem summaries;

[0071] (ii) This invention expands the decoding search space of sequence-to-sequence models, generates more candidate summaries for medical problems, and increases the probability of high-quality summaries appearing;

[0072] (III) This invention designs and trains a reorderer model that can measure the quality of candidate summaries based on text semantic similarity, thereby selecting higher quality candidate summaries;

[0073] (iv) This invention introduces a confidence gating mechanism for the reorderer model during the inference stage. This gating mechanism can ensure the original level of the output when the model confidence is insufficient, so that the model can achieve better performance.

[0074] (V) This invention integrates a sequence-to-sequence model and a reorderer model into the generation of medical problem summaries. The integrated model consists of two stages: the first stage uses the sequence-to-sequence model to generate candidate summaries for medical problems, and the second stage uses the reorderer model to evaluate the quality of the candidate summaries and reorder them. The integrated model can effectively improve the performance of the medical problem summary generation model. Attached Figure Description

[0075] The invention will be further described below with reference to the accompanying drawings.

[0076] Figure 1 This is a schematic diagram of a medical problem summary generation device based on a reorderer.

[0077] Figure 2 A schematic diagram of the framework for a medical problem summary generation model based on a reorderer.

[0078] Explanation of Special Terms

[0079] Medical questions: Medical questions refer to descriptions of problems with a patient's health status by the patient or their family. These descriptions are usually quite lengthy and often contain peripheral and irrelevant information that is not conducive to finding answers to the questions, such as the patient's medical history.

[0080] Reference summary of medical questions: A reference summary of medical questions refers to a concise summary of medical questions prepared by doctors or experts. It removes redundant information from the original medical questions, closely reflects the meaning of the medical questions, and facilitates the retrieval of answers from question-and-answer databases. Detailed Implementation

[0081] The medical problem summary generation method and apparatus based on reorderer of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] Example 1:

[0083] The overall model framework structure of this invention is as follows: Figure 2 As shown. By Figure 2 As can be seen, the main framework of this invention includes two stages, and the specific process is as follows: First, the medical problem is fed into the sequence model to generate the corresponding probability distribution; then, the probability distribution is decoded using the beam search decoding algorithm to obtain candidate summaries of the medical problem, and the medical problem and its candidate summaries are fed into the reorderer model encoder; the reorderer model encoder is responsible for encoding the medical problem and its candidate summaries to obtain the embedding representation of the medical problem and the embedding representation of its candidate summaries; next, the text semantic similarity between the obtained embedding representation of the medical problem and the embedding representation of its candidate summaries is calculated, and the candidate summaries are reordered in descending order according to the text semantic similarity score to obtain the candidate summary with the highest score; finally, the confidence gating is responsible for receiving the candidate summary with the highest score and the first summary of the candidate summaries obtained by the beam search decoding algorithm, and selecting and outputting the final summary through the confidence gating.

[0084] Example 2:

[0085] The medical problem summary generation method based on reorderer of the present invention includes the following specific steps:

[0086] S1. Constructing a Medical Question Summary Generation Dataset: First, it is necessary to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to form a medical question summary generation dataset. Then, it is divided into a training dataset, a validation dataset, and a test dataset.

[0087] S101. Download publicly available medical problem summaries from the internet to generate a dataset;

[0088] For example, there are many publicly available datasets online for generating medical problem summaries, such as the CHQ-Summ dataset. The data items in the CHQ-Summ dataset are shown below:

[0089]

[0090] Here, CHQ represents the patient's medical problem description, and FAQ is a reference summary written by a doctor or expert based on the patient's medical problem description; note that in the following description, CHQ will be abbreviated as medical problem, and FAQ will be abbreviated as reference summary of medical problem or reference summary.

[0091] S102. Preprocess the dataset obtained in S101 and split it into training dataset, validation dataset and test dataset;

[0092] For example, the CHQ-Summ dataset is collected from the Yahoo-L6 corpus. Therefore, you first need to download the corpus, then use the script provided by the authors of the CHQ-Summ dataset to extract the CHQ-Summ dataset from it, and convert it into a JSON file. The format of the processed JSON file is shown below:

[0093] [{“chq”:“A bit of advice for My grandfather who has stage 3melenoma?My grabdfather is 86years old and has had skin cancer for a few years.He hasrecently had surgery to remove 2basil cell carcinomas and just had one ofthose surgerys where they go in with ink and they test to see which lymphnodes the cancer has metastasized to.They removed 3lymph nodes and thenstopped.(they went in on his forhead)The doctors said that he can either goback in and continue removing lymph nodes,or he can go for another round ofthe drug,interferon.The other option that we're entertaining is if he doesn'topt for any of the choices,and just lets it go.He's a strong man,who walks onhis own...does everything for himself.However,the past few surgeries he's hadhave taken him a while to recover from.What do you suggest we do?Does anyoneknow the prognosis of stage 3melenoma,if he were to not treat it at all?I'minterested in any advice,or comments about the situation.Again he's 86yearsold,soon to be 87.Thanks!”,“faq”:“Is lymph node surgery or Interferon thebetter treatment option for an elderly person with stage 3melanoma?”},{“chq”:“Help for Physical Symptoms in Eating Disorder Recovery?I am recovering frombulimia and my entire body and digestive system are totally damaged and outof whack.Whenever I quit purging,the backlash is HORRIBLE-my face,hands,bellyand ankles swell up ridiculously and my digestive system just shuts down.Theworst part is that with the bloating(salt balance problem)comes HORRENDOUSmood swings,crying and excessive fatigue...can ANYONE tell me what is causingthis,and why stopping bulimia results in such terrible symptoms?please be asspecific as you can as to what kind of deficiency this indicates.And more importantly, how do I correct the imbalance? drinking more water seems logical for the bloating,but drinking more fluid of ANY kind(even pedialyte)justmakes me LOADS worse much faster! Is there a supplement or medication I need? Should I fast? Please help! ","faq":"Does recovering from bulimia cause thesymptoms such as bloating,swelling,mood swings,crying,and excessivefatigue? ”}].

[0094] After obtaining the dataset in JSON format, it is split into training dataset, validation dataset, and test dataset; the training dataset, validation dataset, and test dataset contain 800, 300, and 407 data entries, respectively.

[0095] S103. Load the dataset. Create a function to load the JSON format dataset and use this function to load the training dataset, validation dataset, and test dataset obtained in S102.

[0096] For example, in Python, the code described above would be implemented as follows:

[0097]

[0098]

[0099] The parameter dataset_dir is the storage path of the dataset on the computer; the train.json file contains the training dataset; the val.json file contains the validation dataset; and the test.json file contains the test dataset. After the data is loaded, each dataset list contains several pairs (CHQ, FAQ), where CHQ is the medical question and FAQ is the reference summary.

[0100] S2. Construct and train a medical question summary generation model. The main operations include: constructing a sequence-to-sequence model, constructing a cross-entropy loss function, optimizing the sequence-to-sequence model training, and generating candidate summaries for medical questions. Specific steps are as follows:

[0101] S201. Constructing a sequence-to-sequence model: First, construct the structure of the sequence-to-sequence model, and then load the pre-trained sequence-to-sequence model weights and parameter configuration file.

[0102] For example: In the Transformer of Huggingface, the code implementation of the above description is as follows:

[0103] config=BartConfig.from_pretrained(args.model)

[0104] model=BartForConditionalGeneration.from_pretrained(args.model)

[0105] Here, args.model is the model name, with a value of facebook / bart-large, indicating that the pre-trained sequence-to-sequence model used is the LARGE version of the BART model released by Facebook; config is the default parameter configuration table for the BART model; and model is the BART model itself, which loads the pre-trained parameter weights.

[0106] S202. Construct the cross-entropy loss function. For the sequence-to-sequence model constructed in S201, its standard training framework is maximum likelihood estimation. For a specific training sample {CHQ,FAQ} in the training dataset loaded in S103, where CHQ represents a medical question, denoted as Q, and FAQ represents a reference summary, denoted as S; maximum likelihood estimation is equivalent to minimizing the l tokens {s1, ..., s1} in the reference summary S. j ,...,s l The sum of the negative log-likelihoods of} is used to optimize the following cross-entropy loss function:

[0107]

[0108] Among them, s * This refers to the tokens currently generated by the model; S ＜j This refers to a predefined starting word s0 and a current word s j The lexical table formed by the previous lexical units, i.e. {s0,s1,…,s j-1};p true This represents a one-hot encoded distribution within the standard maximum likelihood estimation framework; θ refers to the parameters of f. The probability distribution is caused by the parameters of f; f is a model function. Given a medical problem Q, the goal of the medical problem summarization task is to learn a function f to generate a summary S of the medical problem Q. * The specific formula is as follows:

[0109] S * ←f(Q) (2)

[0110] S203. Optimize the sequence-to-sequence model training by using Adam as the optimization algorithm. The learning rate of the Adam optimizer is set to 1e-5. The sequence-to-sequence model constructed in S201 is optimized and trained on the training dataset loaded in S103 using the cross-entropy loss function constructed in S202.

[0111] For example, the simplified PyTorch code implementation of the optimizer described above and the sequence optimization to sequence model training process is as follows:

[0112]

[0113]

[0114] Here, `model.model.parameters()` represents the model's parameters, `lr` is the learning rate, `args.epoch` is the number of epochs for model training, `optimizer.zero_grad()` means to zero out the gradients, `loss` is the loss calculated using the cross-entropy loss function formula in S202 after one epoch of training, `loss.backward()` is used to calculate the gradients of all learnable parameters in the model, and `optimizer.step()` instructs the optimizer to update the parameters based on the calculated gradients.

[0115] S204. Generate candidate summaries for the medical questions. The sequence trained in S203 is transferred to the sequence model, denoted as g(·). For a specific sample {CHQ,FAQ} in the training dataset loaded in S103, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S), g(·) can be used to generate a probability distribution D for Q, as shown in the following formula:

[0116] D=g(Q) (3)

[0117] Use a beam search decoding algorithm to generate a set of candidate summaries for a given medical problem Q.

[0118]

[0119] Where BeamSearch(·) refers to the beam search algorithm, and n represents the number of candidate summaries; the ROUGE-L F1 score from the ROUGE scoring method is used to calculate the candidate summary set. Candidate summary C i The score between the candidate abstract and the reference abstract S, and the scored candidate abstract set are represented as follows:

[0120]

[0121] in, They are sorted in descending order based on their scores; samples with scores are used as supervision signals for optimization of the reorderer model built in S301; it should be noted that these scores are only used in the training process and not in the inference process, because the labels, i.e., the reference summary S, cannot be used during inference.

[0122] For example, in Transformer, the code implementation of the functions for generating candidate summaries and the scoring function described above is shown below:

[0123]

[0124]

[0125]

[0126] In this function, get_summaries() is the function that generates candidate summaries, and cal_score() is the function that scores the candidate summaries.

[0127] S3. Construct and train the reorderer model. The main operations include: constructing the reorderer model, constructing the ranking loss function, and optimizing the reorderer model training. The specific steps are as follows:

[0128] S301. Construct a reordering model. For a specific training sample {CHQ,FAQ} in the training dataset loaded in S103, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S), use the method in S204 to generate a set of candidate summaries for the medical question Q and score each candidate summary to obtain a scored set of candidate summaries. The core idea of ​​the text semantic similarity-based reorderer model is to calculate Q and The cosine similarity between them is then used to measure the similarity. The quality; using the pre-trained language model RoBERTa as the encoder of the reorderer model for Q, S and Encode them to obtain their embedding representation E Q E S and ECi *:

[0129]

[0130] Where i = 1, 2, ..., n, n is the number of candidate summaries generated; embedding(·) is a function that uses the encoder to obtain the text embedding representation;

[0131] For example, in Transformer, the code implementation described above would be as follows:

[0132] self.encoder=RobertaModel.from_pretrained(encoder)

[0133] self.pad_token_id=pad_token_id

[0134] input_mask=text_id! =self.pad_token_id

[0135] out=self.encoder(text_id, attention_mask=input_mask)[0]

[0136] doc_emb = out[:,0,:]

[0137] input_mask=summary_id! =self.pad_token_id

[0138] out=self.encoder(summary_id, attention_mask=input_mask)[0]

[0139] summary_emb = out[:,0,:]

[0140] input_mask=candidate_id! =self.pad_token_id

[0141] out=self.encoder(candidate_id, attention_mask=input_mask)[0]

[0142] candidate_emb=out[:,0,:].view(batch_size,candidate_num,-1)

[0143] Here, `encoder` is the encoder type, and its value is `roberta-base`, indicating that the base version of the pre-trained language model RoBERTa provided by HuggingFace is used; `doc_emb` represents the embedding representation of the medical question Q, `summary_emb` represents the embedding representation of the reference summary S, and `candidate_emb` represents the candidate summary. Embedded representation;

[0144] S302. Construct a ranking loss function. For the reorderer model constructed in S301, construct a ranking loss function to enhance its effect on evaluating the quality of candidate summaries. For a specific training sample {CHQ,FAQ} in the training dataset loaded in S103, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S), use the method in S204 to generate a set of candidate summaries for the medical question Q and score each candidate summary to obtain a scored set of candidate summaries. The encoder in the reorderer model of S301 is used to obtain Q, S, and The embedding representation of E Q E S and The formula for the constructed ranking loss function is as follows:

[0145]

[0146] Where i = 1, 2, ..., n, and n is the number of candidate summaries; λ ij = (ji)*λ is the sorting interval, where λ is a hyperparameter controlling the size of the interval; max(x,y) returns the maximum value between x and y; sim(·) refers to the calculation of cosine similarity, where sim(E Q E S For example, the specific calculation formula is as follows:

[0147]

[0148] The calculation method is similar and will not be repeated here;

[0149] S303. Optimize the training of the reorderer model. Use Adam as the optimization algorithm. Set the initial learning rate of the Adam optimizer to 2e-7 and the maximum learning rate to 2e-3. Use the ranking loss function built in S302 to optimize and train the reorderer model built in S301 on the training dataset loaded in S103.

[0150] For example, a simplified PyTorch code implementation of the optimizer and optimized reorderer model training process described above is as follows:

[0151]

[0152] In this code, `reranker.parameters()` represents the encoder parameters, `init_lr` is the initial learning rate (which gradually increases as the model trains until it reaches its maximum), `args.epoch` is the number of training epochs, `optimizer.zero_grad()` resets the gradients to zero, `loss` is the loss calculated using the ranking loss function formula in S302 after one training epoch, `loss.backward()` calculates the gradients of all learnable parameters in the model, and `optimizer.step()` instructs the optimizer to update the parameters based on the calculated gradients.

[0153] S4. Inference is performed using a medical question summary generation model and a reordering model. The main operations include: generating candidate summaries for the medical question, encoding the medical question and its candidate summaries, calculating text semantic similarity and reordering, and selecting the final summary through a confidence gating mechanism. The specific steps are as follows:

[0154] S401. Generate candidate summaries for medical questions. The sequence trained in S203 is transferred to the sequence model, denoted as g(·). For a specific sample {CHQ,FAQ} in the validation or test dataset loaded in S103, where CHQ represents the medical question (Q) and FAQ represents the reference summary (S), g(·) can be used to generate a probability distribution D for Q, as shown in the following formula:

[0155] D=g(Q) (9)

[0156] Use a beam search decoding algorithm to generate a set of candidate summaries for a given medical problem Q.

[0157]

[0158] Where BeamSearch(·) refers to the beam search algorithm, and n represents the number of candidate summaries, which is set to 16;

[0159] S402. Encode the medical problem and its candidate summaries. For a specific sample {CHQ,FAQ} in the validation or test dataset loaded in S103, where CHQ represents the medical problem, denoted as Q; for the candidate summaries of Q obtained in S401, denoted as... The encoder of the reorderer model trained in S303 is used to pair Q and The candidate summaries in the dataset are encoded to obtain the medical question and its candidate summary embedding representation E. Q and

[0160]

[0161] Where i = 1, 2, ..., n, n is the number of candidate summaries generated; embedding(·) is a function that uses the encoder to obtain the text embedding representation;

[0162] S403. Calculate text semantic similarity and reorder the texts, using the embedding representations E of the medical questions and their candidate summaries obtained in S402. Q and Calculate the textual semantic similarity between each candidate abstract and its medical question, and then select the candidate abstract C with the highest score based on the textual semantic similarity. max The specific formula is as follows:

[0163]

[0164] in, It is a calculation and E Q The cosine similarity between them is calculated using the following formula:

[0165]

[0166] argmax(·) is a function that selects the candidate summary with the highest text semantic similarity score;

[0167] S404. Select the final abstract using a confidence gating mechanism, choosing the candidate abstract C with the highest score obtained in S403. max and the candidate summary set generated in S402 The C1 value is fed into the confidence gating to obtain the final summary C. final The specific formula for confidence gating is as follows:

[0168]

[0169] in, It is to calculate C max Embedded representation The embedding representation of E and Q Q The cosine similarity between them is calculated using the following formula:

[0170]

[0171] and The calculation method is similar and will not be repeated here; η is the confidence threshold, which is a hyperparameter, and the specific formula for its value is as follows:

[0172] η=[λ×10 / 2,λ×10] (16)

[0173] Here, λ is a hyperparameter in the sorting loss function in S302.

[0174] The model proposed in this invention achieves better results than current state-of-the-art models on the CHQ-Summ dataset. The experimental results are compared in the table below:

[0175] Method R1 R2 RL T5 35.17 18.87 32.33 PEGASUS 35.89 18.86 33.27 ProphetNet 40.46 22.80 38.13 BART 42.40 24.00 39.79 Our Model 44.00 25.47 41.15

[0176] The model of this invention was compared with existing models. The first four lines show the experimental results of existing models [Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer, The Journal of Machine Learning Research; Zhang J, Zhao Y, Saleh M, et al. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization, International Conference on Machine Learning; Qi W, Yan Y, Gong Y, et al. ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training, EMNLP 2020; Lewis M, Liu Y, Goyal N, et al. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension, ACL2020.]. The last line shows the experimental results of the model of this invention. R1, R2, and RL represent the F1 scores of ROUGE (Recall-Oriented Understudy for Gisting Evaluation) for ROUGE-1, ROUGE-2, and ROUGE-L, respectively. This shows that the present invention has made a significant improvement over existing models.

[0177] Example 3:

[0178] As attached Figure 1 As shown, the medical problem summary generation device based on reorderer in Embodiment 2 includes: a medical problem summary generation dataset construction unit, a medical problem summary generation model construction and training unit, a reorderer model construction and training unit, and a medical problem summary generation model and reorderer model inference unit, which respectively implement the functions of steps S1, S2, S3, and S4 in the reorderer-based medical problem summary generation method. The specific functions of each unit are as follows:

[0179] The medical problem summary generation dataset building unit is responsible for preprocessing the original dataset and splitting it into training dataset, validation dataset, and test dataset.

[0180] The medical problem summary generation model building and training unit is responsible for building a sequence-to-sequence model, training the model's summary generation capability using the cross-entropy loss function, and generating candidate summaries for the reorderer model.

[0181] The reorderer model building and training unit is responsible for building the reorderer model and using the ranking loss to train the model to evaluate the ability of candidate summary quality.

[0182] The medical problem summary generation model and reorderer model inference unit is used to combine the trained medical problem summary generation model and reorderer model to build a complete inference process for medical problem summary generation.

[0183] Example 4:

[0184] An electronic device, characterized in that the electronic device includes a storage medium based on Embodiment 2 and a processor; the storage medium stores a plurality of instructions, which are loaded by the processor to execute the steps of the medical problem summary generation method based on reorderer of Embodiment 2; the processor is used to execute the instructions in the storage medium.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating medical problem summaries based on a reorderer, characterized in that, The method includes the following steps: Step S1, constructing a medical question summary generation dataset: First, it is necessary to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to form a medical question summary generation dataset, and then split it into a training dataset, a validation dataset and a test dataset; Step S2 involves building and training a medical problem summary generation model. The main operations include: building a sequence-to-sequence model, building a cross-entropy loss function, optimizing the training of the sequence-to-sequence model, and generating candidate summaries for medical problems. Step S3: Construct and train the reorderer model. The main operations include: constructing the reorderer model, constructing the ranking loss function, and optimizing the training of the reorderer model. Step S4 involves reasoning using a medical problem summary generation model and a reorderer model. The main operations include: generating candidate summaries for the medical problem, encoding the medical problem and its candidate summaries, calculating text semantic similarity and reordering them, and selecting the final summary through a confidence gating mechanism. The steps S1, constructing the medical problem summary generation dataset, and S2, constructing and training the medical problem summary generation model, are as follows: Step S1: Construct a medical problem summary and generate a dataset: S101. Download publicly available medical problem summaries from the internet to generate a dataset; S102. Preprocess the dataset obtained in S101 and split it into training dataset, validation dataset and test dataset; S103. Load the dataset. Create a function to load the JSON format dataset and use this function to load the training dataset, validation dataset, and test dataset obtained in S102. Step S2: Construct and train a medical problem summary generation model: S201. Constructing a sequence-to-sequence model: First, construct the structure of the sequence-to-sequence model, and then load the pre-trained sequence-to-sequence model weights and parameter configuration file. S202, Construct the cross-entropy loss function; S203. Optimize sequence-to-sequence model training. Use Adam as the optimization algorithm. On the training dataset loaded in S103, use the cross-entropy loss function constructed in S202 to optimize and train the sequence-to-sequence model constructed in S201. S204. Generate candidate summaries for medical questions, and convert the sequences trained in S203 into a sequence model. For a specific sample {CHQ, FAQ} in the training dataset loaded by S103, where CHQ represents a medical question, denoted as... FAQ stands for Reference Summary, denoted as ;use It can be Generate a probability distribution The specific formula is as follows: ； Generate a given medical question using a beam search decoding algorithm. A set of candidate abstracts : ； in This refers to the beam search algorithm. The number of candidate abstracts is represented; the set of candidate abstracts is calculated using the ROUGE-L F1 index in the ROUGE score. Candidate abstracts and reference abstract The scores between the two groups, and the set of candidate summaries after scoring, are represented as follows: ； in, They are sorted in descending order based on their scores; samples with scores are used as supervision signals for optimizing the reorderer model built in S301; it is important to note that these scores are only used in the training process, not in the inference process, because labels, i.e., reference summaries, cannot be used during inference. ; Step S3 involves constructing and training the reorderer model, as detailed below: S301. Construct the reordering model. For a specific training sample {CHQ,FAQ} in the training dataset loaded in S103, where CHQ represents a medical question, denoted as... FAQ stands for Reference Summary, denoted as Using the methods in S204 for medical problems Generate a set of candidate summaries and score each candidate summary to obtain a scored set of candidate summaries. The core idea of ​​the text semantic similarity-based reorderer model is to calculate... and The cosine similarity between them is then used to measure the similarity. The quality; using the pre-trained language model RoBERTa as the encoder of the reorderer model. , and Encode them to obtain their embedded representations. , and : ； in, , It is the number of candidate summaries generated; It is a function that uses an encoder to obtain the embedded representation of text; S302. Construct the ranking loss function; S303. Optimize the reorderer model training. Use Adam as the optimization algorithm. On the training dataset loaded in S103, use the ranking loss function constructed in S302 to optimize and train the reorderer model constructed in S301.

2. The method for generating medical problem summaries based on reorderers according to claim 1, characterized in that, Step S202 involves constructing the cross-entropy loss function, as follows: For the sequence-to-sequence model constructed in S201, the standard training framework is maximum likelihood estimation; for a specific training sample {CHQ, FAQ} in the training dataset loaded in S103, where CHQ represents a medical question, denoted as... FAQ stands for Reference Summary, denoted as Maximum likelihood estimation is equivalent to minimizing the reference summary. In each word element The sum of the negative log-likelihoods is used to optimize the following cross-entropy loss function: ； in, This refers to the lexical units currently generated by the model; It refers to a predefined starting word. and the current word unit The lexicon table formed by the previous lexicons, i.e. ; A one-hot encoded distribution in the standard maximum likelihood estimation framework; It refers to The parameters, It is by The probability distribution caused by the parameters; It is a model function, given a medical problem. The goal of the medical problem summarization task is to learn a function. To generate medical questions Abstract .

3. The method for generating medical problem summaries based on reorderers according to claim 1, characterized in that, Step S302 involves constructing the ranking loss function, as follows: For the reorderer model constructed in S301, a ranking loss function is constructed to enhance its effectiveness in evaluating the quality of candidate summaries; for a specific training sample {CHQ, FAQ} in the training dataset loaded in S103, where CHQ represents a medical question, denoted as... FAQ stands for Reference Summary, denoted as ; Using the method in S204 for medical issues Generate a set of candidate summaries and score each candidate summary to obtain a scored set of candidate summaries. ; The encoder of the reorderer model in S301 is used to obtain , and Embedded representation , and The formula for the constructed ranking loss function is as follows: ； in, , It represents the number of candidate abstracts; It is the corresponding interval for sorting. It is a hyperparameter that controls the size of the corresponding interval; Indicates return and The maximum value in; This refers to the calculation of cosine similarity, in order to... For example, the specific calculation formula is as follows: ； , The calculation method is similar and will not be repeated here.

4. The method for generating medical problem summaries based on reorderers according to claim 1, characterized in that, Step S4 uses a medical problem summary generation model and a reorderer model for inference, as detailed below: S401. Generate candidate summaries for medical problems, and convert the sequences trained in S203 into a sequence model. For a specific sample {CHQ, FAQ} in the validation or test dataset loaded in S103, where CHQ represents a medical question, denoted as... FAQ stands for Reference Summary, denoted as ;use It can be Generate a probability distribution The specific formula is as follows: ； Generate a given medical question using a beam search decoding algorithm. A set of candidate abstracts : ； in This refers to the beam search algorithm. The number of candidate summaries is set to 16. S402. Encode the medical question and its candidate summary. For a specific sample {CHQ, FAQ} in the validation or test dataset loaded in S103, where CHQ represents the medical question, denoted as... For the results obtained in S401 The candidate abstract, denoted as The encoder of the reorderer model trained in S303 is used to... and The candidate summaries are encoded to obtain an embedding representation of the medical question and its candidate summaries. and : ； in, , It is the number of candidate summaries generated; It is a function that uses an encoder to obtain the embedded representation of text; S403. Calculate the semantic similarity of the texts and re-rank them; S404. Select the final summary through a confidence gating mechanism.

5. The method for generating medical problem summaries based on reorderers according to claim 4, characterized in that, Step S403, which calculates text semantic similarity and re-ranks the texts, is as follows: Embedded representation of the medical question and its candidate summary obtained in S402 and The textual semantic similarity between each candidate abstract and its medical question is calculated, and then the candidate abstract with the highest score is selected based on the textual semantic similarity. The specific formula is as follows: ； in, It is a calculation and The cosine similarity between them is calculated using the following formula: ； It is a function that selects the candidate summary with the highest text semantic similarity score.

6. The method for generating medical problem summaries based on reorderers according to claim 4, characterized in that, Step S404 selects the final digest through a confidence gating mechanism, as follows: The candidate summary with the highest score obtained from S403 and the candidate summary set generated in S402 In The data is sent to the confidence gating system to obtain the final summary. The specific formula for confidence gating is as follows: ； in, It is a calculation Embedded representation and Embedded representation The cosine similarity between them is calculated using the following formula: ； and The calculation method is similar and will not be repeated here; This is the confidence threshold, a hyperparameter, and its specific value is determined by the following formula: ；； in, It is a hyperparameter in the sorting loss function in S302.

7. A medical problem summary generation apparatus based on a reorderer, implementing the medical problem summary generation method based on a reorderer as described in any one of claims 1 to 6, characterized in that, The device includes: The medical problem summary generation dataset building unit is responsible for preprocessing the original dataset and splitting it into training dataset, validation dataset, and test dataset; The medical problem summary generation model building and training unit is responsible for building a sequence-to-sequence model, training the model's summary generation ability using the cross-entropy loss function, and generating candidate summaries for the reorderer model; The reorderer model building and training unit is responsible for building the reorderer model and using the ranking loss to train the model to evaluate the quality of candidate summaries. The medical problem summary generation model and reorderer model inference unit is used to combine the trained medical problem summary generation model and reorderer model to build a complete inference process for medical problem summary generation.

8. An electronic device, characterized in that, The electronic device includes a storage medium and a processor; the storage medium stores a plurality of instructions, which are loaded by the processor to execute the steps of the medical problem summary generation method based on a reorderer as described in any one of claims 1 to 6; A processor is used to execute instructions stored in a storage medium.

Images