Medical question summary generation method and device based on contrastive learning

Abstract

The present invention discloses a method and device for generating medical problem summaries based on contrastive learning, which belongs to the fields of artificial intelligence and natural language processing. The technical problem to be solved by the present invention is how to use natural language processing technology to automatically generate short summaries for longer patient health problems, so as to facilitate the retrieval of common questions that match them in a standard question-and-answer database. The technical solutions adopted are: ① A method for generating medical problem summaries based on contrastive learning, the method comprising the following steps: S1, constructing a medical problem summary generation data set; S2, constructing difficult negative examples; S3, constructing a medical problem summary generation model; S4, training the medical problem summary generation model. ② A device for generating medical problem summaries based on contrastive learning, the device comprising: a medical problem summary generation data set construction unit, a difficult negative example construction unit, a medical problem summary generation model construction unit and a medical problem summary generation model training unit.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and natural language processing, and in particular to a method and device for generating medical problem summaries based on contrastive learning. Background Art

[0002] With the rapid development of the internet, the medical industry has actively integrated with it, resulting in the emergence of a large number of online applications that both provide convenience for patients and reduce the pressure on doctors. Among these, online medical Q&A communities have grown rapidly. Patients can post their medical questions in these communities, and doctors will answer their questions online at their convenience. This model significantly reduces the number of steps patients need to take to consult with doctors, eliminating the traditional process of visiting a hospital, registering, and queuing. This lowers the barrier to entry for medical consultation and improves doctors' efficiency. However, as the number of users asking questions online increases, the number of patients raising questions far exceeds the number of doctors answering them, and the rate of increase in questions far outstrips the rate of answers. This has led to a rapid decline in the patient experience of online medical Q&A. To address this issue, many researchers have proposed various automated medical Q&A systems based on natural language processing. These systems are mostly based on text matching technology, selecting matching answers from existing Q&A databases and recommending them to patients. This approach can quickly meet patient needs while reducing doctors' workload. However, medical questions raised by patients often contain a lot of useless peripheral information, such as everyday language or the patient's medical history. Furthermore, patients often lack medical expertise, and the medical questions they describe may include non-professional terms that differ from those used by doctors. These factors make it difficult for medical automatic question-answering systems based on text matching to match the correct answers to such questions in the database, severely limiting their development. In this context, medical automatic question-answering systems urgently need a technology that can summarize lengthy patient medical questions into concise, easy-to-search and match common questions—in other words, medical question summary generation technology.

[0003] In recent years, many neural network models and methods have been applied to medical question summary generation technology. Common neural network models used for medical question summary generation include pointer generation network, ProphetNet and BART, among which the method based on the BART model performs best. At present, a trend in the development of medical question summary generation technology is to use various specific methods to enhance neural network models. Common methods include data augmentation, reinforcement learning and contrastive learning. Methods based on data augmentation and reinforcement learning often lack attention to the focus of medical questions, which leads to the frequent focus errors and focus loss problems in the medical question summaries generated by such methods. Methods based on contrastive learning have improved these problems, but methods based on contrastive learning still face the challenge of how to effectively construct negative samples for contrastive learning. Summary of the Invention

[0004] The present invention addresses the shortcomings of current medical question summary generation methods. The technical task of the present invention is to provide a medical question summary generation method and device based on contrastive learning; to solve how to use natural language processing technology to refine lengthy patient medical questions into concise and clear question summaries to facilitate retrieval and matching by the medical question-answering system; to make up for the defect that the existing medical question-answering system cannot answer lengthy questions, and to improve the processing capacity of the medical question-answering system. The method and device propose a medical question summary generation model architecture based on contrastive learning, which is mainly composed of a sequence-to-sequence model, a contrastive learning module, and a difficult negative example generator. Among them, the sequence-to-sequence model is responsible for encoding and decoding the input medical questions; the contrastive learning module is responsible for adjusting the parameters of the sequence-to-sequence model encoder to achieve the purpose of enhancing the encoding representation of the input medical questions; the difficult negative example generator is responsible for providing high-quality negative examples for contrastive learning.

[0005] The technical task of the present invention is achieved in the following manner: a method for generating a medical question summary based on contrastive learning, the method comprising the following steps:

[0006] Step S1, constructing a medical question summary generation dataset: First, it is necessary to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to construct a medical question summary generation dataset, and then split it into a training dataset, a validation dataset, and a test dataset;

[0007] Step S2, constructing hard negative examples: First, traverse the reference abstracts of all medical questions in the training dataset and construct a medical entity dictionary; then, for each medical question reference abstract, identify the medical entities in it and perform similar replacement operations on these medical entities to obtain hard negative examples;

[0008] Step S3: Build a medical question summary generation model. The main operations include: loading the dataset, building a sequence-to-sequence model architecture and a medical question encoder, building a positive and negative sample encoder, building an encoder momentum update module, building a simple negative sample and its storage queue, and building an average pooling layer.

[0009] Step S4, training the medical problem summary generation model, uses the medical problem summary generation training data set obtained in step S1 to train the medical problem summary generation model constructed in step S3 to optimize the model parameters. The main steps include: constructing a loss function and optimizing model training; wherein the loss function consists of three parts: a cross-entropy loss function, a contrastive learning loss function for simple negative examples, and a contrastive learning loss function for difficult negative examples.

[0010] Preferably, the difficult negative examples are constructed as follows:

[0011] After obtaining the medical problem summary generation dataset, we first traverse the reference summaries of all medical problems in the training dataset to build a medical entity dictionary; then, we identify the medical entities in the reference summary of each medical problem and perform similar replacement operations on these medical entities to obtain difficult negative samples.

[0012] The construction of the medical entity dictionary specifically includes: traversing the reference abstracts of medical problems in all data pairs in the training data set, identifying the medical entities in the reference abstract of each medical problem, and forming the medical entity dictionary with the identified non-repeated medical entities.

[0013] The hard negative examples are generated as follows: for each reference summary of a medical problem containing a medical entity, the medical entities contained in the reference summary are first removed from the medical entity dictionary to obtain a candidate medical entity dictionary; L×X medical entities are then randomly selected from the candidate medical entity dictionary as candidate replacement entities, where L is a hyperparameter and X is the number of medical entities contained in the reference summary itself; the process is then repeated L×X times, each time replacing a medical entity in the reference summary with an entity in the candidate replacement entity; through the above process, each reference summary will eventually generate L×X hard negative examples.

[0014] It should be noted that if the reference summary does not contain medical entities or the medical entities therein are not correctly identified, the reference summary will not be used to generate hard negative examples.

[0015] More preferably, the sequence-to-sequence model architecture and medical question encoder are constructed as follows:

[0016] First, select a suitable sequence-to-sequence model. Then load the pre-trained sequence-to-sequence model and its parameter configuration file. The loaded sequence-to-sequence model contains an encoder, a generation module, and a decoder. Finally, use the encoder in the sequence-to-sequence model as the medical question encoder. The specific formula is as follows:

[0017] R' C =Encoder B (C)

[0018] Among them, Encoder B represents the medical problem encoder, C represents the input medical problem, R' C A coded representation representing a medical problem.

[0019] More preferably, the positive and negative sample encoder is constructed as follows:

[0020] The initial structure and parameters of the positive and negative sample encoder are exactly the same as those of the medical question encoder. Therefore, we only need to copy the structure and parameters of the medical question encoder to complete the construction. The specific formula is as follows:

[0021] R' S =Encoder S (S)

[0022] Among them, Encoder S represents the positive and negative sample encoder, S represents the positive or negative sample corresponding to the medical problem, R' S The encoded representation representing positive or negative examples.

[0023] More preferably, the encoder momentum update module is constructed as follows:

[0024] The encoder momentum update module uses momentum update to maintain consistency between the medical question encoder and the positive and negative sample encoders. That is, the parameters of the positive and negative sample encoders are slowly updated as the parameters of the medical question encoder change. The specific formula is as follows:

[0025] θ k ←mθ k +(1-m)θ q

[0026] Among them, θ k are the parameters of the positive and negative sample encoders, θ q are the parameters of the medical question encoder and m∈[0,1) is the momentum coefficient.

[0027] Preferably, the construction of simple negative examples and their storage queue are specifically as follows:

[0028] First, construct a simple negative example, then define the data structure of the queue and the dequeue and enqueue operations of the simple negative example in the queue.

[0029] The specific construction of simple negative examples is as follows: for a training data sample {CHQ1, FAQ1}, where CHQ1 is a medical question and FAQ1 is a reference summary corresponding to the medical question; CHQ1 should be semantically similar to its own reference summary FAQ1, and not similar to other CHQ1. i Reference Summary FAQ for (i≠1) i There is a big difference; therefore, FAQ1 can be used as a positive example of CHQ1, and other CHQ i Reference Summary FAQ i As a simple negative example for CHQ1.

[0030] More preferably, the construction of the average pooling layer is as follows:

[0031] The medical question encoder and positive and negative sample encoder are used to encode the medical question and positive and negative samples respectively to obtain their feature representations. The average pooling layer is responsible for extracting the mean features of these feature representations. The specific formula is as follows:

[0032] R C =AvgPooling(R' C )

[0033] R S =AvgPooling(R' S )

[0034] Among them, AvgPooling represents the average pooling layer; R' C Represents the use of medical question encoder Encoder B The encoded representation obtained by encoding the medical problem C; R' S Represents the use of positive and negative sample encoders K The encoded representation obtained by encoding the positive or negative sample S; R C and R S Represents R' C and R' S Representation after the average pooling layer.

[0035] More preferably, the training medical question summary generation model is as follows:

[0036] The medical question summary generation training dataset is used to train the medical question summary generation model to optimize the model parameters. The main steps include: constructing a loss function and optimizing model training; among them, the loss function consists of three parts: cross-entropy loss function, contrastive learning loss function for simple negative examples, and contrastive learning loss function for difficult negative examples.

[0037] The loss function is constructed specifically as follows: the loss function includes three parts, namely the cross entropy loss function, the contrastive learning loss function of simple negative examples, and the contrastive learning loss function of difficult negative examples.

[0038] Construct a cross entropy loss function. For sequence-to-sequence models, the standard training algorithm is maximum likelihood estimation. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , whose reference abstract is The token representation is Using the maximum likelihood estimation algorithm is equivalent to minimizing l words in The sum of the negative log-likelihood of , that is, optimizing the following cross entropy loss function:

[0039]

[0040] in, Refers to a pre-defined starting word and the current word The word table composed of the previous word, that is p represents a one-hot distribution in the standard maximum likelihood estimation framework; θ refers to the parameter of f, is the probability distribution caused by the parameters of f; f is a model function. Given a medical problem C, the goal of the medical problem summary task is to learn a function f to generate a reference summary S of medical problem C. The specific formula is as follows:

[0041] S←f(C)

[0042] Construct a contrastive learning loss function for simple negative examples. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , let its reference abstract be Let the reference summary of other medical problems be According to the construction and storage method of simple negative samples, Considered as C i A positive example, Considered as C i The simple negative example, the contrastive learning loss function of the simple negative example is defined as follows:

[0043]

[0044] in, Is a medical problem Ci The vector representation obtained after the medical question encoder and average pooling layer; It is C i Reference Summary The vector representation obtained after the positive and negative sample encoder and the average pooling layer; A reference summary of other medical issues The vector representation obtained after the positive and negative sample encoder and the average pooling layer; It is calculated and The cosine similarity of is as follows:

[0045]

[0046] τ is a temperature hyperparameter; Q is a simple negative sample storage queue, the size of which is a hyperparameter; and The calculation method is similar and will not be repeated here.

[0047] Construct a contrastive learning loss function for difficult negative examples. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , let its reference abstract be Will Considered as C i positive sample; let the hard negative sample be h, and regard it as C i The contrastive learning loss function of difficult negative examples is defined as follows:

[0048]

[0049] in, Is a medical problem C i The vector representation obtained after the medical question encoder and average pooling layer; It is C i Reference Summary The vector representation obtained after the positive and negative sample encoder and the average pooling layer; R h is the vector representation of the hard negative example h after passing through the positive and negative sample encoder and the average pooling layer; It is calculated and R h The cosine similarity of is as follows:

[0050]

[0051] τ is a temperature hyperparameter; H is a set for storing difficult negative examples, whose size depends on the number of difficult negative examples sampled during training, and its number is a hyperparameter; and The calculation method is similar and will not be repeated here.

[0052] Construct the total loss function, which is obtained by adding the cross entropy loss function, the contrastive learning loss function of simple negative examples, and the contrastive learning loss function of difficult negative examples. The specific formula is as follows:

[0053]

[0054] in, is the cross entropy loss function, is the contrastive learning loss function for simple negative examples, is the contrastive learning loss function for hard negative examples.

[0055] The optimization model training is specifically as follows: Adam is used as the optimization algorithm, the learning rate is set to 1e-5, and the medical question summary generation model is optimized and trained on the training data set.

[0056] When the model of this method has not been fully trained, it needs to be trained on the training data set to optimize the model parameters; when the model training is completed, the model can automatically generate a summary for the input medical question.

[0057] Preferably, the method of constructing a medical problem summary to generate a dataset is as follows:

[0058] First, we need to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to form a medical question summary generation dataset, and then divide it into a training dataset, a validation dataset, and a test dataset.

[0059] A device for generating a medical problem summary based on contrastive learning, comprising:

[0060] The medical problem summary generation dataset construction unit is responsible for preprocessing the original dataset and dividing the original dataset into training dataset, verification dataset and test dataset.

[0061] The difficult negative example construction unit is responsible for building a medical entity dictionary and constructing difficult negative examples for contrastive learning in the training dataset.

[0062] The medical question summary generation model building unit is used to build a dataset loading module, a sequence-to-sequence model architecture, a medical question encoder, a positive and negative sample encoder, an encoder momentum update module, simple negative samples and their storage queue, and an average pooling layer.

[0063] The medical problem summary generates a model training unit, which is used to construct the loss function and optimization function required in the model training process and complete the model training.

[0064] An electronic device includes a storage medium and a processor; the storage medium stores a plurality of instructions, the instructions are loaded by the processor to execute the steps of the above-mentioned medical problem summary generation method based on contrastive learning; the processor is used to execute the instructions in the storage medium.

[0065] The method and device for generating medical question summaries based on contrastive learning of the present invention have the following advantages:

[0066] (1) The present invention uses a high-performance sequence-to-sequence model as a basic model and applies it to the generation of medical problem summaries, providing a basic guarantee for the generation of medical problem summaries;

[0067] (2) The present invention uses two encoders to encode the medical question and its positive and negative examples respectively, and adopts momentum update to optimize the parameters of the two encoders, which saves computing resources and significantly alleviates the problem of parameter consistency loss that occurs when encoding with two different encoders;

[0068] (3) The present invention uses a queue to store simple negative examples. Combined with the characteristics of momentum update, it allows the queue size to be set to a large size, which allows the model to utilize more simple negative examples and improve the training effect of the model;

[0069] (4) This invention proposes a novel difficult negative example generator for contrastive learning. This generator considers medical entities in medical problems as problem focuses and randomly replaces these problem focuses with other medical entities to generate high-quality difficult negative examples for contrastive learning, thereby improving the effectiveness of contrastive learning.

[0070] (5) The present invention uses a large number of simple negative examples for contrastive learning, which enhances the encoding representation of the input medical problem and thus improves the overall quality of the generated medical problem summary;

[0071] (6) The present invention uses high-quality difficult negative examples for contrastive learning, which improves the model's attention to the focus of medical problems in the input medical problems, thereby alleviating the problems of focus errors and focus loss in the generated medical problem summaries. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The present invention will be further described below with reference to the accompanying drawings.

[0073] Figure 1 Schematic diagram of the structure of a medical question summary generation device based on contrastive learning.

[0074] Figure 2 Schematic diagram of the framework of the medical question summary generation model based on contrastive learning.

[0075] Explanation of special terms

[0076] Medical questions: Medical questions refer to descriptions of the patient's health status by the patient or his / her family. These descriptions are usually lengthy and often contain some peripheral and irrelevant information that is not conducive to retrieving the answer to the question, such as the patient's medical history.

[0077] Reference summary of medical questions: A reference summary of a medical question refers to a concise medical question obtained by summarizing the medical question by a doctor or expert. It eliminates some redundant information in the original medical question and is close to the meaning expressed by the medical question, which is conducive to retrieving the answer corresponding to the question from the question and answer database.

[0078] Medical entity: Medical entity refers to medical concepts such as disease name, disease symptoms, body parts, drug information, examination or inspection items, etc.

[0079] Positive examples: A positive example of a sentence refers to a sentence with the same semantics as the sentence; a sentence and its positive example, although not exactly the same sentences, express the same semantics.

[0080] Negative examples: The negative examples of a sentence refer to sentences that are semantically different from the sentence; a sentence and its negative examples are substantially different in semantics.

[0081] Medical question encoder: The encoder is part of the sequence-to-sequence model and is used to encode sentences; the medical question encoder specifically refers to an encoder used to encode medical questions.

[0082] Positive and negative sample encoder: The encoder is part of the sequence-to-sequence model and is used to encode sentences; the positive and negative sample encoder specifically refers to an encoder specifically used to encode positive and negative samples for medical problems.

[0083] Decoder: The decoder is the part of the sequence-to-sequence model that decodes the encoded sentence to generate the generated summary sentence.

[0084] JSON: JSON is a lightweight data exchange format that uses a text format that is completely independent of programming languages to store and represent data. DETAILED DESCRIPTION

[0085] The method and apparatus for generating medical question summaries based on contrastive learning of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments.

[0086] Example 1:

[0087] The overall model framework structure of the present invention is as follows Figure 2 As shown. Figure 2It can be seen that the main framework structure of the present invention includes a data processing module, a negative sample generation module, an encoder module, a contrastive learning module, a generation module and a decoder module. Among them, the data processing module first uses a word segmenter to segment the input data, and then generates a corresponding data set iterator, and sends the obtained data set iterator to the medical problem encoder in the encoder module for encoding. The encoded data set iterator contains multiple groups of medical problems and their reference summaries; the negative sample generation module is responsible for receiving the reference summaries corresponding to the medical problems, and then uses a simple negative sample generator and a difficult negative sample generator to generate simple negative samples and difficult negative samples for contrastive learning, and sends these negative samples to the positive and negative sample encoder in the encoder module; the encoder module contains two encoders, namely a medical problem encoder and a positive and negative sample encoder, wherein the medical problem encoder is responsible for encoding the input medical problems, and the positive and negative sample encoder is responsible for encoding the positive and negative samples of the medical problems, wherein the positive sample of the medical problem refers to the reference summary corresponding to the medical problem, and the negative sample of the medical problem is obtained through the negative sample. The simple negative examples and difficult negative examples obtained by the generation module, and the encoded medical problems will serve as inputs to the contrastive learning module and the generation module, while the encoded positive and negative examples will serve as inputs to the contrastive learning module; the contrastive learning module has two submodules, namely the contrastive learning module for simple negative examples and the contrastive learning module for difficult negative examples. The contrastive learning module for simple negative examples uses simple negative examples for contrastive learning, which enhances the encoding representation of the input medical problem, thereby improving the overall quality of the generated medical problem summary, while the contrastive learning module for difficult negative examples uses difficult negative examples for contrastive learning, which improves the encoder's attention to the focus of the input medical problem, thereby alleviating the focus errors and focus loss problems of the generated medical problem summary; the generation module is responsible for receiving the encoded medical problem and generating the corresponding summary encoding sequence; the decoder module is responsible for decoding the summary encoding sequence generated by the model prediction and obtaining the final medical problem summary. The above is a brief introduction to the structure of this model invention. It should be noted that the encoder, generation module and decoder mentioned here are all from the sequence-to-sequence model.

[0088] Example 2:

[0089] The method for generating medical problem summaries based on contrastive learning of the present invention comprises the following specific steps:

[0090] S1. Construct a medical question summary generation dataset: First, we need to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to construct a medical question summary generation dataset, and then divide it into a training dataset, a validation dataset, and a test dataset.

[0091] S101. Download the publicly available medical problem summary generation dataset from the Internet.

[0092] For example, there are many publicly available datasets on the Internet for generating summaries of medical questions, such as the MeQSum dataset.

[0093] The data items in the MeQSum dataset are as follows:

[0094]

[0095] The CHQ represents the patient's medical problem description, and the FAQ is a reference summary written by a doctor or expert based on the patient's medical problem description. Note: In the following descriptions, the CHQ is referred to as the medical problem, and the FAQ is referred to as the reference summary or reference summary of the medical problem.

[0096] S102: Preprocess the data set obtained in S101 and divide it into a training data set, a verification data set, and a test data set.

[0097] For example, the original file format of the MeQSum dataset is xlsx. In order to load different datasets in a unified way in the subsequent steps, the MeQSum dataset is first converted into a txt file, and then the txt file is converted into a JSON file. The format of the processed JSON file is as follows:

[0098] [{"chq":"SUBJECT:methadone causing COPD MESSAGE:My daughter said there was a study done and it concluded that methadone can cause COPD.Do you know if there is any truth to this and if so where can I get info on it.Thankyou[NAME]","faq":"What are the side effects of methadone?"},

[0099] {"chq":""SUBJECT:shingles MESSAGE:I am having symptoms of shingles,norash or blisters,is it too late to get the vaccine? I have had the chicken poxand take acylovir on a as needed basis for blisters I get on my buttocks.,"faq":"I am having symptoms of shingles; is it too late to get the vaccine?"}]

[0100] After obtaining the dataset in JSON format, we split it into a training dataset, a validation dataset, and a test dataset. The training dataset, validation dataset, and test dataset each contain 400, 100, and 500 data items, respectively.

[0101] S2. Construct difficult negative examples: First, traverse the reference summaries of all medical problems in the training dataset and build a medical entity dictionary; then identify the medical entities in the reference summary of each medical problem and perform similar replacement operations on these medical entities to obtain difficult negative examples.

[0102] S201, construct a medical entity dictionary, traverse the reference abstracts of medical problems in all data pairs in the training data set obtained in S102, identify the medical entities in the reference abstract of each medical problem, and form a medical entity dictionary with the identified non-repeated medical entities.

[0103] For example, there are many methods for medical named entity recognition. Here we use the medical named entity recognition module provided by StanfordNLP's Stanza package. In Python, the code described above is implemented as follows:

[0104]

[0105] Among them, nlp is a medical named entity recognition processor, which loads four medical named entity recognition models: i2b2, BC5CDR, ncbi_disease, and Radiology. The medical entity types supported by these four models are shown in the following table:

[0106]

[0107] med_entities is the medical entity dictionary to be constructed. faq_doc is the result object obtained by performing named entity recognition on the medical question reference abstract obj['faq'] . This object contains the entity list faq_doc.entities identified in the medical question reference abstract. For each medical entity ent in the entity list, if it is not already included in the medical entity dictionary, it is added to the medical entity dictionary.

[0108] S202. Generate difficult negative samples. For each reference summary of a medical problem containing a medical entity, first remove the medical entities contained in the reference summary itself from the medical entity dictionary obtained in S201 to obtain a candidate medical entity dictionary; then randomly select L×X medical entities from the candidate medical entity dictionary as candidate replacement entities, where L is a hyperparameter and X is the number of medical entities contained in the reference summary itself; then loop L×X times, each time replacing a medical entity in the reference summary with an entity in the candidate replacement entity; through the above processing, each reference summary will eventually generate L×X difficult negative samples.

[0109] It should be noted that if the reference summary does not contain medical entities or the medical entities therein are not correctly identified, the reference summary will not be used to generate hard negative examples.

[0110] For example, in Python, the code described above is implemented as follows:

[0111]

[0112]

[0113]

[0114] Among them, the reference summaries that can identify medical entities are stored in perturbation_list and written to the w1 file; the reference summaries that cannot identify medical entities are stored in no_perturbation_list and written to the w2 file. The meaning of faq_doc is the same as in S201. Entities stores all entities contained in a reference summary, and the length of entities is X described above. Candidate medical entities are stored in candidate_med_ent; rand_ent stores candidate replacement entities; the medical entities in rand_ent are randomly selected from candidate_med_ent, and the number of random selections is L×X. L is the args.sample_size in the program, which is a hyperparameter representing the number of samples and can be set to 128 in actual application. When performing entity replacement, first take a medical entity from the head of rand_ent, then replace the selected entity in the current reference summary with this entity, and remove this entity from rand_ent to prevent subsequent reuse. The replaced reference summaries are put into samples, and finally there will be L×X replaced reference summaries in samples. These replaced reference summaries are the required difficult negative examples.

[0115] More specifically, for the following pair of data:

[0116]

[0117]

[0118] First, identify the medical entity in the FAQ as methadone (a drug). Assume that the entire training dataset has been traversed and the following medical entity dictionary has been constructed:

[0119] index Medical Entity 0 amoxicillin 1 bupropion 2 cetirizine … … n allopurinol

[0120] There is a medical entity in the FAQ. Assuming that the value of args.sample_size is set to 4, then 1×4 medical entities will be randomly sampled from the medical entity dictionary as candidate replacement entities, and the methadone in the FAQ will be replaced with the randomly sampled medical entity, resulting in the following 4 hard negative examples:

[0121] What are the side effects of amoxicillin? What are the side effects of bupropion? What are the side effects of cetirizine? What are the side effects of allopurinol?

[0122] S3. Build a medical question summary generation model. The main operations include: loading the dataset, building the sequence-to-sequence model architecture and medical question encoder, building the positive and negative sample encoder, building the encoder momentum update module, building a simple negative sample and its storage queue, and building an average pooling layer. The specific steps are as follows:

[0123] S301. Load the dataset and create two functions to load JSON-formatted datasets. One function is responsible for loading the training dataset obtained in S102, including the data that can generate difficult negative examples in S202. The other function is responsible for loading the remaining data in the training dataset obtained in S102 and the data in the validation dataset. After the two datasets are loaded, use the word segmenter to perform word segmentation on each piece of data. The specific formula is as follows:

[0124]

[0125] Among them, Tokenizer represents the BART Tokenizer released by Facebook; CHQ represents medical question, FAQ represents reference summary; C represents the medical question after word segmentation; S represents the reference summary after word segmentation; for convenience of expression, C and S are referred to as medical question and reference summary in subsequent steps.

[0126] For example, in Python, the code described above is implemented as follows:

[0127] Load the portion of the dataset containing hard negative examples:

[0128]

[0129] The parameter dataset is the computer directory where the dataset is stored, and the file train_perturbation.json contains the data from the training dataset that can generate hard negative examples. After the data is loaded, the data list contains several triplets (CHQ, FAQ, perturbations), where CHQ represents the medical question, FAQ represents the reference summary, and perturbations represents the list of hard negative examples.

[0130] Load the portion of the dataset that does not contain hard negative examples:

[0131]

[0132]

[0133] The parameter dataset is also the computer directory where the dataset is stored. The file train_no_perturbation.json contains data from the training dataset that cannot generate hard negative examples, which only accounts for a small portion of the training dataset. The file val.json contains the validation dataset. The file test.json contains the test dataset. After the data is loaded, the data list contains several pairs (CHQ, FAQ), where CHQ represents a medical question and FAQ represents a reference summary.

[0134] After the dataset is loaded, the BART Tokenizer released by Facebook is used to perform tokenization on CHQ and FAQ in each bigram (CHQ, FAQ).

[0135] S302: Build a sequence-to-sequence model. First, select a suitable sequence-to-sequence model. Then load the pre-trained sequence-to-sequence model and its parameter configuration file. The loaded sequence-to-sequence model includes an encoder, a generation module, and a decoder. Finally, use the encoder in the sequence-to-sequence model as the medical question encoder. The specific formula is as follows:

[0136] R' C =Encoder B (C) (2)

[0137] Among them, Encoder B represents the medical problem encoder, C represents the input medical problem, R' C A coded representation representing a medical problem.

[0138] For example, in Huggingface's Transformer, the code implementation for the above description is as follows:

[0139] config=BartConfig.from_pretrained(args.model)

[0140] self.model=BartForConditionalGeneration.from_pretrained(args.model)

[0141] args.model is the model name, and its value is facebook / bart-large, indicating that the pre-trained sequence-to-sequence model used is the LARGE version of the BART model released by Facebook. config is the default parameter configuration table for the BART model. self.model is the BART model body, which includes an encoder, a generator module, and a decoder. The encoder is used as the medical question encoder.

[0142] S303: Construct a positive and negative sample encoder. The initial structure and parameters of the positive and negative sample encoder are exactly the same as those of the medical question encoder obtained in S302. Therefore, it is only necessary to copy the structure and parameters of the medical question encoder to the positive and negative sample encoder to complete the construction. The specific formula is as follows:

[0143] R' S =Encoder S (S) (3)

[0144] Among them, Encoder S represents the positive and negative sample encoder, S represents the positive or negative sample corresponding to the medical problem, R' S The encoded representation representing positive or negative examples.

[0145] For example, in Huggingface's Transformer, the code implementation for the above description is as follows:

[0146]

[0147] Here, dim refers to the dimension of the word vector. Since the length of each sentence is not necessarily the same, shorter sentences need to be padded. Padding_idx is the index value specified during padding. The self.encoder_k in the code is the positive and negative sample encoder; config is the default parameter configuration table of the BART model mentioned in S302; nn.Embedding() is the layer that implements the mapping between words and word vectors; BartEncoder() is used to create the encoder in the BART model. self.model.get_encoder().parameters() is the parameter of the medical question encoder, self.encoder_k.parameters() is the parameter of the positive and negative sample encoder, param_k.data = param_base.data copies the parameters of the medical question encoder to the positive and negative sample encoder. param_k.requires_grad = False sets the state of the positive and negative sample encoder to not participate in the gradient calculation. The parameters of the positive and negative sample encoder are updated through the encoder momentum update module in S304.

[0148] S304: Construct an encoder momentum update module. This module uses momentum update to maintain consistency between the medical question encoder obtained in S302 and the positive and negative sample encoders obtained in S303. That is, the parameters of the positive and negative sample encoders are slowly updated as the parameters of the medical question encoder change. The specific formula is as follows:

[0149] θ k ←mθ k +(1-m)θ q (4)

[0150] Among them, θ k are the parameters of the positive and negative sample encoders, θ q are the parameters of the medical question encoder and m∈[0,1) is the momentum coefficient.

[0151] For example, the momentum update function described above is implemented in Pytorch as follows:

[0152]

[0153] Among them, self.model.get_encoder().parameters() are the parameters of the medical question encoder, self.encoder_k.parameters() are the parameters of the positive and negative sample encoder; param_k.data is the value of the specific parameters of the positive and negative sample encoder; param_q.data is the value of the specific parameters of the medical question encoder; self.M is the momentum coefficient hyperparameter. In the specific example, the value of self.M is 0.999.

[0154] S305: Construct a simple negative sample storage queue. First, construct a simple negative sample, and then define the data structure of the queue and the dequeueing and enqueuing operations of the simple negative sample in the queue.

[0155] S30501. Construct a simple negative example. For a training data sample {CHQ1, FAQ1}, CHQ1 is a medical question and FAQ1 is the reference summary corresponding to the medical question. CHQ1 should be semantically similar to its own reference summary FAQ1, and should be similar to other CHQ1. i Reference Summary FAQ for (i≠1) i There is a big difference; therefore, FAQ1 can be used as a positive example of CHQ1, and other CHQ i Reference Summary FAQ i As a simple negative example for CHQ1.

[0156] For example, assume that the batch_size value is 2, the queue size is 4, and the initial state of the queue is as follows (the top is the end of the queue, and the bottom is the head of the queue):

[0157]

[0158] Now there is a new batch:

[0159]

[0160] Then, the simple negative examples for CHQ1 and CHQ2 are the four FAQs in the queue from other data pairs: What are the treatments for hand tremor?, How is Megalencephalic leukoencephalopathy with subcortical cysts (MLC) inherited?, What are the treatments for Ankylosing Spondylitis?, and What are the symptoms of asthma?. At the same time, after CHQ1 and CHQ2 are calculated, the FAQs at the head of the queue, What are the symptoms of asthma? and What are the treatments for Ankylosing Spondylitis?, will be replaced by What are the side effects of methadone? and What causes a rise in blood pressure when asleep? in the current batch. The final state of the queue is as follows (the top is the end of the queue, the bottom is the head of the queue):

[0161]

[0162]

[0163] S30502. Define the data structure of the queue and the dequeueing and enqueuing operations of the simple negative examples obtained in S30501.

[0164] For example, the queue described above is implemented in Pytorch as follows:

[0165] Queue data structure definition:

[0166] self.register_buffer("queue",torch.randn(dim,args.contrast_K))

[0167] self.queue=nn.functional.normalize(self.queue,dim=0)

[0168] self.register_buffer("queue_ptr",torch.zeros(1,dtype=torch.long))

[0169] Here, dim refers to the dimension of the word vector, and contrast_K is a hyperparameter, which refers to the size of the queue and is set to 4096 in practice. Note that nn.functional.normalize() computes the p-norm along the specified dimension. Dim is a parameter of the normalize() function, which refers to the dimension computed by the function and is different from the dim passed in torch.randn(dim,args.contrast_K) .

[0170] The queue dequeue and enqueue operations are implemented as follows:

[0171]

[0172] Batch_size refers to the number of batches of data processed by the program at a time, and its value can be modified in the program parameters. ptr is the queue pointer; self.K is the queue size. At ptr = (ptr + batch_size) % self.K, the modified pointer position modulo the queue size is used to make the queue a circular queue, avoiding wasted queue space.

[0173] S306: Construct an average pooling layer. Use the medical question encoder and positive and negative sample encoder obtained in S302 and S303 to encode the medical question and positive and negative samples respectively to obtain their feature representations. The average pooling layer is responsible for extracting the mean features of these feature representations. The specific formula is as follows:

[0174]

[0175] Among them, AvgPooling represents the average pooling layer; R' C Represents the use of the medical problem encoder obtained in S302 B The encoded representation obtained by encoding the medical problem C; R' S Represents the use of the positive and negative sample encoder obtained in S303 K The encoded representation obtained by encoding the positive or negative sample S; R C and R S Represents R' C and R' S Representation after the average pooling layer.

[0176] For example, the code implementation of the average pooling layer mentioned above in Pytorch is:

[0177]

[0178] S4. Train the medical problem summary generation model. Use the medical problem summary generation training dataset obtained in step S1 to train the medical problem summary generation model constructed in step S3 to optimize the model parameters. The main steps include: constructing a loss function and optimizing model training; wherein the loss function consists of three parts: a cross-entropy loss function, a contrastive learning loss function for simple negative examples, and a contrastive learning loss function for difficult negative examples.

[0179] S401. Construct a loss function. The loss function includes three parts: cross entropy loss function, contrastive learning loss function for simple negative examples, and contrastive learning loss function for difficult negative examples.

[0180] S40101. Construct a cross entropy loss function. For the sequence-to-sequence model constructed in S302, the standard training algorithm is maximum likelihood estimation. For a specific training example {medical problem, reference abstract} in the training data set obtained in S102, let the medical problem be C i , whose reference abstract is The token representation is Using the maximum likelihood estimation algorithm is equivalent to minimizing l words in The sum of the negative log-likelihood of , that is, optimizing the following cross entropy loss function:

[0181]

[0182] in, Refers to a pre-defined starting word and the current word The word table composed of the previous word, that is p represents a one-hot distribution in the standard maximum likelihood estimation framework; θ refers to the parameter of f, is the probability distribution caused by the parameters of f; f is a model function. Given a medical problem C, the goal of the medical problem summary task is to learn a function f to generate a reference summary S of medical problem C. The specific formula is as follows:

[0183] S←f(C)(7)

[0184] S40102, construct a contrastive learning loss function for simple negative examples. For a specific training example {medical problem, reference abstract} in the training data set obtained in S102, let the medical problem be C i , let its reference abstract be Let the reference summary of other medical problems be According to the construction and storage method of simple negative samples defined in S30501, Considered as C i A positive example, Considered as C i The simple negative example, the contrastive learning loss function of the simple negative example is defined as follows:

[0185]

[0186] in, Is a medical problem C i The vector representation obtained after passing through the medical question encoder constructed in S302 and the average pooling layer constructed in S306; It is C i Reference Summary The vector representation obtained after passing through the positive and negative sample encoder constructed in S303 and the average pooling layer constructed in S306; A reference summary of other medical issues The vector representation obtained after passing through the positive and negative sample encoder constructed in S303 and the average pooling layer constructed in S306; It is calculated and The cosine similarity of is as follows:

[0187]

[0188] τ is a temperature hyperparameter; Q is a simple negative sample storage queue defined in S30502, and the size of the queue is a hyperparameter defined in S30502; and The calculation method is similar and will not be repeated here.

[0189] S40103, construct a contrastive learning loss function for difficult negative examples. For a specific training example {medical problem, reference abstract} in the training data set obtained in S102, let the medical problem be C i , let its reference abstract be Will Considered as C i positive sample; let the difficult negative sample generated in S202 be h, and regard it as C i The contrastive learning loss function of difficult negative examples is defined as follows:

[0190]

[0191] in, Is a medical problem C i The vector representation obtained after passing through the medical question encoder constructed in S302 and the average pooling layer constructed in S306; It is C i Reference Summary The vector representation obtained after the positive and negative sample encoder constructed in S303 and the average pooling layer constructed in S306; R h is the vector representation obtained after the hard negative example h passes through the positive and negative example encoder constructed in S303 and the average pooling layer constructed in S306; It is calculated and R h The cosine similarity of is as follows:

[0192]

[0193] τ is a temperature hyperparameter; H is a set for storing difficult negative examples, whose size depends on the number of difficult negative examples sampled during training, and its number is a hyperparameter; and The calculation method is similar and will not be repeated here.

[0194] S40104. Construct a total loss function. The total function is obtained by adding the cross entropy loss function in S40101, the contrastive learning loss function of the simple negative example in S40102, and the contrastive learning loss function of the difficult negative example in S40103. The specific formula is as follows:

[0195]

[0196] in, is the cross entropy loss function, is the contrastive learning loss function for simple negative examples, is the contrastive learning loss function for hard negative examples.

[0197] S402. Optimize model training, use Adam as the optimization algorithm, set the learning rate to 1e-5, and optimize the training of the medical problem summary generation model on the training data set; when the model of this method has not been fully trained, it needs to be trained on the training data set to optimize the model parameters; when the model training is completed, the model can automatically generate a summary for the input medical problem.

[0198] For example, the optimization function and the optimization training process described above are implemented in Pytorch as follows:

[0199]

[0200] Among them, model.model.parameters() is the parameters of the model; lr is the learning rate, optimizer.zero_grad() means to clear the gradient to zero; loss is the loss calculated according to the total loss formula in S40104 after one round of model training; loss.backward() is used to calculate the gradient of all learnable parameters in the model; optimizer.step() means to let the optimizer update the parameters according to the calculated gradient.

[0201] When the model training is completed, the model can automatically generate a summary for the input medical question. The specific steps are as follows: first, the medical question is sent to the word segmenter described in S301 for word segmentation; then the segmented medical question is sent to the medical question encoder described in S302 for encoding; then the encoded medical question is sent to the generation module described in S302 to obtain a summary coding sequence; finally, the summary coding sequence is sent to the decoder described in S302 to obtain a medical question summary.

[0202] The model proposed in this paper achieves better results than the current advanced models on the MeQSum dataset. The comparison of experimental results is shown in the table below:

[0203] Method R1 R2 RL ProphetNet+QTR+QFR 45.52 27.54 48.19 RQE+MTL+Data Augmentation 49.20 29.50 44.80 Our Model 52.85 36.06 50.48

[0204] The model of the present invention is compared with the existing model. Among them, the first two rows are the experimental results of the models of the prior art, which are from Document 1 and Document 2 respectively; Document 1, namely Yadav S, Gupta D, Abacha AB, et al. Reinforcement Learning for Abstractive Question Summarization with Question-aware Semantic Rewards, ACL-IJCNLP2021; Document 2, namely Mrini K, Dernoncourt F, Yoon S, et al. A gradually soft multi-task and data-augmented approach to medical question understanding, ACL-IJCNLP2021, and the last row is the experimental result of the model of the present invention. R1, R2 and RL represent the F1 scores of ROUGH-1, ROUGH-2 and ROUGH-L in the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score, respectively. It can be seen that the present invention has a significant improvement compared with the existing model.

[0205] Example 3:

[0206] As attached Figure 1 As shown, the medical problem summary generation device based on contrastive learning according to Example 2 includes: a medical problem summary generation data set construction unit, a difficult negative example construction unit, a medical problem summary generation model construction unit, and a medical problem summary generation model training unit, which respectively implement the functions of steps S1, S2, S3, and S4 in the medical problem summary generation method based on contrastive learning. The specific functions of each unit are as follows:

[0207] The medical problem summary generates a dataset construction unit, which is used to preprocess the original dataset and divide the original dataset into a training dataset, a validation dataset, and a test dataset.

[0208] A difficult negative example construction unit is used to build a medical entity dictionary and construct difficult negative examples for contrastive learning in the training dataset.

[0209] The medical question summary generation model building unit is used to build a dataset loading module, a sequence-to-sequence model architecture, a medical question encoder, a positive and negative sample encoder, an encoder momentum update module, simple negative samples and their storage queue, and an average pooling layer.

[0210] The medical problem summary generates a model training unit, which is used to construct the loss function and optimization function required in the model training process and complete the model training.

[0211] Example 4:

[0212] An electronic device, characterized in that the electronic device includes a storage medium based on Example 2 and a processor; the storage medium stores multiple instructions, the instructions are loaded by the processor to execute the steps of the medical problem summary generation method based on contrastive learning in Example 2; the processor is used to execute the instructions in the storage medium.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical question summary generation method based on contrastive learning, characterized in that: The method comprises the following steps: Step S1, constructing a medical question summary generation dataset: First, it is necessary to obtain medical questions raised by users in the medical Q&A community and reference summaries written by experts to construct a medical question summary generation dataset, and then split it into a training dataset, a validation dataset, and a test dataset; Step S2, constructing hard negative examples: First, traverse the reference abstracts of all medical questions in the training dataset and construct a medical entity dictionary; then, for each medical question reference abstract, identify the medical entities in it and perform similar replacement operations on these medical entities to obtain hard negative examples; Step S3: Build a medical question summary generation model. The main operations include: loading the dataset, building a sequence-to-sequence model architecture and a medical question encoder, building a positive and negative sample encoder, building an encoder momentum update module, building a simple negative sample and its storage queue, and building an average pooling layer. Step S4: training the medical problem summary generation model. The medical problem summary generation training dataset obtained in step S1 is used to train the medical problem summary generation model constructed in step S3 to optimize model parameters. The main steps include: constructing a loss function and optimizing model training; wherein the loss function is composed of three parts: a cross-entropy loss function, a contrastive learning loss function for simple negative examples, and a contrastive learning loss function for difficult negative examples. The construction of simple negative examples and their storage queue in step S3 are as follows: First, we construct a simple negative example, then define the queue data structure and the dequeue and enqueue operations of the simple negative example in the queue; The specific construction of simple negative examples is as follows: for a training data sample {CHQ1, FAQ1}, where CHQ1 is a medical question and FAQ1 is a reference summary corresponding to the medical question; CHQ1 should be semantically similar to its own reference summary FAQ1, and not similar to other CHQ1. i Reference Summary FAQ for (i≠1) i There is a big difference; therefore, FAQ1 can be used as a positive example of CHQ1, and other CHQ i Reference Summary FAQ i As a simple negative example of CHQ1; The construction of the average pooling layer in step S3 is as follows: The medical question encoder and positive and negative sample encoder are used to encode the medical question and positive and negative samples respectively to obtain their feature representations. The average pooling layer is responsible for extracting the mean features of these feature representations. The specific formula is as follows: R C =AvgPooling(R' C ) R S =AvgPooling(R' S ) Among them, AvgPooling represents the average pooling layer; R' C Represents the use of medical question encoder Encoder B The encoded representation obtained by encoding the medical problem C; R' S Represents the use of positive and negative sample encoders K The encoded representation obtained by encoding the positive sample or negative sample S; R C and R S Represents R' C and R' S Representation after the average pooling layer.

2. The method for generating medical problem summaries based on contrastive learning according to claim 1, characterized in that: The construction of difficult negative examples is as follows: After obtaining the medical problem summary generation dataset, we first traverse the reference summaries of all medical problems in the training dataset to build a medical entity dictionary. Then, we identify the medical entities in the reference summary of each medical problem and perform similar replacement operations on these medical entities to obtain hard negative examples. The medical entity dictionary is constructed by traversing the reference abstracts of medical problems in all data pairs in the training data set, identifying the medical entities in the reference abstracts of each medical problem, and forming the medical entity dictionary with the identified non-repeated medical entities; The hard negative examples are generated as follows: for each reference summary of a medical question containing a medical entity, first, the medical entities contained in the reference summary are removed from the medical entity dictionary to obtain a candidate medical entity dictionary; then, L×X medical entities are randomly selected from the candidate medical entity dictionary as candidate replacement entities, where L is a hyperparameter and X is the number of medical entities contained in the reference summary; then, a loop is repeated L×X times, each time replacing a medical entity in the reference summary with an entity in the candidate replacement entity; through the above process, each reference summary will eventually generate L×X hard negative examples; It should be noted that if the reference summary does not contain medical entities or the medical entities therein are not correctly identified, the reference summary will not be used to generate hard negative examples.

3. The method for generating medical problem summaries based on contrastive learning according to claim 1, characterized in that: The sequence-to-sequence model architecture and medical question encoder are as follows: First, select a suitable sequence-to-sequence model. Then load the pre-trained sequence-to-sequence model and its parameter configuration file. The loaded sequence-to-sequence model contains an encoder, a generation module, and a decoder. Finally, use the encoder in the sequence-to-sequence model as the medical question encoder. The specific formula is as follows: R' C =Encoder B (C) Among them, Encoder B represents the medical problem encoder, C represents the input medical problem, R' C A coded representation representing a medical problem.

4. The method for generating medical problem summaries based on contrastive learning according to claim 1, characterized in that: The construction of the positive and negative sample encoder is as follows: The initial structure and parameters of the positive and negative sample encoder are exactly the same as those of the medical question encoder. Therefore, we only need to copy the structure and parameters of the medical question encoder to complete the construction. The specific formula is as follows: R' S =Encoder S (S) Among them, Encoder S represents the positive and negative sample encoder, S represents the positive or negative sample corresponding to the medical problem, R' S The encoded representation representing positive or negative examples.

5. The method for generating medical problem summaries based on contrastive learning according to claim 1, characterized in that: The encoder momentum update module is constructed as follows: The encoder momentum update module uses momentum update to maintain consistency between the medical question encoder and the positive and negative sample encoders. That is, the parameters of the positive and negative sample encoders are slowly updated as the parameters of the medical question encoder change. The specific formula is as follows: i k ←mθ k +(1-m)θ q Among them, θ k are the parameters of the positive and negative sample encoders, θ q are the parameters of the medical question encoder and m∈[0,1) is the momentum coefficient.

6. The method for generating medical problem summaries based on contrastive learning according to claim 1, characterized in that: The training of the medical question summary generation model is as follows: The medical question summary generation model was trained using the medical question summary generation training dataset to optimize model parameters. The main steps included constructing a loss function and optimizing model training. The loss function consisted of three parts: a cross-entropy loss function, a contrastive learning loss function for easy negative examples, and a contrastive learning loss function for hard negative examples. The loss function is constructed as follows: the loss function includes three parts: a cross entropy loss function, a contrastive learning loss function for simple negative examples, and a contrastive learning loss function for difficult negative examples; Construct a cross entropy loss function. For sequence-to-sequence models, the standard training algorithm is maximum likelihood estimation. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , whose reference abstract is The token representation is Using the maximum likelihood estimation algorithm is equivalent to minimizing l words in The sum of the negative log-likelihood of , that is, optimizing the following cross entropy loss function: in, Refers to a pre-defined starting word and the current word The word table composed of the previous word, that is p represents a one-hot distribution in the standard maximum likelihood estimation framework; θ refers to the parameter of f, is the probability distribution caused by the parameters of f; f is a model function. Given a medical problem C, the goal of the medical problem summary task is to learn a function f to generate a reference summary S of medical problem C. The specific formula is as follows: S←f(C) Construct a contrastive learning loss function for simple negative examples. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , let its reference abstract be Let the reference summary of other medical problems be According to the construction and storage method of simple negative samples, Considered as C i A positive example, Considered as C i The simple negative example, the contrastive learning loss function of the simple negative example is defined as follows: in, Is a medical problem C i The vector representation obtained after the medical question encoder and average pooling layer; It is C i Reference Summary The vector representation obtained after the positive and negative sample encoder and the average pooling layer; A reference summary of other medical issues The vector representation obtained after the positive and negative sample encoder and the average pooling layer; It is calculated and The cosine similarity of is as follows: τ is a temperature hyperparameter; Q is a simple negative sample storage queue, the size of which is a hyperparameter; and The calculation method is similar and will not be repeated here; Construct a contrastive learning loss function for difficult negative examples. For a specific training example {medical problem, reference abstract} in the training dataset, let the medical problem be C i , let its reference abstract be Will Considered as C i positive sample; let the hard negative sample be h, and regard it as C i The contrastive learning loss function of difficult negative examples is defined as follows: in, Is a medical problem C i The vector representation obtained after the medical question encoder and average pooling layer; It is C i Reference Summary The vector representation obtained after the positive and negative sample encoder and the average pooling layer; R h is the vector representation of the hard negative example h after passing through the positive and negative sample encoder and the average pooling layer; It is calculated and R h The cosine similarity of is as follows: τ is a temperature hyperparameter; H is a set for storing difficult negative examples, whose size depends on the number of difficult negative examples sampled during training, and its number is a hyperparameter; and The calculation method is similar and will not be repeated here; Construct the total loss function, which is obtained by adding the cross entropy loss function, the contrastive learning loss function of simple negative examples, and the contrastive learning loss function of difficult negative examples. The specific formula is as follows: in, is the cross entropy loss function, is the contrastive learning loss function for simple negative examples, is the contrastive learning loss function for hard negative examples; The optimization model training is specifically as follows: using Adam as the optimization algorithm, setting the learning rate to 1e-5, and optimizing the medical question summary generation model on the training dataset; When the model of this method has not been fully trained, it needs to be trained on the training data set to optimize the model parameters; when the model training is completed, the model can automatically generate a summary for the input medical question.

7. A device for generating medical problem summaries based on contrastive learning, which implements the method for generating medical problem summaries based on contrastive learning as described in any one of claims 1 to 6, characterized in that: The device includes: The medical problem summary generation dataset construction unit is responsible for preprocessing the original dataset and dividing the original dataset into training dataset, validation dataset and test dataset; The difficult negative example construction unit is responsible for building a medical entity dictionary and constructing difficult negative examples for contrastive learning in the training dataset; The medical question summary generation model building unit is used to build a dataset loading module, a sequence-to-sequence model architecture, a medical question encoder, a positive and negative example encoder, an encoder momentum update module, simple negative examples and their storage queue, and an average pooling layer; The medical problem summary generates a model training unit, which is used to construct the loss function and optimization function required in the model training process and complete the model training.

8. An electronic device, characterized in that: The electronic device includes a storage medium and a processor; the storage medium stores a plurality of instructions, the instructions being loaded by the processor to execute the steps of the medical question summary generation method based on contrastive learning according to any one of claims 1 to 6; A processor is configured to execute instructions in the storage medium.

Images