Medical literature abstract automatic generation method and system, medium and equipment

Abstract

The invention provides a medical literature abstract automatic generation method and system, a medium and equipment. The medical literature abstract automatic generation method comprises the steps of obtaining a medical literature text; converting each sentence in the medical literature text into a corresponding embedded vector; converting the embedded vector corresponding to each sentence into a feature vector of semantic information contained in each sentence; the feature vector of the semantic information contained in each sentence is screened, the feature vector of a target sentence is determined, and the feature vector of the target sentence is used for representing that the target sentence can be added into an abstract text; and decoding the feature vector of the target sentence to generate an abstract text. According to the method, in combination with the deep learning model of the self-attention mechanism, the semantic accuracy, specialty and readability of the abstract can be remarkably improved through the synergistic effect of the embedded layer, the encoder, the classifier and the decoder in the generation of the medical literature abstract, and more valuable information support is provided for medical research and clinical practice.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a method, system, medium and device for automatically generating medical literature abstracts, aiming to improve the efficiency and accuracy of generating medical literature abstracts and provide a convenient way for scientific researchers and clinical workers to obtain information. Background Art

[0002] With the continuous deepening and subdivision of medical research, the knowledge system in the medical field has shown an explosive growth trend, and the number of medical literature has also increased sharply. These documents cover a wide range of content from basic medical research to clinical application practice, providing researchers with rich data and insights. However, faced with such a huge amount of literature, how to efficiently and accurately screen out the core content of each document has become a difficult task that researchers must face.

[0003] Although the traditional manual summarization method can ensure a certain degree of accuracy and depth, its inherent limitations are becoming increasingly prominent. First, manual summarization is time-consuming and laborious, requiring researchers to invest a lot of time and energy to read, understand and refine the content of the literature, which is undoubtedly a huge burden for researchers who are already burdened with heavy research tasks. Secondly, manual summarization is easily affected by subjective factors. Different researchers may have deviations when understanding the literature and refining the key points, resulting in inconsistent summary results. Finally, faced with the needs of large-scale literature processing, the manual summarization method obviously cannot meet the requirements of efficiency and scale.

[0004] Therefore, it is particularly urgent and important to develop an automated, efficient and accurate method for generating medical literature abstracts. Summary of the invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a method, system, medium and device for automatically generating medical literature abstracts.

[0006] A method for automatically generating medical literature abstracts according to the present invention comprises:

[0007] Access medical literature texts;

[0008] Convert each sentence in the medical literature text into a corresponding embedding vector;

[0009] Convert the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence;

[0010] Screening the feature vectors of the semantic information contained in each sentence to determine the feature vector of the target sentence, wherein the feature vector of the target sentence is used to indicate that the target sentence can be added to the summary text;

[0011] The feature vector of the target sentence is decoded to generate a summary text.

[0012] This application converts each word or sub-word unit in a sentence into a corresponding embedding vector through an embedding layer. These embedding vectors capture the basic semantic information of the word or sub-word unit, so that each sentence in the medical literature text is effectively converted into an embedding vector rich in semantic information, providing a solid foundation for subsequent natural language processing tasks. Furthermore, this application converts the embedding vector into a high-quality feature vector through an encoder, capturing the semantic relationship between words, contextual dependency, and the deep meaning of the sentence as a whole, so that the feature vector can more accurately reflect the true intention and meaning of the sentence, and the high-quality feature vector can be used in a variety of different natural language processing tasks. Furthermore, this application screens the feature vector through a classifier. The classifier can evaluate the relevance of each sentence to the original theme or core content based on the semantic information in the feature vector. By screening out highly relevant feature vectors, it can ensure that the generated summary is closely around the original theme, improving the accuracy and usefulness of the summary. Furthermore, the present application decodes the feature vector through a decoder. The decoder is trained to learn and imitate the grammatical rules and expression habits of human language, so that the generated summary text is grammatically correct and the expression is fluent and natural, avoiding the common stiffness and incoherence problems of machine-generated text. When generating the summary, the decoder will consider the content of the generated text to ensure the coherence and consistency of the entire summary.

[0013] Preferably, converting each sentence in the medical literature text into a corresponding embedding vector comprises:

[0014] Encode each word of each sentence into a token embedding vector;

[0015] Encode the position information of each word in the sentence into a position embedding vector;

[0016] Assigning a specific fragment embedding vector to each of the sentences, wherein the specific fragment embedding vector is used to distinguish vector representations of different sentences;

[0017] The embedding vector is obtained by summing the tag embedding vector, the position embedding vector and the fragment embedding vector.

[0018] This application converts words into high-dimensional vector representations through a tag embedding layer. These vectors can capture the semantic information of words. This conversion enables the model to understand the semantic relationship between words, such as similarity, antonym relationship, etc. In the medical field, this helps the model distinguish between professional terms and common vocabulary, and accurately understand their meaning in a specific context. This application encodes the position information of words in a sentence into a feature vector through a position embedding layer. This step is crucial for understanding the grammatical structure and semantic hierarchy of a sentence. In medical literature, the structure of a sentence is often complex, containing multiple clauses and modifiers. The position embedding layer can help the model identify these structures and understand the dependencies between different parts. When processing text containing multiple sentences, this application can distinguish the vector representations of different sentences through a fragment embedding layer, which is particularly important for tasks that require understanding the relationship between sentences. In medical literature, different sentences may discuss different topics or viewpoints. The fragment embedding layer helps the model distinguish these differences and better understand the structure and logic of the entire text.

[0019] Preferably, converting the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence comprises:

[0020] Converting the embedding vector into a first vector, wherein the first vector includes the interdependencies between different parts within each sentence;

[0021] An average vector is obtained by performing an average calculation on the first vectors, and the average vector is used as a feature vector of each sentence.

[0022] This application converts the sentence embedding vector into a new vector through the joint action of the self-attention mechanism and the feedforward neural network layer. This new vector not only contains the semantic information of the original word, but also incorporates the dependency between different parts of the sentence. Therefore, the new vector is richer and more accurate in semantic representation and can better reflect the overall meaning of the sentence. The self-attention mechanism allows the model to consider the interaction between all words in the sentence at the same time when processing the sentence, so that the model can capture the dependency between different parts of the sentence, such as the subject-predicate relationship, the modification relationship, etc. In medical literature, this ability is particularly important because medical sentences often contain complex structures and professional terms, and an in-depth understanding of their internal relationships is required to accurately understand their meaning. The new vector of the sentence is averaged to obtain an average vector as the vector representation of the sentence. This step simplifies the representation of the vector so that each sentence can be represented by a vector of fixed length. This representation method is convenient for subsequent processing, such as classification, clustering, similarity calculation, etc. At the same time, the average vector also retains most of the important information in the sentence, so that the model can maintain good performance in subsequent tasks.

[0023] Preferably, screening the feature vector of the semantic information contained in each sentence to determine the feature vector of the target sentence includes:

[0024] Calculate the probability value of adding the feature vector of the semantic information contained in each sentence into the summary text;

[0025] The feature vector of the semantic information contained in the sentence whose probability value is not less than a preset threshold is determined as the feature vector of the target sentence.

[0026] This application maps the feature vector to a probability value in the interval (0, 1) through a sigmoid function. This probability value reflects the importance or relevance of the sentence to the summary. This probability evaluation method makes the generation of the summary more flexible, and the preset threshold can be adjusted according to different needs to generate summaries of different lengths. Based on the calculated probability, the feature vectors above the preset threshold are screened out. These sentences often contain key information or core ideas in the original text. This precise screening method helps to ensure the accuracy and representativeness of the summary and improve the quality of the summary. Each sentence has a clear probability value to indicate the possibility of being selected as a summary sentence. This probability interpretation makes the generation process of the summary more transparent and understandable, and helps users or researchers evaluate the rationality and accuracy of the summary. At the same time, the sigmoid function and the probability screening method do not rely on specific text types or domain knowledge, have strong versatility, can be well applied to the text summary generation task of medical literature, and provide high-quality summaries.

[0027] Preferably, the calculation formula of the probability value is:

[0028] Z=W o R i +B o ,

[0029] Y i =σ(z),

[0030] Among them, R i is the feature vector of the semantic information contained in each sentence, W o and B o are the weights and biases of the classifier model, and σ is the sigmoid function, defined as:

[0031]

[0032] Preferably, decoding the feature vector of the target sentence to generate a summary text includes:

[0033] Focusing on the feature vector of the target sentence through a cross attention mechanism, and converting the feature vector of the target sentence into a corresponding word or phrase;

[0034] Based on the feature vector of the target sentence to be output next time and / or the previous output text, by predicting the probability distribution of words or phrases, the words or phrases with the highest probability are selected as the output text, and the output text is output multiple times;

[0035] Sort the output text of multiple outputs to generate summary text.

[0036] This application uses the decoder's cross-attention mechanism to annotate the feature vector output by the classifier, which can focus on the feature vector output by the classifier, so that the decoder can fully consider the contextual information of the entire sentence when generating words or phrases. This enhanced contextual understanding ability helps to generate a more coherent summary that conforms to the logic of the original text. Unlike traditional methods that directly generate sentences or paragraphs, this application generates words or phrases one by one through the decoder, providing finer-grained control, which helps to more accurately capture the key information in the original text and avoid generating redundant or irrelevant content. Based on the feature vector output by the classifier next time and / or the last output text, the decoder makes multiple predictions and selects the word or phrase with the highest probability as the output. This iterative process allows the model to dynamically adjust subsequent predictions based on the generated content, thereby continuously optimizing the quality of the summary. Sorting the output texts of multiple outputs to generate summary texts, this step helps to further improve the relevance and accuracy of the summary. Sorting can ensure that the most important information is included first in the summary, so that users can get the core content of the original text faster.

[0037] Preferably, when decoding the feature vector of the target sentence to generate a summary text: all information data of the next time node is masked by a masked self-attention mechanism.

[0038] This application achieves the masking of future information by introducing a masking matrix in the self-attention calculation process. During the decoding process, the masking matrix sets all connections pointing to future time to invisible or zero values, thereby ensuring that when generating the output of the current position, the decoder only relies on the generated sequence and the output of the encoder, and will not be disturbed by future information that has not yet been generated. This not only effectively prevents information leakage, but also makes the decoding process more consistent with the logical order of human language generation: that is, gradually building the entire sentence or paragraph from left to right. In addition, the masked self-attention mechanism also enhances the generalization ability and robustness of the model.

[0039] The present invention also provides a medical literature abstract automatic generation system, comprising:

[0040] A document text acquisition module is used to acquire medical document text;

[0041] An embedding vector acquisition module, used for converting each sentence in the medical literature text into a corresponding embedding vector;

[0042] A feature vector acquisition module, used to convert the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence;

[0043] A feature vector screening module, used to screen the feature vectors of the semantic information contained in each sentence to determine the feature vector of the target sentence;

[0044] The summary text generation module is used to decode the feature vector of the target sentence to generate a summary text.

[0045] The present invention also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for automatically generating medical literature abstracts are implemented.

[0046] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned method for automatically generating medical literature abstracts when executed by the processor.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention combines the deep learning model with the self-attention mechanism in the generation of medical literature abstracts. Through the synergistic effect of the embedding layer, encoder, classifier and decoder, it can significantly improve the semantic accuracy, professionalism and readability of the abstract, providing more valuable information support for medical research and clinical practice.

[0049] The present invention converts sentences into corresponding embedding vectors, so that each sentence in the medical literature text is effectively converted into an embedding vector rich in semantic information; the embedding vector is converted into a feature vector, which can more accurately reflect the true intention and meaning of the sentence; the feature vectors are screened, and by screening out highly relevant feature vectors, it can ensure that the generated summary is closely around the original theme, improving the accuracy and usefulness of the summary; the feature vectors are decoded to generate a coherent and consistent summary text. Through the above steps, the semantic accuracy, professionalism and readability of the summary can be significantly improved, providing more valuable information support for medical research and clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0051] Figure 1A schematic diagram of the steps of the method for automatically generating medical literature abstracts;

[0052] Figure 2 Schematic diagram of the steps to convert a sentence into an embedding vector;

[0053] Figure 3 A flowchart of the steps for preprocessing medical literature text;

[0054] Figure 4 Schematic diagram of the steps to output the token embedding vector for the token embedding layer;

[0055] Figure 5 A schematic diagram of the steps for embedding a position vector for sine / cosine position encoding;

[0056] Figure 6 Schematic diagram of the steps for outputting a fragment embedding vector for the fragment embedding layer;

[0057] Figure 7 Schematic diagram of the principle of embedding layer for specific sentence information input;

[0058] Figure 8 Schematic diagram of the principle of distinguishing different sentences for the fragment embedding layer;

[0059] Fig. 9 A flowchart of the steps for converting an embedding vector into a vector representation containing semantic information;

[0060] Fig.10 Schematic diagram of the overall process of pre-training and fine-tuning BioBERT;

[0061] Fig.11 Schematic diagram of the principle of self-attention mechanism;

[0062] Fig.12 A flowchart of the steps for selecting feature vectors that can be added to the summary text;

[0063] Fig.13 A schematic diagram of the principle of calculating the probability that a feature vector can be added to the summary text;

[0064] Fig.14 Schematic diagram of the steps to generate summary text for feature vector decoding;

[0065] Fig.15 Schematic diagram of the principle of the cross attention mechanism;

[0066] Fig.16 Schematic diagram of the principle of forming output text for generating words or phrases;

[0067] Fig.17 is a structural block diagram of a medical literature abstract automatic generation system in one embodiment;

[0068] Fig.18 It is a structural block diagram of a system for automatically generating medical literature abstracts in another embodiment.

[0069] The figure shows:

[0070] Document text acquisition module 1 Pre-training language model module 7

[0071] Embedding vector acquisition module 2 Multi-task learning framework module 8

[0072] Feature vector acquisition module 3 Post-processing module 9

[0073] Feature vector screening module 4 System integration and testing module 10

[0074] Summary text generation module 5 User interface and interaction module 11

[0075] Data preprocessing module 6 Deployment and maintenance module 12 DETAILED DESCRIPTION

[0076] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0077] The present application provides a method for automatically generating a medical document summary, comprising: obtaining medical document text; inputting the medical document text into a self-attention mechanism deep learning model, and converting each sentence in the medical document text into a corresponding embedding vector through an embedding layer; converting the embedding vector corresponding to each sentence into a feature vector of semantic information contained in each sentence through an encoder; screening the feature vector of the semantic information contained in each sentence through a classifier to determine the feature vector of a target sentence; and decoding the feature vector of the target sentence through a decoder to generate a summary text.

[0078] This application converts each word or sub-word unit in a sentence into a corresponding embedding vector through an embedding layer. These embedding vectors capture the basic semantic information of the word or sub-word unit, so that each sentence in the medical literature text is effectively converted into an embedding vector rich in semantic information, providing a solid foundation for subsequent natural language processing tasks. This application converts the embedding vector into a high-quality feature vector through an encoder, capturing the semantic relationship between words, contextual dependency, and the deep meaning of the sentence as a whole, so that the feature vector can more accurately reflect the true intention and meaning of the sentence. At the same time, the high-quality feature vector can be used in a variety of different natural language processing tasks. This application screens the feature vector through a classifier. The classifier can evaluate the relevance of each sentence to the original theme or core content based on the semantic information in the feature vector. By screening out highly relevant feature vectors, it can ensure that the generated summary is closely around the original theme, improving the accuracy and usefulness of the summary. The present application decodes the feature vector through a decoder. The decoder is trained to learn and imitate the grammatical rules and expression habits of human language, so that the generated summary text is grammatically correct and the expression is fluent and natural, avoiding the common stiffness and incoherence problems of machine-generated text. When generating the summary, the decoder will consider the content of the generated text to ensure the coherence and consistency of the entire summary.

[0079] like Figure 1 As shown, a method for automatically generating medical literature abstracts comprises the following steps:

[0080] Step S1: Obtain medical literature text, segment the medical literature text into independent sentences, input the medical literature text into the self-attention mechanism deep learning model, and convert each sentence in the medical literature text into a corresponding embedding vector through an embedding layer.

[0081] like Figure 2 As shown, each sentence is converted into an embedding vector through the embedding layer. The specific process is as follows:

[0082] Step S101: convert each word in the sentence into a vector representation through a token embedding layer to obtain a token embedding vector;

[0083] Step S102: encoding the position information of the word in the sentence into a feature vector through a position embedding layer to obtain a position embedding vector;

[0084] Step S103: assigning a specific fragment embedding vector to each sentence through the fragment embedding layer to distinguish the vector representations of different sentences;

[0085] Step S104: sum the tag embedding vector, the position embedding vector and the fragment embedding vector to obtain the sentence embedding vector.

[0086] like Figure 3 As shown, specifically, before performing sentence segmentation, the medical literature text can be preprocessed. The preprocessing process is as follows:

[0087] Step SA1: clean the medical literature text to remove unnecessary formats, symbols (such as HTML tags, special characters, etc.), extra spaces and line breaks, etc., to ensure the purity of the text content;

[0088] Step SA2: confirm the language of the text, because medical literature may involve multiple languages, such as English, French, German, etc. Different word segmentation and sentence segmentation strategies may need to be adopted for different languages;

[0089] Step SA3: Unify the text encoding into UTF-8 or other compatible formats to ensure that no garbled characters will appear during the processing.

[0090] Specifically, for the sentence segmentation of medical literature text, the most basic method is to use punctuation marks such as period (.), question mark (?), exclamation mark (!) as a sign of the end of the sentence. However, in medical literature, these punctuation marks may also appear in abbreviations, formulas, quotations or brackets, so simple segmentation rules cannot be directly applied. Therefore, this application uses natural language processing tools to perform contextual analysis on the text to identify which punctuation marks do indicate the end of the sentence. This requires consideration of the vocabulary, grammatical structure and semantic information before and after the punctuation marks. For example, natural language processing tools such as NLTK, spaCy, jieba, Gensim, TextBlob, etc. can be selected as sentence segmentation tools. These tools usually identify sentence boundaries based on punctuation marks, sentence structure or grammatical rules of a specific language. For special cases in medical literature, such as abbreviations (such as Dr., etc.), quotations (such as "He said,'The test is positive.'"), list items (such as 1. First item; 2. Second item.), etc., special rules or algorithms can be designed to handle them to ensure the accuracy of sentence segmentation.

[0091] Specifically, for the self-attention mechanism deep learning model, the Transformer model can be selected. The Transformer model is a neural network model based on the self-attention mechanism. The Transformer model abandons the traditional recurrent neural network (RNN) structure and uses the attention mechanism to process sequence data. Its core advantage is that it can process all elements in the sequence in parallel, which greatly improves the training efficiency of the model. The Transformer model is mainly composed of two parts: the encoder and the decoder. Each part contains multiple identical layers (usually 6 layers or more). Each layer is mainly composed of a self-attention mechanism module and a feedforward neural network module. The self-attention mechanism is the core of the Transformer model. It allows the model to pay attention to other words in the sequence while encoding each word, thereby capturing the dependency between words. The key to this mechanism is to calculate the attention score of each word in the input sequence to other words and perform weighted summation based on these scores.

[0092] Specifically, the tag embedding layer, also called the word embedding layer, is used to convert each word in a sentence into a vector representation. Word embedding is a method of mapping words or tags in a vocabulary to vector representations in a high-dimensional space. These vectors can usually capture the semantic and syntactic relationships between words, so that similar words in the vector space are close to each other in meaning. The word embedding tools that can be selected include Word2Vec, GloVe, BERT, etc.

[0093] like Figure 4 As shown, specifically, for the embedding of the marker embedding layer, the specific process is as follows:

[0094] Step SB1, collect all the words that appear in the training data set to form a preliminary vocabulary list, pre-process the collected words, such as lowercase, remove punctuation, stemming or lemmatization, etc., to reduce the size of the vocabulary and improve the generalization ability of the model. According to the processed word list, build a vocabulary (or dictionary) containing unique words and their indexes. The size of the vocabulary (i.e., the number of words contained) is an important hyperparameter, which needs to be determined according to the specific requirements of the task and the size of the data set;

[0095] Step SB2: Initialize a matrix of all zeros or random decimals according to the size of the vocabulary and the defined embedding dimension. Each row of this matrix will correspond to the embedding vector of a word in the vocabulary. The embedding dimension refers to the length of the embedding vector and is also a key hyperparameter of the embedding layer. A larger embedding dimension can capture more semantic information, but it will also increase the complexity and computational cost of the model.

[0096] Step SB3: Convert each word in the sentence into an index in the vocabulary. This is usually achieved through a lookup operation, that is, matching each word with a word in the vocabulary and returning the corresponding index. For each word index in the sentence, use the embedding matrix to find the corresponding embedding vector. This is usually achieved by indexing the matrix rows, that is, using the index as the row number to retrieve the corresponding row in the embedding matrix. The embedding vectors of all the words in the sentence are stacked up in order to form a two-dimensional matrix. Each row of this matrix corresponds to the embedding vector of a word in the sentence, and the entire matrix represents the embedded representation of the sentence.

[0097] Specifically, for the position embedding layer, in the deep learning model, since the self-attention mechanism allows the model to process all words in the sequence at the same time, and the interactions between words are parallel, the model itself does not directly understand the order of words. In order to make up for this defect, this application introduces a position embedding layer to provide the position information of the word in the sentence.

[0098] Specifically, for the embedding of the position embedding layer, you can choose the following methods:

[0099] Method 1: Absolute position embedding, assigning a unique embedding vector to each possible word position. These vectors are part of the model parameters and are optimized along with other parameters during training. For a sentence of length L, L different position embedding vectors are required, or to handle longer sentences, a maximum length limit can be set and the embedding vectors prepared accordingly.

[0100] Method 2: Relative position embedding. Unlike absolute position embedding, relative position embedding focuses on the relative distance between words rather than their absolute positions in the sentence. This method usually requires modifying the self-attention mechanism so that it can utilize relative position information.

[0101] Method 3: Sine / cosine position encoding (also called position encoding) uses a combination of sine and cosine functions to generate position embeddings. The advantage of this method is that it does not require learning additional parameters and can naturally extend to longer sequences than those seen during training.

[0102] like Figure 5 As shown, specifically, for sine / cosine position encoding, it is implemented through the following process:

[0103] Step SC1, initialize the position index, assign a unique index to each position in the sentence (starting from 0 or 1, depending on the implementation);

[0104] Step SC2, calculate the position embedding, for each position index pos and each dimension i of the embedding vector (where i usually starts from 0), use the sine or cosine function to calculate the value of the position in the dimension, usually, the value of the even dimension is calculated using the sine function, the value of the odd dimension is calculated using the cosine function, and each dimension uses a different wavelength (controlled by i);

[0105] Step SC3: Combine into a position embedding matrix, combine the values ​​of all positions in all dimensions into a two-dimensional matrix, where each row represents a position embedding vector.

[0106] Specifically, for the fragment embedding layer, in natural language processing (NLP) tasks, especially those involving multiple sentences or text fragments (such as question answering systems, text summarization, dialogue systems, etc.), it becomes particularly important to distinguish the vector representations of different sentences. The introduction of the fragment embedding layer (also called segment embedding layer or sentence embedding layer) aims to provide a unique representation for each sentence or text fragment so that the model can distinguish them and understand the relationship between them.

[0107] like Figure 6 As shown, specifically, for the embedding of the fragment embedding layer, the specific process is as follows:

[0108] Step SD1: Create an embedding matrix. According to the number of fragments and the dimension of the embedding vector, create an embedding matrix, where each row of the matrix corresponds to the embedding vector of a fragment.

[0109] Step SD2, indexing the embedding vector, during the forward propagation of the model, the corresponding embedding vector is retrieved from the embedding matrix according to the identifier (such as index or identifier ID) of the input sentence;

[0110] Step SD3: Combine with other embeddings to combine the fragment embedding vector with the word embedding vector or other types of embedding vectors (such as position embedding) to form a complete representation of the sentence.

[0111] Through the fragment embedding layer, each sentence or text fragment is given a unique embedding vector that occupies a different position in the embedding space, allowing the model to distinguish different sentences and understand the relationship between them. For example, in a question-answering system, questions and answers can be treated as different fragments and distinguished through the fragment embedding layer so that the model can more accurately understand the relationship between them.

[0112] Specifically, for the sum of the three embedding vectors, the output vectors of each tag corresponding to the above three embedding layers (token embedding, fragment embedding, position embedding) are summed element-wise, which means that for each tag in the sequence, its token embedding vector, fragment embedding vector and position embedding vector are added together to obtain a new vector. This series of summed vectors together constitute the embedding vector representation of the entire sentence or text fragment, which contains the vocabulary information, fragment information and position information in the text, and is the basis for the model to perform subsequent processing (such as classification, relationship judgment, etc.).

[0113] Specifically, for the structure of the embedding layer, before the medical literature is input into BERT, the medical literature input is converted into an embedding vector through the following three embedding layers: token embedding layer, segment embedding layer, and position embedding layer. The embedding vector is the sum of the three different embedding output vectors of the token embedding layer, segment embedding layer, and position embedding layer. The token embedding layer is used to output the vector representation of the word itself. WordPiece refers to dividing the word into a limited set of common sub-word units, which can achieve a compromise between the effectiveness of the word and the flexibility of the character. The position embedding layer is used to encode the position information of the word into a feature vector. The method of constructing the position embedding is mainly to initialize a position embedding and then learn it through training. The segment embedding layer is used to distinguish the vector representation of two sentences, which is used in asymmetric sentences such as question answering.

[0114] like Figure 7 As shown in the figure, suppose there are two sentences: "Paris is a beautiful city.", "I loveParis.", first the sentences are tokenized, then the [CLS] tag is added to the beginning of the sentence, and the [SEP] tag is added to the end of the sentence, and then these words are input into three embedding layers: tag embedding, fragment embedding and position embedding.

[0115] like Figure 8As shown, specifically, modify the input data format and add a [CLS] tag to the beginning of each sentence, so that the representation of each sentence can be obtained through this tag. For example, R1 in the figure represents sent one and R2 represents senttwo. The fragment embedding layer is used to distinguish two given sentences and return the embedding EA or EB of the two sentences. However, in the text summary setting, more than two sentences will be input. At this time, an interval fragment embedding is used. The interval fragment embedding is used to distinguish multiple given sentences, mapping words that appear in odd-indexed sentences to EA and even-indexed sentences to EB. Suppose there are three sentences: all words in sentence 1 are mapped to EA, all words in sentence 2 are mapped to EB, and all words in sentence 3 are mapped to EA.

[0116] Step S2: The encoder converts the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence.

[0117] like Fig. 9 As shown in the figure, the embedding vector corresponding to each sentence is converted into a feature vector of the semantic information contained in each sentence through the encoder. The specific process is as follows:

[0118] Step S201: converting the sentence embedding vector into a first vector through the encoder's self-attention mechanism and the feedforward neural network layer, where the first vector contains the interdependencies between different parts of each sentence;

[0119] Step S202: averaging the first vectors to obtain an average vector as a feature vector of each sentence.

[0120] Specifically, for the encoder, it is used to encode the transformed embedding vector into a feature vector of fixed length to obtain the contextual representation of the text. When constructing a medical literature abstract encoder, a medical field knowledge base or a pre-trained model can be integrated to enhance the model's understanding of medical terms and concepts. Such an encoder can more accurately identify key information in medical literature and generate more professional abstracts. The encoder of this application uses BERT. The full name of BERT is Bidirectional Encoder Representation from Transformer, that is, BERT is an encoder based on the Transformer model. A transformer encoder unit is generated by a multi-head-Attention+Layer Normalization+feedforword+Layer Normalization superposition. Each layer of BERT is composed of such an encoder unit. This transformer structure can use context to predict the token of the mask, thereby capturing a bidirectional relationship. BERT uses a bidirectional Transformer as an encoder for feature extraction, and uses multiple attention mechanisms to obtain more contextual information, which greatly improves the ability of the language model to extract features. Specifically, you can use models pre-trained on medical field data, such as BioBERT or PubMedBERT. These models have learned medical terms and concepts. BioBERT is a pre-trained language representation model for the biomedical field.

[0121] like Fig.10As shown, specifically, about BioBERT, the overall process of pre-training and fine-tuning BioBERT, BioBERT is initialized using the weights of BERT, which has been pre-trained on a general domain corpus (English Wikipedia and BooksCorpus), and then BioBERT is pre-trained on a biomedical domain corpus (PubMed abstracts and PMC full-text articles). Fine-tuning BioBERT requires only minimal architectural modifications to apply BioBERT to a variety of downstream text mining tasks. BioBERT is adjusted for the following three representative biomedical text mining tasks: NER, RE, and named entity recognition. Named entity recognition is one of the most basic biomedical text mining tasks, which involves identifying numerous domain-specific proper nouns in the biomedical corpus. While most previous solutions are built on different combinations of LSTM and CRF, BERT has a simple architecture based on a bidirectional Transformer. BERT uses a single output layer to calculate only the BIO2 probability at the token level based on the representation from the last layer. While previous work in biomedical NER often uses word embeddings trained on the PubMed or PMC corpora, BioBERT directly learns WordPiece embeddings during pre-training and fine-tuning. For the evaluation metrics of NER, entity-level accuracy, recall, and F1 score are used.

[0122] Specifically, for relation classification of named entities, relation extraction is the task of classifying relations of named entities in a biomedical corpus. The sentence classifier of the original version of BERT is used, which uses the [CLS] tag to classify relations. Based on the [CLS] token representation from BERT, a single output layer is used to perform sentence classification. The target named entities in the sentence are anonymized using predefined labels (such as @GENE$ or @DISEASE$). For example, a feature vector with two target entities (gene and disease in this case) is "Serine at position 986 of @GENE$ may be an independent genetic predictor of angiography @DISEASE$".

[0123] Specifically, for the self-attention mechanism, the self-attention mechanism is one of the core components in the Transformer model, which allows the model to consider the relationship between each word and other words in the sentence when processing sentences. Specifically, the self-attention mechanism captures the dependency between words by calculating the similarity scores of all word pairs in the sentence and then recombining the word representations based on these scores.

[0124] like Fig.11 As shown, specifically, the implementation of the self-attention mechanism is achieved through the following process:

[0125] Step SE1: In the self-attention mechanism, the embedding vector of each word is split into three vectors, namely the query vector (Q), the key vector (K), and the value vector (V), which are obtained from the original embedding vector through linear transformation;

[0126] Step SE2: For each pair of words in the sentence, calculate the similarity score between their query vector and key vector, usually using dot product or scaled dot product;

[0127] Step SE3: Normalize the similarity scores through the softmax function to obtain attention weights, which reflect the importance of each word to other words;

[0128] Step SE4: Use the attention weights to perform weighted summation on the value vector to obtain a new representation for each word, which incorporates the information of other words in the sentence.

[0129] Specifically, for the feedforward neural network layer, the feedforward neural network layer follows the self-attention mechanism, and it further processes the new representation of each word output by the self-attention mechanism. The feedforward neural network layer is usually a multi-layer perceptron (MLP), which contains multiple linear transformation layers and nonlinear activation functions (such as ReLU). The role of the feedforward neural network layer is to perform nonlinear transformation on the dependency captured by the self-attention mechanism, thereby extracting higher-level features.

[0130] Specifically, for the integration of the self-attention mechanism and the feedforward neural network layer, the self-attention mechanism and the feedforward neural network layer are stacked alternately to form the so-called "encoder block". Each encoder block receives the embedding vector of a sentence as input and outputs an updated vector. This vector contains the interdependence between different parts of the sentence. By stacking multiple encoder blocks, the model can gradually deepen its understanding of the sentence structure. After a series of processing by multiple encoder blocks, the final output is a new vector. This new vector is a high-level representation of the sentence embedding vector. It not only contains the information of each word in the sentence, but also captures the dependency between words through the self-attention mechanism. This new vector can be used for various NLP tasks, such as text classification, sentiment analysis, machine translation, etc.

[0131] Specifically, for the averaging of the new vectors, before averaging, it is necessary to ensure that the new vectors of the sentence have the same dimension. This is because mathematically, only vectors with the same dimension can perform element-by-element operations. The vector representations of all words in the sentence are averaged element by element. Specifically, the values ​​of all vectors in the same dimension are added together, and then divided by the number of vectors, that is, the number of words in the sentence. This process will generate a new vector whose value in each dimension is the average of the values ​​of the original vector in that dimension. For example, suppose the sentence "The cat sits on the mat" is encoded into 5 word vectors, each of which is 768-dimensional. Then, these 5 768-dimensional vectors are averaged element by element, and finally a 768-dimensional average vector is obtained as the feature vector of the entire sentence.

[0132] It should be noted that the feature vector can be generated in the following ways. This embodiment adopts the way 1. In other embodiments, the way 2, way 3 or way 4 can be adopted:

[0133] Method 1: Average pooling: average all vectors output by the encoder to obtain an average vector as the representation of the sentence.

[0134] Method 2: Use embedding of special tags. If the input sequence contains a special tag (such as the [CLS] tag in BERT), the encoder output vector corresponding to the tag can be used as the representation of the sentence.

[0135] Method 3: Maximum pooling: Perform a maximum pooling operation on all vectors output by the encoder, select the maximum value on each dimension, and form a new vector as the representation of the sentence.

[0136] Method 4: Weighted average: According to the importance of each token (which can be obtained by attention score, etc.), the encoder output is weighted averaged to obtain the representation of the sentence.

[0137] Step S3: Screen the feature vector of the semantic information contained in each sentence through a classifier to determine the feature vector of the target sentence.

[0138] like Fig.12 As shown, the specific process of step S3 is as follows:

[0139] Step S301: Calculate the probability value of adding the feature vector of the semantic information contained in each sentence into the summary text through the classifier;

[0140] Step S302: Based on the calculated probability value, the feature vector of the semantic information contained in the sentence whose probability value is not less than the preset threshold is determined as the feature vector of the target sentence. The calculation formula of the probability value is:

[0141] Z=W o R i +B o ,

[0142] Y i =σ(z),

[0143] Among them, Ri is the feature vector of the semantic information contained in each sentence, Wo and Bo are the weight and bias of the classifier model respectively, and σ is the sigmoid function, which is defined as:

[0144]

[0145] Specifically, for the classifier, the BERT encoder can get the representation of each token, but here we do not need the representation of each token, but the representation of each sentence, because the extractive summary only selects important sentences. If we can get the representation of each sentence, we can input the representation of the sentence into a classifier to let the classifier determine whether the sentence is important.

[0146] Specifically, the structure of the classifier includes an input layer, a hidden layer, and an output layer. The input layer receives the vector representation of each sentence as input. Depending on the complexity of the task, one or more hidden layers can be designed to extract higher-level features. These layers usually contain linear transformations (such as fully connected layers) and nonlinear activation functions (such as ReLU). The output layer uses the Sigmoid function as an activation function to generate a probability value between 0 and 1 for each sentence. This probability value represents the possibility of the sentence being selected to be added to the summary text.

[0147] Specifically, for the Sigmoid function, the Sigmoid function is a logical function whose output range is between 0 and 1. It is usually used in binary classification problems to compress any real value into the interval (0, 1) to represent the probability of an event. In the task of summary generation, the output of the Sigmoid function can be interpreted as the probability of a sentence being selected to be added to the summary text. The closer the probability value output by the Sigmoid function is to 1, the more likely the sentence is to be selected to be added to the summary text; conversely, the closer the probability value is to 0, the less likely the sentence is to be selected. The sigmoid function and probability screening method do not rely on specific text types or domain knowledge, so they are highly versatile. They can be applied to various types of text summary generation tasks such as news, academic papers, and medical literature, and provide high-quality summaries for different types of text.

[0148] Specifically, for the screening of feature vectors, in practical applications, sentences can be selected based on these probability values ​​to construct summaries. For example, the top N sentences with the highest probability values ​​can be selected as the content of the summary, or a threshold can be set to select only sentences with probability values ​​exceeding the threshold. This probability evaluation method makes the generation of summaries more flexible, and the preset threshold can be adjusted according to different needs to generate summaries of different lengths.

[0149] Specifically, in extractive summarization, we only need to select important sentences. We already know how to obtain the representation of sentences from the above. Now we input these representations into a simple binary classifier to determine whether the input sentence can be added to the summary. This classifier layer is usually called the summary layer. Obviously, only a Sigmoid classifier is needed to calculate this probability. By minimizing the binary classification loss between the true probability and the predicted probability, we can fine-tune the pre-trained BERT model and the summary layer at the same time.

[0150] like Fig.13 As shown, the classifier returns the probability that each sentence can be added to the summary. For each sentence i in the document, a feature vector R is obtained. i , and then input it into the summary layer to get the probability Y that can be added to the summary i .

[0151] Step S4: Decode the feature vector of the selected target sentence through a decoder, and generate a summary text based on the decoding result.

[0152] like Fig.14 As shown, the specific process of step S4 is as follows:

[0153] Step S401: focusing on the feature vector of the target sentence output by the classifier through the cross attention mechanism of the decoder, and converting the feature vector of the target sentence into a corresponding word or phrase;

[0154] Step S402: Based on the feature vector of the target sentence output next time by the classifier and / or the previous output text, the decoder predicts the probability distribution of words or phrases, selects the word or phrase with the highest probability as the output text, and outputs it multiple times;

[0155] Step S403: sorting the output texts outputted multiple times to generate summary texts.

[0156] Specifically, for the decoder, the task of the decoder is to convert the sentence-level representation into a sequence of words or phrases. The decoder usually contains a multi-layer structure, such as a self-attention layer, a cross-attention layer, and a feedforward neural network. The self-attention layer is similar to the self-attention layer in the encoder, but with a masking mechanism. During the decoding process, since future information is unknown, the decoder needs to use a masked self-attention mechanism to cover up the information of future positions, ensuring that only the previous words are considered when generating the next word, thereby avoiding information leakage.

[0157] like Fig.15 As shown, specifically, for conversion into words or phrases through the cross-attention mechanism, the specific process is as follows:

[0158] Step SF1: At each time step of decoding, the decoder’s cross-attention layer receives the feature vector from the classifier as “keys” and “values”, and uses the decoder’s hidden state at the current time step as “queries”. This step allows the decoder to focus on and refer to relevant information in the entire feature vector when generating the next word or phrase.

[0159] Step SF2, by calculating the similarity (such as dot product) between the "query" and each "key", attention scores are obtained, which are then normalized (such as through a softmax function) to indicate the contribution of each value to generating the next output;

[0160] Step SF3: According to the attention score, perform weighted summation on the “value” to obtain a context vector, which contains information from the feature vector that is useful for the current generation step of the decoder.

[0161] like Fig.16 As shown, specifically, for generating words or phrases to form output text, the specific process is as follows:

[0162] Step SG1: After the cross-attention layer, the self-attention layer and feed-forward neural network of the decoder further process the context vector and finally generate an output vector;

[0163] Step SG2: This output vector is mapped to a vector space of the size of the vocabulary, usually through a linear layer and a softmax function. This step converts the output vector into a probability distribution, representing the probability of each word or phrase in the vocabulary being selected.

[0164] Step SG3: According to the probability distribution, select the word or phrase with the highest probability as the output of the current time step. Before generating the next word or phrase, the decoder may add the current output to its hidden state as the input for the next step.

[0165] Specifically, for multiple outputs of the output text, the above steps (from the decoder's cross-attention layer to the vocabulary mapping) are repeated at each time step until the predetermined output length is reached and a specific end symbol (such as <end>), or the confidence of the decoder output is below a certain threshold.

[0166] Specifically, for the process of generating summary text, the context representation of the text is input into the decoder, the hidden state of the decoder is initialized, and then the summary is gradually generated based on the context representation and the previously generated text. The output of the classifier is used as the input of the decoder, and the decoder starts to generate a text summary.

[0167] In other embodiments, in step S4, there is also the following step: decoding the feature vector of the target sentence, and when generating the summary text: masking all information data of the next time node through a masked self-attention mechanism.

[0168] Specifically, for the masked self-attention mechanism, during the decoding process, the next word or character of the target sequence is gradually generated according to the encoded representation provided by the encoder and the generated sequence information. At each time of the generated sequence, the decoder must avoid "peeking" into the future information that has not yet been generated to ensure the logic and coherence of the generation process. The present application applies the masked self-attention mechanism to the decoder, and realizes the masking of future information by introducing a masking matrix in the self-attention calculation process. Specifically, the masking matrix is ​​a matrix of the same shape as the self-attention input sequence, but its element values ​​are set according to whether masking is required (i.e., whether the current position is allowed to pay attention to the future position). During the decoding process, the masking matrix sets all connections pointing to future time steps to invisible or zero values, thereby ensuring that when generating the output of the current position, the decoder only relies on the generated sequence (including all information before the current position) and the output of the encoder, without being disturbed by the future information that has not yet been generated, which not only effectively prevents information leakage, but also makes the decoding process more in line with the logical order of human language generation: that is, from left to right (or from top to bottom, depending on the writing habits of the specific language), gradually constructing the entire sentence or paragraph. In addition, the masked self-attention mechanism also enhances the generalization and robustness of the model because it forces the model to learn predictions based on current and past information rather than relying on unrealistic future information.

[0169] like Fig.17 As shown, a medical literature abstract automatic generation system is used to execute the above-mentioned medical literature abstract automatic generation method, and the system includes: a literature text acquisition module 1, an embedding vector acquisition module 2, a feature vector acquisition module 3, a feature vector screening module 4, and a summary text generation module 5.

[0170] The document text acquisition module 1 is used to acquire the medical document text;

[0171] Embedding vector acquisition module 2, used to convert each sentence in the medical literature text into a corresponding embedding vector;

[0172] A feature vector acquisition module 3, used for converting the embedding vector corresponding to each sentence into a feature vector of semantic information contained in each sentence through an encoder;

[0173] A feature vector screening module 4 is used to screen the feature vectors of the semantic information contained in each sentence through a classifier to determine the feature vector of the target sentence;

[0174] The summary text generation module 5 is used to decode the feature vector of the target sentence through a decoder to generate a summary text.

[0175] like Fig.18 As shown, in other embodiments, a medical literature abstract automatic generation system includes the above-mentioned literature text acquisition module 1, embedding vector acquisition module 2, feature vector acquisition module 3, feature vector screening module 4, abstract text generation module 5, and further includes:

[0176] Data preprocessing module 6, pre-trained language model module 7, multi-task learning framework module 8, post-processing module 9, system integration and testing module 10, user interface and interaction module 11, deployment and maintenance module 12.

[0177] Specifically, the data preprocessing module 6 is responsible for collecting and preprocessing medical literature data, including text cleaning, sentence segmentation, part-of-speech tagging and term standardization, developing automated text cleaning tools to remove noise data, using part-of-speech taggers in the medical field to improve the accuracy of tagging, and transmitting the preprocessed text data to the pre-trained language model module for in-depth analysis. The pre-trained language model module 7 is used to use a large amount of medical literature for pre-training based on the Transformer architecture, learn medical terminology and context, use BERT or its variants as the basic model, adapt to the medical summary task through Fine-tuning, and apply the trained model to the multi-task learning framework module to extract key information and generate summaries.

[0178] Specifically, the multi-task learning framework module 8 is used to design a multi-task learning framework with a shared encoder and an independent decoder, while optimizing the tasks of abstract generation and key information extraction. The self-attention mechanism of the Transformer is used to enable the model to capture long-distance dependencies and improve the accuracy of information extraction. The extracted key information and the generated abstract draft are transmitted to the post-processing module for further optimization. The post-processing module 9 is used to optimize the fluency, check the consistency and refine the generated abstract to ensure the readability and accuracy of the abstract, develop natural language generation (NLG) technology to improve the fluency of the abstract, use a terminology consistency algorithm to ensure the accurate use of medical terms, and transmit the optimized abstract to the system integration and testing module for system-level testing.

[0179] Specifically, the system integration and testing module 10 is used to integrate all modules into a unified system, conduct end-to-end system testing, ensure that all modules work together, implement automated test scripts, simulate user operation processes, evaluate system performance and summary quality, and transmit test results and system performance data to the deployment and maintenance module to provide a basis for system optimization. The user interface and interaction module 11 is used to provide users with an intuitive interface to upload medical literature, view summary results, and provide feedback, implement responsive design, support access from multiple devices, provide visual display and comparison functions for summary results, receive user literature uploads and feedback, and transmit user interaction data to the system integration and testing module. The deployment and maintenance module 12 is responsible for the deployment, monitoring and maintenance of the system, ensure the stable operation of the system, and update it according to feedback, integrate the monitoring system, track system performance indicators in real time, implement automated deployment and rollback mechanisms, and collect system operation data and user feedback for continuous system improvement and functional upgrades.

[0180] It should be understood that the embodiments of the present invention can be implemented in hardware, software or a combination thereof. Software can be stored in a volatile or non-volatile storage device (regardless of whether the data is deletable or rewritable) like a ROM, in a memory like a RAM, a memory chip, a device or an integrated circuit, or in a storage medium on which data can be optically or magnetically recorded and from which data can be read by a machine (e.g., a computer), such as a compact disc (CD), a digital versatile disc (DVD), a disk, or a tape. Moreover, the embodiments of the present invention can be implemented in a computer or portable terminal having a controller and a memory, and the memory is an example of a non-temporary computer-readable storage medium suitable for storing one or more programs including commands for implementing the embodiments of the present invention. Therefore, the present invention includes a program having a code for implementing the device or method defined by the claims, and a machine-readable non-temporary storage medium storing the program. The program can be transferred electrically by a medium such as a communication signal transmitted via a wired or wireless connection, and the program and its equivalent are included in the present invention.

[0181] The device may receive a program from a program providing device connected via a cable or wirelessly and store the program. The program providing device may include a program containing commands for implementing the embodiments of the present invention, a memory for storing information required by the embodiments of the present invention, a communication module for communicating with the mobile device via a cable or wirelessly, and a controller for transmitting the program to the device automatically or upon request of the device.

[0182] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.< / end>

Claims

1. A method for automatically generating medical literature abstracts, characterized in that: include: Access medical literature texts; Convert each sentence in the medical literature text into a corresponding embedding vector; Convert the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence; Screening the feature vectors of the semantic information contained in each sentence to determine the feature vector of the target sentence, wherein the feature vector of the target sentence is used to indicate that the target sentence can be added to the summary text; The feature vector of the target sentence is decoded to generate a summary text.

2. The method for automatically generating medical literature abstracts according to claim 1, characterized in that: The converting each sentence in the medical document text into a corresponding embedding vector comprises: Encode each word of each sentence into a token embedding vector; Encode the position information of each word in the sentence into a position embedding vector; Assigning a specific fragment embedding vector to each of the sentences, wherein the specific fragment embedding vector is used to distinguish vector representations of different sentences; The embedding vector is obtained by summing the tag embedding vector, the position embedding vector and the fragment embedding vector.

3. The method for automatically generating medical literature abstracts according to claim 1, characterized in that: The converting the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence comprises: Converting the embedding vector into a first vector, wherein the first vector includes the interdependencies between different parts within each sentence; An average vector is obtained by performing an average calculation on the first vectors, and the average vector is used as a feature vector of each sentence.

4. The method for automatically generating medical literature abstracts according to claim 1, characterized in that: The screening of the feature vector of the semantic information contained in each sentence to determine the feature vector of the target sentence includes: Calculate the probability value of adding the feature vector of the semantic information contained in each sentence into the summary text; The feature vector of the semantic information contained in the sentence whose probability value is not less than a preset threshold is determined as the feature vector of the target sentence.

5. The method for automatically generating medical literature abstracts according to claim 4, characterized in that: The calculation formula of the probability value is: Z=W o R i +B o , Y i =σ(z), Among them, R i is the feature vector of the semantic information contained in each sentence, W o and B o are the weights and biases of the classifier model, and σ is the sigmoid function, defined as:

6. The method for automatically generating medical literature abstracts according to claim 1, characterized in that: The decoding of the feature vector of the target sentence to generate a summary text comprises: Focusing on the feature vector of the target sentence through a cross attention mechanism, and converting the feature vector of the target sentence into a corresponding word or phrase; Based on the feature vector of the target sentence to be output next time and / or the previous output text, by predicting the probability distribution of words or phrases, the words or phrases with the highest probability are selected as the output text, and the output text is output multiple times; Sort the output text of multiple outputs to generate summary text.

7. The method for automatically generating medical literature abstracts according to claim 6, characterized in that: When decoding the feature vector of the target sentence and generating a summary text: all information data of the next time node is masked through a masked self-attention mechanism.

8. A medical literature abstract automatic generation system, characterized in that: include: A document text acquisition module is used to acquire medical document text; An embedding vector acquisition module is used to convert each sentence in the medical literature text into a corresponding embedding vector; A feature vector acquisition module, used to convert the embedding vector corresponding to each sentence into a feature vector of the semantic information contained in each sentence; A feature vector screening module, used to screen the feature vectors of the semantic information contained in each sentence to determine the feature vector of the target sentence; The summary text generation module is used to decode the feature vector of the target sentence to generate a summary text.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically generating medical literature abstracts according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the method for automatically generating medical literature abstracts according to any one of claims 1 to 7 are implemented.

Images