A normative medical text rewriting method based on a generative adversarial network

Abstract

The application discloses a kind of based on the normative medical text rewriting method of generative adversarial network, comprising: extracting medical question and answer corpus of colloquialization and standardization is handled, obtains data set;Using Transformer model constructs standardization medical text generator and colloquialization medical text generator, pretraining is carried out through user health term mapping table, obtains standardization medical text;Using LSTM neural network constructs standardization medical text discriminator and colloquialization medical text discriminator;Combined with medical text features, standardization medical text discriminator and colloquialization medical text discriminator are respectively optimized using loss function;Standardization medical text generator and colloquialization medical text generator are optimized using reinforcement learning mode.The application realizes the mutual migration rewriting between colloquialization text and standardization text, solves the problem of excessive dependence on labeled corpus of traditional text migration model, so that the model is still reliable without parallel corpus, reduces the workload of artificial labeling data.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a method for rewriting standard medical text based on generative adversarial networks. Background Technology

[0002] Text style transfer has always been a hot topic in the field of natural language generation. It means transforming or generating text with a specific style or attributes while preserving the original text's semantic content, ensuring the newly generated text is fluent and realistic. Beyond transferring writing style or sentiment, text style transfer can be applied to chatbot dialogue systems, text rewriting, and the checking or generation of professional copywriting documents. Most existing text generation models suffer from training difficulties and generate content with grammatical errors or semantic gaps. The application of text style transfer models can flexibly alleviate the training difficulties of text generation models.

[0003] In recent years, the development of deep learning technology has enabled natural language processing to be widely applied in various scenarios and complex tasks. In the medical field, online consultation technology has gradually become widespread, and the establishment of various medical and health websites allows patients to conduct self-diagnosis through question-and-answer methods without leaving home. However, due to a lack of professional medical knowledge, users often encounter problems such as unclear descriptions of their symptoms and colloquial expressions when using these platform tools, leading to comprehension difficulties for AI-assisted diagnosis. This obstacle is not limited to machine reading comprehension; it is often bidirectional. The patient's colloquial descriptions or the doctor's professional terminology create communication barriers between doctors and patients, resulting in low efficiency in online consultations. Therefore, the application of text style transfer technology in text rewriting and text standardization provides a good solution to these problems.

[0004] Currently, text style transfer methods can be broadly categorized into two types: supervised learning and unsupervised learning. Supervised learning, similar to machine translation, uses parallel datasets for style transfer, resulting in high-precision and effective text conversion. Most existing text style transfer models also employ this end-to-end model, similar to statistical machine translation. However, these models lack labeled corpora, and manually labeling data requires significant human and material resources. Therefore, research on text style transfer models has shifted towards unsupervised learning methods.

[0005] Compared to supervised style transfer models, similar to machine translation, unsupervised learning models can effectively separate text attributes from content, train models without requiring large amounts of paired data, and produce ideal generated text. However, current research progress on unsupervised text style transfer models is far slower than that on image style transfer because of the text discreteness problem when applying style transfer to text. This discreteness leads to a loss of text fluency and content integrity during the transfer process, resulting in low-quality generated text and poor generalization. Secondly, evaluating model quality is difficult. Unlike image style discrimination, language style is vaguely defined, making it even more challenging. Summary of the Invention

[0006] The purpose of this invention is to provide a standardized medical text rewriting method based on generative adversarial networks, enabling bidirectional conversion between patients' colloquial descriptions of their conditions and the professional standardized terminology used by doctors and AI-assisted diagnoses.

[0007] To achieve the above objectives, this application proposes a canonical medical text rewriting method based on generative adversarial networks, comprising:

[0008] We extracted and processed colloquial and standardized medical Q&A data to obtain a dataset.

[0009] A standardized medical text generator is built using the Transformer model. With conversational medical text generator Standardized medical texts are obtained through pre-training using a user health terminology mapping table.

[0010] A normalized medical text discriminator D is constructed using an LSTM neural network. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X);

[0011] Combining medical text features, a loss function is used to optimize the normalized medical text discriminator D. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X);

[0012] Optimizing the Standardized Medical Text Generator using Reinforcement Learning With conversational medical text generator

[0013] Furthermore, colloquial statements in the dataset are used as X-style samples, while statements containing standardized terms are used as pseudo-parallel samples for the Y-target style to be converted. Colloquial statements in the test set that can be mapped to terminology are labeled using a user health terminology mapping table, and this annotation is provided as a hidden layer to the standardized medical text generator.

[0014] Furthermore, the dataset includes: dataset X = {x1, x2, ..., x...} i ,…,x n The dataset Y = {y1, y2, ..., y} i ,…,y n}, where i represents the i-th sample, n represents the total number of samples, and x and y represent the colloquial style sample statement and the standardized style sample statement, respectively; the colloquial style sample statement is represented as: This represents the t-th word in the sentence, where T represents the sentence length, i.e., the number of words.

[0015] To link colloquial-style sample sentences with standardized-style sample sentences, after identifying medical entities in each sentence through word segmentation, the non-standardized colloquial-style sample sentences are annotated using a user health terminology mapping table. The annotation sequence is denoted as follows: Sample statements that require standardization are marked with 1, while sample statements that do not require standardization are marked with 0.

[0016] Furthermore, a standardized medical text generator is constructed using the Transformer model. With conversational medical text generator Specifically, a standardized medical text generator is constructed using the CycleGAN architecture. With conversational medical text generator The two generators generate in opposite directions, and when connected, they can form a closed loop and provide feedback information to each other.

[0017] Furthermore, a normalized medical text generator is pre-trained using maximum likelihood estimation. With conversational medical text generator The specific method is as follows: the maximum length of the generated sentence is set to 30 words, the given word embedding dimension Embedding_size is 512, and both the encoder and decoder are six-layer structures; the user health term mapping table is set as the generated word list, and word vectors are pre-trained together using the training set divided from colloquial style sample sentences and normalized style sample sentences to generate the initial value of the Embedding corresponding to the word.

[0018] Furthermore, an LSTM neural network is used to construct a standardized medical text discriminator D. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X), specifically:

[0019] The last hidden layer H of the LSTM neural networkn Replace with a binary logistic regression layer to determine whether the input medical text comes from real samples in dataset Y or samples generated by a normalized medical text generator.

[0020] The high-dimensional sequence of input medical text is non-linearly transformed to obtain the embeddings of the words in the sequence. These embeddings are then input into each basic cell and combined with a fully connected hidden layer to obtain the probability of outputting each word.

[0021] Furthermore, the loss function is used to optimize the normalized medical text discriminator D. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X), specifically:

[0022] With the generator's adjustable parameter θ1 fixed, real samples from dataset Y and samples generated by a normalized medical text generator were randomly sampled. Then minimize the cross-entropy; normalize the medical text discriminator D. Φ1(Y) The loss function for (Y) is as follows:

[0023]

[0024]

[0025] L all =β1L1+β2L2

[0026] Where L1 is the generative adversarial loss of the normalized medical text discriminator, β1 is the loss term coefficient, L2 is the sequence labeling loss, β2 is the loss term coefficient, and the range of β1 and β2 is less than 0.5;

[0027] With the generator's adjustable parameter θ2 fixed, real samples and samples generated by the colloquial medical text generator were randomly sampled from dataset X. Then minimize the cross-entropy; the colloquial medical text discriminator D Φ2(X) The loss function for (X) is as follows:

[0028]

[0029]

[0030] L' all =β1L'1+β2L'2

[0031] Where L'1 is the generative adversarial loss of the colloquial medical text discriminator, β1 is the loss term coefficient, L'2 is the sequence labeling loss, β2 is the loss term coefficient, and the range of β1 and β2 is less than 0.5.

[0032] Furthermore, reinforcement learning is employed to optimize the standardized medical text generator, specifically as follows:

[0033] The standardized medical text generator is optimized by minimizing the cross-entropy loss function:

[0034]

[0035] The gradient of the above formula is equal to the following:

[0036]

[0037] By combining reinforcement learning mechanisms, a standardized medical text generator can be obtained. The reward function is:

[0038]

[0039] Among them, R s To improve the style accuracy of generated text, R c α represents the semantic retention rate, and α is the harmonic weight parameter, with a value ranging from 0 to 1.

[0040] For the style accuracy reward of the generated text, the probability value of the generated sentence in the target style distribution is calculated, and the probability value is used as the reward function, as shown in the formula:

[0041]

[0042] The reward for semantic preservation is calculated using the cosine similarity between the generated statement and the original statement embedding, denoted as:

[0043]

[0044] The standardized medical text generator Expected reward is expressed as:

[0045]

[0046] in, This represents the probability that the generator will generate a statement. This means that when the generator receives the first t-1 words, it randomly samples the t-th word and records it as... This represents the expected future reward at the current position t-1.

[0047] Furthermore, reinforcement learning is used to optimize the spoken medical text generator, specifically:

[0048] The conversational medical text generator is optimized by minimizing the cross-entropy loss function.

[0049]

[0050] The gradient of the above formula is equal to the following:

[0051]

[0052] By combining reinforcement learning mechanisms, a conversational medical text generator can be obtained. The reward function is:

[0053]

[0054] Among them, R s To improve the style accuracy of generated text, R c α represents the semantic retention rate, and α is the harmonic weight parameter, with a value ranging from 0 to 1.

[0055] For the style accuracy reward of the generated text, the probability value of the generated sentence in the colloquial style distribution is calculated, and the probability value is used as the reward function, as shown in the formula:

[0056]

[0057] The reward for semantic preservation is calculated using the cosine similarity between the generated statement and the original statement embedding, denoted as:

[0058]

[0059] The colloquial medical text generator Expected reward is expressed as:

[0060]

[0061] in, This represents the probability that the generator will generate a statement. This means that when the generator receives the first t-1 words, it randomly samples the t-th word and records it as... This represents the expected future reward at the current position t-1.

[0062] Furthermore, in the process of optimizing the standardized medical text discriminator, the colloquial medical text discriminator, the standardized medical text generator, and the colloquial medical text generator, discriminators and generators with opposite objectives are introduced to compete with each other until a Nash equilibrium is reached.

[0063] Compared with existing technologies, the technical solutions adopted in this invention have the following advantages: This invention enables mutual transfer and rewriting between colloquial and standardized text, solving the problem of excessive reliance on labeled corpora in traditional text transfer models. This ensures the model remains reliable even without parallel corpora, reducing the workload of manually labeled data. By using a Transformer as the generator and introducing reinforcement learning for optimization, it addresses the issue that the Generative Adversarial Network (CycleGAN) model cannot be optimized using gradient descent in the text domain. Furthermore, it fundamentally avoids the exposure bias problem caused by traditional maximum likelihood estimation optimization methods and ensures model stability. Attached Figure Description

[0064] Figure 1 This is a schematic diagram of the improved generative adversarial network model structure of the present invention;

[0065] Figure 2 This is a diagram of the improved generator model of the present invention;

[0066] Figure 3 This is a diagram showing the result of rewriting the code based on a portion of the test set samples in this invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the application; that is, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0068] Example 1

[0069] This embodiment provides a method for rewriting canonical medical text based on generative adversarial networks, characterized by including:

[0070] Step S1: Extract and process the colloquial and standardized medical question-and-answer data to obtain a dataset;

[0071] Specifically, this application selects and processes the cMedQQ paraphrase recognition dataset provided by CBLUE. The purpose of paraphrase recognition is to determine whether two statements have the same semantic meaning, expressing the same meaning in different statements. cMedQQ contains 16,070 pairs of training sentences and 2,000 test sentences. Colloquial sentences in the dataset are used as X-style samples, and sentences containing standardized words are used as pseudo-parallel samples for the Y-style target to be converted. To ensure the stability of the sentences generated by the model, a user health terminology mapping table is additionally used to annotate colloquial sentences in the test set that can be mapped to terms, which are then provided as a hidden layer to the standardized medical text generator.

[0072] The dataset includes: dataset X = {x1, x2, ..., x} i ,…,x n The dataset Y = {y1, y2, ..., y} i ,…,y n}, where i represents the i-th sample, n represents the total number of samples, and x and y represent the colloquial style sample statement and the standardized style sample statement, respectively; the colloquial style sample statement is represented as: This represents the t-th word in the sentence, where T represents the sentence length, i.e., the number of words.

[0073] To link colloquial-style sample sentences with standardized-style sample sentences, after identifying medical entities in each sentence through word segmentation, the non-standardized colloquial-style sample sentences are annotated using a user health terminology mapping table. The annotation sequence is denoted as follows: Sample statements that require standardization are marked with 1, while sample statements that do not require standardization are marked with 0.

[0074] Step S2: Construct a standardized medical text generator using the Transformer model. With conversational medical text generator Standardized medical texts are obtained through pre-training using a user health terminology mapping table.

[0075] Specifically, since the purpose of this application is to achieve mutual conversion between colloquial text and standardized text containing technical terminology, a CycleGAN structure was chosen to construct two generators. The standardized medical text generator... With conversational medical text generator The two generators generate in opposite directions, so connecting them can form a closed loop that provides feedback to each other and optimizes the model. This solves the problem of no paired data in supervised learning models, and compared to other unsupervised learning models, the generated sentences are relatively stable, ensuring the quality of text conversion.

[0076] A normalized medical text generator was pre-trained using maximum likelihood estimation. With conversational medical text generator The specific method is as follows: the maximum length of the generated sentence is set to 30 words, the given word embedding dimension Embedding_size is 512, and both the encoder and decoder are six-layer structures; the user health term mapping table is set as the generated word list, and word vectors are pre-trained together using the training set divided from colloquial style sample sentences and normalized style sample sentences to generate the initial value of the Embedding corresponding to the word.

[0077] Step S3: Construct a standardized medical text discriminator D using an LSTM neural network. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X);

[0078] Specifically, the last hidden layer H of the LSTM neural network n Replace with a binary logistic regression layer to determine whether the input medical text comes from real samples in dataset Y or samples generated by a normalized medical text generator.

[0079] For the discriminator's input, unlike the one-hot vectors used in traditional generative adversarial networks, this method chooses embeddings as the discriminator's input. One-hot encoding cannot represent the similarity between two different words. For example, the one-hot encoded vectors of the words "double lungs" and "right hilum" have a similarity of 0, failing to reflect the relationship between them. Furthermore, when the input sequence is long, the one-hot encoding matrix has excessively high dimensionality, which limits the model's performance to some extent. Therefore, this method first performs a non-linear transformation on the high-dimensional sequence of the input medical text to obtain the embeddings of the words in the sequence, and then inputs them into each basic unit cell, combining them with a fully connected hidden layer to obtain the probability of outputting each word.

[0080] Step S4: Combining medical text features, optimize the normalized medical text discriminator D using a loss function. Φ1(Y) (Y) and the colloquial medical text discriminator D Φ2(X) (X);

[0081] Specifically, with the generator's adjustable parameter θ1 fixed, real samples from the randomly sampled dataset Y and samples generated by the normalized medical text generator are used. Then minimize the cross-entropy; normalize the medical text discriminator D. Φ1(Y) The loss function for (Y) is as follows:

[0082]

[0083]

[0084] L all =β1L1+β2L2

[0085] Where L1 is the generative adversarial loss of the normalized medical text discriminator, β1 is the loss term coefficient, L2 is the sequence labeling loss, β2 is the loss term coefficient, and the range of β1 and β2 is less than 0.5;

[0086] With the generator's adjustable parameter θ2 fixed, real samples and samples generated by the colloquial medical text generator were randomly sampled from dataset X. Then minimize the cross-entropy; the colloquial medical text discriminator D Φ2(X) The loss function for (X) is as follows:

[0087]

[0088]

[0089] L' all =β1L'1+β2L'2

[0090] Where L'1 is the generative adversarial loss of the colloquial medical text discriminator, β1 is the loss term coefficient, L'2 is the sequence labeling loss, β2 is the loss term coefficient, and the range of β1 and β2 is less than 0.5.

[0091] Step S5: Optimize the standardized medical text generator using reinforcement learning. With conversational medical text generator

[0092] Specifically, reinforcement learning addresses the limitations of traditional gradient descent optimization and the lack of differentiability issues when applying Cycle GAN structures to text, by eliminating reliance on gradient optimization. Reinforcement learning uses agents to continuously interact and correct errors in the environment, learning a policy that maximizes rewards. However, typical reinforcement learning strategies rely on the discriminator providing an overall reward value at the end of the sequence, which remains unstable for discrete data like text, potentially leading to word order confusion or generating sentences completely unrelated to the original meaning. Therefore, it's crucial to consider future output at each time step.

[0093] Step S6, repeat steps S4-S5 until Nash equilibrium is reached.

[0094] Specifically, to make the normalized colloquial medical text more closely resemble real text, the above steps are iterated repeatedly to improve the performance of the normalized medical text generator. Therefore, a discriminator with opposing objectives is introduced to compete against the generator. Through continuous competition, performance is improved, and the model training reaches an ideal Nash equilibrium state, where the distribution of samples generated by the generator is consistent with the distribution in the dataset, and the discriminator cannot completely distinguish between the generated samples and samples obtained from the real dataset. This improves the model performance for transferring between colloquial and normalized text.

[0095] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A method for rewriting canonical medical text based on generative adversarial networks, characterized in that, include: We extracted and processed colloquial and standardized medical Q&A data to obtain a dataset. A standardized medical text generator is built using the Transformer model. With conversational medical text generator Standardized medical text is obtained through pre-training using a user health terminology mapping table. A standardized medical text discriminator was constructed using an LSTM neural network. Conversational Medical Text Discriminator ; By combining medical text features, loss functions are used to optimize the normalized medical text discriminator. Conversational Medical Text Discriminator ; Optimizing the Standardized Medical Text Generator using Reinforcement Learning With conversational medical text generator ; A standardized medical text generator is built using the Transformer model. With conversational medical text generator Specifically, a standardized medical text generator is constructed using the CycleGAN structure. With conversational medical text generator The two generators generate in opposite directions, and when connected, they can form a closed loop and provide feedback information to each other. A normalized medical text generator was pre-trained using maximum likelihood estimation. With conversational medical text generator The specific method is as follows: the maximum length of the generated sentence is set to 30 words, the given word embedding dimension Embedding_size is 512, and both the encoder and decoder are six-layer structures; the user health term mapping table is set as the generated word list, and word vectors are pre-trained together using the training set divided from colloquial style sample sentences and normalized style sample sentences to generate the initial value of the Embedding corresponding to the word; The normalized medical text discriminator is optimized using the loss function. Conversational Medical Text Discriminator Specifically: exist Random sampling under fixed conditions Real samples and samples generated by a standardized medical text generator Then minimize the cross-entropy; normalize the medical text discriminator. The loss function is as follows: in To standardize the generative adversarial loss of medical text discriminators, For the loss term coefficient, For sequence labeling loss, For the loss term coefficient, and The range is less than 0.5; exist Random sampling under fixed conditions Real samples and samples generated by the colloquial medical text generator Then minimize the cross-entropy; a colloquial medical text discriminator. The loss function is as follows: Generative adversarial loss for discriminators of colloquial medical text. For the loss term coefficient, For sequence labeling loss, For the loss term coefficient, and The range is less than 0.5; In the process of optimizing the standardized medical text discriminator, the colloquial medical text discriminator, the standardized medical text generator, and the colloquial medical text generator, discriminators and generators with opposite objectives are introduced to compete with each other until a Nash equilibrium is reached. The dataset contains colloquial statements as X-style samples, and statements containing standardized terms as pseudo-parallel samples for the Y-style target to be converted. Colloquial statements in the test set that can be mapped to terminology are labeled using a user health terminology mapping table, and this annotation is provided as a hidden layer to the standardized medical text generator. ; The dataset includes: dataset , Where i represents the i-th sample, n represents the total number of samples, and x and y represent the colloquial style sample statement and the standardized style sample statement, respectively; the colloquial style sample statement is represented as: , This represents the t-th word in the sentence, where T represents the sentence length, i.e., the number of words. To link colloquial-style sample sentences with standardized sample sentences, after identifying medical entities in each sentence through word segmentation, the non-standardized colloquial-style sample sentences are annotated using a user health terminology mapping table. The annotation sequence is denoted as follows: The sample statements that need to be standardized are marked with 1, while the sample statements that do not need to be standardized are marked with 0.

2. The method for rewriting canonical medical text based on generative adversarial networks according to claim 1, characterized in that, A standardized medical text discriminator was constructed using an LSTM neural network. Conversational Medical Text Discriminator Specifically: The last hidden layer of the LSTM neural network Replace with a binary logistic regression layer. It is confirmed that the input medical text is from The real samples are still those generated by the standardized medical text generator. ; The high-dimensional sequence of input medical text is non-linearly transformed to obtain the embeddings of the words in the sequence. These embeddings are then input into each basic cell and combined with a fully connected hidden layer to obtain the probability of outputting each word.

Images