Medical big language question and answer method based on diagnosis and treatment data desensitization

Abstract

The invention discloses a medical big language question and answer method based on data desensitization, and belongs to the technical field of data processing, and the method comprises the steps: obtaining training data in a medical field corpus, carrying out the preprocessing, and obtaining the processed training data through a data desensitization model; establishing a medical big language question and answer model based on the improved enhanced Transform sub-model; inputting the processed training data into a medical big language question and answer model for training to obtain a trained medical big language question and answer model; and performing medical question and answer by using the trained medical big language question and answer model. According to the method, an enhanced Transform module is adopted, the reasoning ability for long-distance context is enhanced, the generalization ability of the model for medical professional knowledge is improved through preprocessing, and more accurate and professional suggestions can be generated; through the data desensitization model, sensitive information in the medical data is identified and converted, and the accuracy and efficiency of privacy protection are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and specifically relates to a medical big language question and answer method based on desensitization of diagnosis and treatment data. Background Art

[0002] Medical question answering is an important application of big models in the medical industry. It uses the powerful capabilities of big model technology in language understanding and reasoning, and provides accurate treatment inquiries, medical consultations, and medication guidance by learning professional knowledge in the medical field, meeting patients' personalized medical needs and improving the efficiency and quality of medical services. Existing medical question answering systems play an important role in the field of smart medicine, but these technologies still face some significant challenges when processing medical data.

[0003] The medical question-answering system based on knowledge base keyword matching technology provides answers by matching with predefined content. This method is inefficient in dealing with complex medical problems and is difficult to expand to large-scale data applications. In addition, keyword matching technology does not effectively identify and protect sensitive information in medical data, increasing the risk of privacy leakage.

[0004] The semantic information in the question is extracted based on the long short-term memory network (LSTM) and the gated recurrent unit (GRU), and the learned medical memory knowledge and attention algorithm are integrated to generate relevant medical advice. However, due to the poor learning effect of long-distance dependencies, the algorithm has limited understanding and expression capabilities of natural language. Although the effect is better than keyword matching, it still cannot meet the requirements of actual applications.

[0005] Medical question-answering based on Transformer and BERT large language models improves the accuracy and intelligence of answers generated by the model. The output of the BERT pre-trained model is used as semantic embedding to design a BERT question-answering model based on the self-attention mechanism. However, the model has not been explored on large-scale data and has failed to deeply tap the potential of the model. In addition, although the existing algorithms manually or automatically desensitize the original data in the data processing part, they do not pay attention to whether the medical text still contains identifiable patient private information. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides a medical question-answering language system based on desensitization of diagnosis and treatment data, which solves the problems existing in the prior art:

[0007] (1) Some large model technologies have limited ability to understand and infer long-distance contexts, especially when complex background knowledge is involved in medical question answering, where the model's ability to understand and infer contextual semantics is limited;

[0008] (2) The existing medical big language question-answering model has insufficient generalization capabilities for medical expertise, and the generated diagnosis and treatment recommendations lack accuracy and professionalism;

[0009] (3) Although existing algorithms manually or automatically desensitize the original data during data processing, they do not pay attention to whether the medical text still contains identifiable patient private information.

[0010] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a medical big language question answering method based on data desensitization, comprising the following steps:

[0011] S1: Obtain training data from the medical field corpus;

[0012] S2: Preprocess the training data and perform data desensitization through a data desensitization model to obtain processed training data;

[0013] S3: Establishing a medical large language question answering model based on the improved enhanced Transformer sub-model;

[0014] S4: inputting the processed training data into the medical big language question answering model for training, thereby obtaining a trained medical big language question answering model;

[0015] S5: Use the trained medical big language question-answering model to conduct medical question-answering.

[0016] The beneficial effects of the present invention are as follows: the present invention proposes a large language model medical question-answering algorithm based on desensitized diagnosis and treatment data, uses an enhanced Transformer module to mine the contextual dependencies of text data, improves the model's generalization ability of language knowledge and medical expertise through pre-training and supervised fine-tuning, performs privacy protection optimization during training, and maximizes the level of privacy protection.

[0017] Further: the preprocessing includes supervised fine-tuning, text deduplication and word segmentation processing;

[0018] The supervised fine-tuning is used to fine-tune the parameters of the training data according to the professional dataset;

[0019] The text deduplication is used to remove identical texts;

[0020] The word segmentation process is used to segment the text to obtain word tokens and obtain a word token dictionary by statistics.

[0021] The beneficial effect of the above further scheme is that through supervised fine-tuning, text deduplication and word segmentation processing, the quality of training data can be improved and the training effect of the model can be improved.

[0022] Further: The data desensitization model is established as follows:

[0023] A1: Obtain medical field corpus;

[0024] A2: Extract privacy entities from each data entry in the medical domain corpus through manual specification and rule-based recognition;

[0025] A3: Input data entries and privacy entities into the LLM large language model;

[0026] A4: Prompt and guide the LLM large language model to perform anonymization processing;

[0027] A5: Output the replacement text of the privacy entity through the LLM large language model;

[0028] A6: Train the LLM large language model using data entries, privacy entities, and replacement texts, and evaluate the privacy protection performance;

[0029] A7: After evaluating the privacy protection performance, the data desensitization model is obtained.

[0030] The beneficial effect of the above further scheme is: by establishing a data desensitization model, it is possible to identify and replace sensitive data in medical data, eliminate privacy risks, and enhance the ability to protect patients' private information.

[0031] Further: The specific steps of evaluating the privacy protection performance are as follows:

[0032] Concatenate the replacement text with the privacy inference instruction and input it into the LLM large language model to output the inference result of the private information;

[0033] Compare the inference results with the real private information and calculate the privacy score based on the comparison results;

[0034] Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

[0035] The beneficial effects of the above further scheme are: by replacing text and privacy reasoning instructions, evaluating the privacy protection performance, and setting the data desensitization capability of the privacy score judgment model, it can ensure that the model's inference ability for private text meets the requirements and maximize the level of privacy protection.

[0036] Further: The expression of the privacy score is as follows:

[0037] p t =rank(y,{y′ 1 ,y′ 2 ,…,y′ K})

[0038] Among them, p t is the privacy score, rank is the score, y is the real private information, {y′ 1 ,y′ 2 ,…,y′ K} is the geometry of the inference result, y′ 1 , y′ 2 and y′ K These are all single inference results output by the LLM large language model.

[0039] The beneficial effect of the above further scheme is: the desensitization ability of the model is quantified through the privacy score, which makes it easier for technical personnel to judge the desensitization ability of the model and provide guidance to meet usage needs.

[0040] Further: the medical large language question answering model includes an analysis module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module and a data desensitization module connected in sequence;

[0041] The word segmentation module is used to segment the input question to obtain word tokens;

[0042] The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary, and map them to a high-dimensional semantic feature space through a word embedding matrix to obtain a question feature matrix;

[0043] The rotation position encoding module is used to add the rotation position encoding matrix to the problem feature matrix to obtain a feature matrix that integrates the position information;

[0044] The enhanced Transformer sub-model is used to process the feature matrix of the fused position information and output feature information;

[0045] The word prediction module is used to map feature information and infer and output question-answering results.

[0046] The beneficial effect of the above further scheme is: establishing a medical large language question and answer model based on several layers of enhanced Transformer sub-models can enhance the reasoning ability of the question and answer model for long-distance contexts and ensure that diagnosis and treatment recommendations are more in line with clinical needs.

[0047] Further: the enhanced Transformer sub-model includes: an RMS regularization sub-module, a self-attention sub-module and a feedforward perception network sub-module connected in sequence;

[0048] The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting;

[0049] The self-attention submodule is used to mine the association relationship between different word tokens in different sub-semantic spaces and generate attention features;

[0050] The feedforward perception network submodule is used to fuse the attention features and output the fused features.

[0051] The beneficial effects of the above further scheme are: the stability of the input features is ensured through the RMS regularization sub-module, and the feature distribution of the attention layer output can be stabilized through the self-attention sub-module, ensuring that the feedforward perception network sub-module can efficiently process feature information, so that the enhanced Transformer sub-model can capture long-distance dependencies while not losing information or interfering due to feature mismatches between layers, ultimately improving the generalization ability and training effect of the model.

[0052] Further: The expression of the enhanced Transformer sub-model is as follows:

[0053]

[0054] in, is the fusion feature output by the l-th layer enhanced Transformer sub-model, and are all learnable parameters of the linear mapping layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, is the input feature, RMSNorm is RMS regularization, Concat is the concatenation and fusion of feature matrices, and Softmax is the activation function.

[0055] The beneficial effects of the above further scheme are as follows: the mathematical expression of the enhanced Transformer sub-model can specifically represent the structure of each module. The enhanced Transformer sub-model introduces the RMS regularization layer at the beginning of the calculation to ensure the stability of the input features. The feedforward perception network sub-module uses the RMS regularization layer again, which can further stabilize the feature distribution of the attention layer output and optimize the calculation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a medical big language question answering method based on data desensitization;

[0057] Figure 2 Schematic diagram of word embedding. DETAILED DESCRIPTION

[0058] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0059] like Figure 1 As shown, a medical big language question answering method based on data desensitization includes the following steps:

[0060] S1: Obtain training data from the medical field corpus;

[0061] S2: Preprocess the training data and perform data desensitization through a data desensitization model to obtain processed training data;

[0062] S3: Establishing a medical large language question answering model based on the improved enhanced Transformer sub-model;

[0063] S4: inputting the processed training data into the medical big language question answering model for training, thereby obtaining a trained medical big language question answering model;

[0064] S5: Use the trained medical big language question-answering model to conduct medical question-answering.

[0065] In one embodiment of the present invention, S1 obtains training data from a medical field corpus, including: Wiki Chinese Encyclopedia, Baidu Encyclopedia, C4 dataset, Chinese Encyclopedia open source dataset, shibing624 medical field data, and actual medical dataset. After preprocessing and ChatGLM-6B word segmentation processing, the scale of the training data is approximately 63.4 billion word tokens.

[0066] In one embodiment of the present invention, preprocessing includes supervised fine-tuning, text deduplication and word segmentation processing;

[0067] Supervised fine-tuning is used to fine-tune the parameters of the training data according to the professional data set, so that the training data has better performance in medical question answering. In the embodiment of the present invention, the professional data set used for supervised fine-tuning includes: shibing624 medical field data, huatuoGPT supervised fine-tuning data, part of the DISC-MED data set and the ChatMed Dataset data set. The loss function of supervised fine-tuning is autoregressive loss. The goal of supervised fine-tuning is to minimize the difference between the predicted reference answer and the predicted answer, so that the predicted answer is as consistent as possible with the diagnosis proposed by the professional doctor.

[0068] Text deduplication is used to remove identical texts to prevent repeated texts from acting as noise and affecting the pre-training effect;

[0069] The word segmentation process is used to segment the text, obtain word tokens, and obtain a word token dictionary. In the embodiment of the present invention, the question text is segmented, the whole question text is separated into single word tokens, and an end symbol token is added after each question text segmentation. <eos>(End of Sentence) is separated from the next question text, and the word tokens after separation are statistically generated into a word token dictionary to facilitate word embedding operations in the medical large language question and answer model.

[0070] In one embodiment of the present invention, the training of the medical large language question-answering model requires a large amount of processed training data. In order to ensure that the patient's privacy information is effectively protected, the medical data needs to be desensitized before training, but common desensitization methods may result in incomplete desensitization, increasing the risk of privacy leakage. The present invention maximizes the level of privacy protection by establishing a data desensitization model; the data desensitization model is established as follows:

[0071] A1: Obtain medical field corpus;

[0072] A2: Extract privacy entities from each data entry in the medical field corpus through manual specification and rule-based recognition; rule-based recognition is set by technicians according to usage requirements, such as gender, age, disease type, etc.

[0073] A3: Input data entries and privacy entities into the LLM large language model;

[0074] A4: Prompt and guide the LLM large language model to perform anonymization processing;

[0075] A5: Output the replacement text of the privacy entity through the LLM large language model;

[0076] A6: Train the LLM large language model using data entries, privacy entities, and replacement texts, and evaluate the privacy protection performance;

[0077] A7: After evaluating the privacy protection performance, the data desensitization model is obtained.

[0078] In one embodiment of the present invention, the extracted data entry c is: "Zhang San, 45 years old, lives in Xuhui District, Shanghai. He went to Shanghai First People's Hospital for treatment on September 1, 2023. He was sent to the intensive care unit due to persistent high fever and shortness of breath. The doctor diagnosed pneumonia, and the condition was relatively serious, requiring immediate intubation and use of a ventilator."

[0079] Based on rule P, the privacy entity P(c) is identified as: ['Mr. Wang', '45 years old', 'Xuhui District, Shanghai', 'September 1, 2023', 'Shanghai First People's Hospital'];

[0080] After anonymization through the LLM large language model, the replacement text s is: "Li Si, 50 years old, lives in Chaoyang District, Beijing. He went to Beijing Union Hospital for treatment on October 5, 2025. He was sent to the intensive care unit due to persistent high fever and shortness of breath. The doctor diagnosed pneumonia, and the condition was relatively serious. He needed to be intubated and ventilated immediately."

[0081] The data entry c, privacy entity P(c) and replacement text s are trained on a small LLM large language model to obtain a data desensitization model and evaluate the privacy protection performance. In order to evaluate whether the desensitized medical text still contains identifiable patient private information, such as name, ID number, medical record, etc., the privacy protection performance of the data desensitization model is evaluated. The desensitized text is inferred through a large language model to generate inferred privacy information. The formula for privacy protection performance is as follows:

[0082] f t ,p t =f p (x t )

[0083] Where: p t The privacy score is used to quantify the privacy risk that still exists in the replaced text; t The text feedback generated for the model can point out clues in the anonymized text that may lead to privacy leaks.

[0084] The performance of privacy protection is evaluated through the following steps:

[0085] The specific steps for evaluating privacy protection performance are as follows:

[0086] Concatenate the replacement text with the privacy inference instruction and input it into the LLM large language model to output the inference result of the private information;

[0087] Compare the inference results with the real private information and calculate the privacy score based on the comparison results;

[0088] Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

[0089] In one embodiment of the present invention, the replacement text is concatenated with the privacy inference instruction, and the inference result about the private information is generated through the LLM large language model, wherein the privacy inference instruction is a predefined template question that can be used to evaluate the privacy protection of the replacement text, such as the pre-set privacy inference instruction: "infer the user's name or identity information from this text". These inference results are compared with the real private information y. If the content that matches y is found in the inference result, the privacy score is calculated according to the ranking:

[0090] p t =rank(y,{y′ 1 ,y′ 2 ,…,y′ K })

[0091] Among them, p t is the privacy score, rank is the score, y is the real private information, {y′ 1 ,y′ 2 ,…,y′ K } is the set of inference results, y′ 1 , y′ 2 and y′ K They are all single inference results output by the LLM large language model. Rank refers to the real private information y in the inference result {y′ 1 ,y′ 2 ,…,y′ K }, which determines the privacy score p t The closer the position is, the greater the privacy risk is. t If no match is found, then p t =K+1, indicating the lowest privacy risk, failure to re-identify private information, where K is an adjustable parameter that technicians can set according to specific application scenarios or requirements. If K is small, the privacy evaluator will only infer a small amount of possible private information, with a limited search range, which may miss some privacy risks; if K is large, the privacy evaluator will explore the anonymized text more deeply, infer more possible private information, and be able to more comprehensively assess privacy risks.

[0092] In one embodiment of the present invention, the LLM large language model also generates feedback f t , detailed explanation of the reasons leading to the identification of private information, guiding the further improvement of the subsequent anonymization process, ensuring that the medical question-answering algorithm effectively protects privacy information during the training process and prevents the patient's sensitive information from being re-identified; the LLM large language model combines the feedback f according to the privacy-optimized meta-instructions t , adjust the replacement text, and the privacy-optimized meta-instructions will guide the LLM large language model on how to further process sensitive information, clearly indicating which information needs to be processed and how to process it, such as completely removing it, further blurring it, or replacing it with irrelevant information to eliminate privacy risks; the LLM large language model will use the privacy-optimized meta-instructions combined with the feedback f t , and continuously perform fine-tuning iterations until the privacy score reaches the set maximum threshold K+1.

[0093] The present invention provides a medical large language question answering method based on data desensitization, and provides a medical large language question answering model, including an analysis module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module and a data desensitization module connected in sequence;

[0094] Among them, the word segmentation module is used to segment the input question and obtain word tokens;

[0095] The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary and map them to a high-dimensional semantic feature space through a word embedding matrix to obtain a question feature matrix;

[0096] A rotation position encoding module is used to add the rotation position encoding matrix to the problem feature matrix to obtain a feature matrix that integrates the position information;

[0097] The enhanced Transformer sub-model is used to process the feature matrix of the fused position information and output the feature information;

[0098] The word prediction module is used to map feature information and infer and output question and answer results.

[0099] In one embodiment of the present invention, the question text "Why do I get cramps in my buttocks when I am pregnant" is input and processed by the word segmentation module to obtain the following: ["pregnancy", "time", "buttocks", "cramp", "muscle", "is", "why"]; then the word embedding module is used to convert the word tokens into serial numbers according to a predefined dictionary, and then each token serial number is mapped to a high-dimensional semantic feature space through the word embedding matrix, such as Figure 2 As shown in the figure, it is a schematic diagram of word embedding. After subsequent learning, the cosine distance of word vectors with similar semantics in the feature space is small. Through the word segmentation module and the word embedding module, a question text can be converted into a question feature matrix F k ∈R k×d , where k is the length of the question text and d is the dimension of the semantic feature space of word embedding.

[0100] In one embodiment of the present invention, the problem feature matrix is ​​processed using a rotation position encoding module; the problem feature matrix F k Add the rotation position encoding matrix p to obtain the feature matrix F of the fused position information P =F k +P, at the same time, the rotation position encoding module can represent the position information of different word tokens in the input question text, display the absolute position information of each word token in the input text, and perceive the relative position relationship between word tokens.

[0101] In one embodiment of the present invention, an enhanced Transformer module is used to process the feature matrix of fused position information; for complex application scenarios such as medical semantic analysis, the present invention improves the performance of the traditional Transformer model in processing long-distance dependencies and semantic understanding, optimizes the stability of the training process, enhances the Transformer model's ability to capture multi-layer semantic space, and proposes an enhanced Transformer sub-model (Extended Transformer, E-Transformer):

[0102] The enhanced Transformer sub-model includes: RMS regularization sub-module, self-attention sub-module and feedforward perception network sub-module connected in sequence;

[0103] The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting. It can increase the stability of the training process without changing the input feature dimension. Its expression is as follows:

[0104]

[0105] in, is the feature after RMS regularization processing, is the input feature, RMSNorm is RMS regularization;

[0106] The self-attention submodule is used to mine the association between different word tokens in different sub-semantic spaces and generate attention features. The self-attention submodule first converts the input features into three feature matrices of index, key, and value through a linear transformation layer, and then calculates the attention features through the self-attention module. The expression is as follows:

[0107]

[0108] in, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization, Concat is the concatenation and fusion of matrices, Linear is the linear transformation layer; the residual term is the connection term It can prevent the degradation of the medical large language question answering model and improve the training effect of the model;

[0109] The feedforward perception network submodule is used to fuse the attention features and output the fused features. It can process and transform the long dependency information encoded by the attention mechanism. Its expression is as follows:

[0110]

[0111] in, is the fusion feature output by the l-th layer enhanced Transformer sub-model, and are all learnable parameters of the linear mapping layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, is the input feature, RMS Norm is RMS regularization, Concat is the concatenation and fusion of feature matrices, and Softmax is the activation function. Through the multi-layer enhanced Transformer sub-model, the long-distance dependencies between texts can be learned to prevent model degradation caused by multi-step training. In the above parameters, the l in the upper right corner represents the l-th layer of enhanced Transformer sub-model, such as: is the output of the l-th layer enhanced Transformer sub-model, and the output after N layers of E-Transformer is

[0112] The present invention proposes an enhanced Transformer sub-model (Extended Transformer, E-Transformer). Compared with the ordinary Transformer model, E-Transformer introduces an RMS regularization layer to ensure the stability of input features, and then performs parallel calculations of multi-head attention layers. After the attention layer, the feedforward perception network sub-module uses the RMS regularization layer again, which can further stabilize the feature distribution of the attention layer output, ensuring that the next module, the feedforward perception network sub-module, can efficiently process the semantic information generated by the previous order. E-Transformer optimizes the calculation process of the Transformer model, so that the Transformer model will not lose or interfere with information due to feature mismatch between layers while capturing long-distance dependencies, and ultimately improves the generalization ability and training effect of the model.

[0113] In one embodiment of the present invention, the word prediction layer can convert the output feature information of the multi-layer E-Transformer encoding into Mapping is performed to infer the next word token of the current iteration, and its expression is as follows:

[0114]

[0115] Among them, P w is the predicted probability distribution of the next word token, is the learnable parameter matrix of the linear mapping layer, is the feature vector output after the p-th linear transformation layer E-Transformer, is the bias term associated with the p-th linear transformation layer E-Transformer.

[0116] The beneficial effects of the present invention are as follows: the present invention adopts an enhanced Transformer module to enhance the reasoning ability of long-distance contexts, ensuring that the medical question-answering large language model can output suggestions that are more in line with clinical needs. The present invention combines pre-training and supervised fine-tuning to process training data, thereby improving the generalization ability of the medical question-answering large language model for medical expertise and generating more accurate and professional suggestions. The present invention combines deep learning with traditional data to establish a data desensitization model, thereby realizing the identification and conversion of sensitive information in the data and improving the accuracy and efficiency of privacy protection.< / eos>

Claims

1. A medical big language question answering method based on data desensitization, characterized in that: The following steps are involved: S1: Obtain training data from the medical field corpus; S2: Preprocess the training data and perform data desensitization through a data desensitization model to obtain processed training data; S3: Establishing a medical large language question answering model based on the improved enhanced Transformer sub-model; S4: inputting the processed training data into the medical big language question answering model for training, thereby obtaining a trained medical big language question answering model; S5: Use the trained medical big language question-answering model to conduct medical question-answering.

2. The medical big language question answering method based on data desensitization according to claim 1 is characterized in that: The preprocessing includes supervised fine-tuning, text deduplication and word segmentation; The supervised fine-tuning is used to fine-tune the parameters of the training data according to the professional dataset; The text deduplication is used to remove identical texts; The word segmentation process is used to segment the text to obtain word tokens and obtain a word token dictionary by statistics.

3. The medical big language question answering method based on data desensitization according to claim 1 is characterized in that: The data desensitization model is established as follows: A1: Obtain medical field corpus; A2: Extract privacy entities from each data entry in the medical domain corpus through manual specification and rule-based recognition; A3: Input data entries and privacy entities into the LLM large language model; A4: Prompt and guide the LLM large language model to perform anonymization processing; A5: Output the replacement text of the privacy entity through the LLM large language model; A6: Train the LLM large language model using data entries, privacy entities, and replacement texts, and evaluate the privacy protection performance; A7: After evaluating the privacy protection performance, the data desensitization model is obtained.

4. The medical big language question answering method based on data desensitization according to claim 3 is characterized in that: The specific steps of evaluating the privacy protection performance are as follows: Concatenate the replacement text with the privacy inference instruction and input it into the LLM large language model to output the inference result of the private information; Compare the inference results with the real private information and calculate the privacy score based on the comparison results; Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

5. The medical big language question answering method based on data desensitization according to claim 4 is characterized in that: The privacy score is expressed as follows: p t =rank(y,{y′1,y′2,…,y′ K }) Among them, p t is the privacy score, rank is the score, y is the real private information, {y′1,y′2,…,y′ K } is the geometry of the inferred result, y′1, y′2 and y′ K These are all single inference results output by the LLM large language model.

6. The medical big language question answering method based on data desensitization according to claim 1 is characterized in that: The medical large language question answering model includes an analysis module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module and a data desensitization module connected in sequence; The word segmentation module is used to segment the input question to obtain word tokens; The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary, and map them to a high-dimensional semantic feature space through a word embedding matrix to obtain a question feature matrix; The rotation position encoding module is used to add the rotation position encoding matrix to the problem feature matrix to obtain a feature matrix that integrates the position information; The enhanced Transformer sub-model is used to process the feature matrix of fused position information and output fused features; The word prediction module is used to map the fused features and infer and output the question-answering results.

7. The medical big language question answering method based on data desensitization according to claim 6 is characterized in that: The enhanced Transformer sub-model includes: an RMS regularization sub-module, a self-attention sub-module and a feedforward perception network sub-module connected in sequence; The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting; The self-attention submodule is used to mine the association relationship between different word tokens in different sub-semantic spaces and generate attention features; The feedforward perception network submodule is used to fuse the attention features and output the fused features.

8. The medical big language question answering method based on data desensitization according to claim 6 is characterized in that: The mathematical expression of the enhanced Transformer sub-model is as follows: in, is the fusion feature output by the l-th layer enhanced Transformer sub-model, and are all learnable parameters of the linear mapping layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, is the input feature, RMSNorm is RMS regularization, Concat is the concatenation and fusion of feature matrices, and Softmax is the activation function.

Images