A medical big language question answering method based on desensitization of diagnosis and treatment data

Abstract

The present invention discloses a medical big language question-answering method based on data desensitization, which belongs to the field of data processing technology and includes: obtaining training data from a medical field corpus and preprocessing it, and obtaining processed training data through a data desensitization model; establishing a medical big language question-answering model based on an improved enhanced Transformer sub-model; inputting the processed training data into the medical big language question-answering model for training to obtain a trained medical big language question-answering model; and performing medical question-answering using the trained medical big language question-answering model. The present invention adopts an enhanced Transformer module to enhance the reasoning ability of long-distance contexts. Through preprocessing, the model's generalization ability of medical expertise is improved, and it can generate more accurate and professional suggestions; through the data desensitization model, sensitive information in medical data is identified and converted, thereby improving the accuracy and efficiency of privacy protection.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a medical big language question-answering method based on desensitization of diagnosis and treatment data. Background Art

[0002] Medical question-answering is a key application of big models in the healthcare industry. Leveraging the powerful capabilities of big model technology in language understanding and reasoning, and learning from medical expertise, it provides precise treatment queries, medical consultations, and medication guidance, meeting patients' personalized medical needs and improving the efficiency and quality of medical services. While existing medical question-answering systems play an important role in smart healthcare, these technologies still face significant challenges when processing medical data.

[0003] Medical question-answering systems based on knowledge-based keyword matching provide answers by matching predefined content. This approach is inefficient for addressing complex medical questions and difficult to scale to large-scale data. Furthermore, keyword matching technology fails to effectively identify and protect sensitive information in medical data, increasing the risk of privacy breaches.

[0004] Based on the long short-term memory network (LSTM) and gated recurrent unit (GRU), semantic information in the question is extracted, and the learned medical memory knowledge and attention algorithm are integrated to generate relevant medical advice. However, due to the poor learning effect of long-distance dependencies, the algorithm has limited understanding and expression capabilities of natural language. Although the effect is better than keyword matching, it still cannot meet the requirements of actual applications.

[0005] Medical question-answering based on large language models using Transformers and BERT has improved the accuracy and intelligence of the answers generated by the model. A BERT question-answering model based on the self-attention mechanism was designed using the output of a pre-trained BERT model as a semantic embedding. However, this model has not been explored on large-scale data, failing to fully tap the model's potential. Furthermore, although existing algorithms perform manual or automatic desensitization on raw data during data processing, they fail to address whether medical text still contains identifiable patient private information. Summary of the Invention

[0006] In response to the above-mentioned deficiencies in the prior art, the present invention provides a medical question-answering language system based on desensitized diagnosis and treatment data, which solves the problems existing in the prior art:

[0007] (1) Some large model technologies have limited ability to understand and infer long-distance contexts, especially when complex background knowledge is involved in medical question answering, where the model's ability to understand and infer contextual semantics is limited;

[0008] (2) The existing medical big language question answering model has insufficient generalization ability for medical professional knowledge, and the generated diagnosis and treatment recommendations lack accuracy and professionalism;

[0009] (3) Although existing algorithms manually or automatically desensitize the original data during data processing, they do not pay attention to whether the medical text still contains identifiable patient private information.

[0010] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a medical language question-answering method based on data desensitization, comprising the following steps:

[0011] S1: Obtain training data from the medical field corpus;

[0012] S2: Preprocess the training data and perform data desensitization using a data desensitization model to obtain processed training data;

[0013] S3: Building a medical large language question answering model based on the improved enhanced Transformer sub-model;

[0014] S4: Inputting the processed training data into the medical big language question answering model for training to obtain a trained medical big language question answering model;

[0015] S5: Use the trained medical big language question answering model to conduct medical question answering.

[0016] The beneficial effects of the present invention are as follows: the present invention proposes a large language model medical question-answering algorithm based on desensitized diagnosis and treatment data, uses an enhanced Transformer module to mine the contextual dependencies of text data, improves the model's generalization ability of language knowledge and medical field expertise through pre-training and supervised fine-tuning, performs privacy protection optimization during training, and maximizes the level of privacy protection.

[0017] Furthermore: the preprocessing includes supervised fine-tuning, text deduplication and word segmentation processing;

[0018] The supervised fine-tuning is used to fine-tune the parameters of the training data based on the professional dataset;

[0019] The text deduplication is used to remove identical texts;

[0020] The word segmentation process is used to segment the text to obtain word tokens, and to obtain a word token dictionary by statistics.

[0021] The beneficial effects of the above further scheme are: through supervised fine-tuning, text deduplication and word segmentation processing, the quality of training data can be improved and the training effect of the model can be improved.

[0022] Furthermore: the data desensitization model is established as follows:

[0023] A1: Obtain medical corpus;

[0024] A2: Extracting privacy entities from each data entry in the medical domain corpus through manual specification and rule-based recognition;

[0025] A3: Input data entries and privacy entities into the LLM large language model;

[0026] A4: Prompt and guide the LLM large language model to perform anonymization processing;

[0027] A5: Output the replacement text of the private entity through the LLM large language model;

[0028] A6: Train the LLM large language model using data entries, privacy entities, and replacement text, and evaluate its privacy protection performance;

[0029] A7: After completing the privacy protection performance evaluation, the data desensitization model is obtained.

[0030] The beneficial effect of the above further solution is: by establishing a data desensitization model, it is possible to identify and replace sensitive data in medical data, eliminate privacy risks, and enhance the ability to protect patients' private information.

[0031] Furthermore, the specific steps of evaluating the privacy protection performance are as follows:

[0032] The replacement text is concatenated with the privacy inference instruction and input into the LLM large language model to output the inference result of the private information.

[0033] Compare the inferred results with the actual private information and calculate the privacy score based on the comparison results;

[0034] Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

[0035] The beneficial effects of the above further scheme are: by replacing text and privacy reasoning instructions, evaluating privacy protection performance, and setting the data desensitization capability of the privacy score judgment model, it can ensure that the model's inference ability for private text meets the requirements and maximize the level of privacy protection.

[0036] Furthermore: the privacy score is expressed as follows:

[0037] p t =rank(y,{y′1,y′2,…,y′ K})

[0038] Among them, p tis the privacy score, rank is the score, y is the real private information, {y′1,y′2,…,y′ K} is the geometry of the inferred result, y′1, y′2 and y′ K These are all single inference results output by the LLM large language model.

[0039] The beneficial effect of the above further solution is: the desensitization ability of the model is quantified through the privacy score, which makes it easier for technical personnel to judge the desensitization ability of the model and provide guidance to meet usage needs.

[0040] Furthermore: the medical large language question answering model includes an analysis module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module and a data desensitization module connected in sequence;

[0041] The word segmentation module is used to segment the input question to obtain word tokens;

[0042] The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary, and map them to a high-dimensional semantic feature space through a word embedding matrix to obtain a question feature matrix;

[0043] The rotation position encoding module is used to add the rotation position encoding matrix and the problem feature matrix to obtain a feature matrix of fused position information;

[0044] The enhanced Transformer sub-model is used to process the feature matrix of the fused position information and output feature information;

[0045] The word prediction module is used to map feature information and infer and output question-answering results.

[0046] The beneficial effect of the above further solution is: establishing a medical large language question-answering model based on several layers of enhanced Transformer sub-models can enhance the question-answering model's reasoning ability for long-distance contexts and ensure that diagnosis and treatment recommendations are more in line with clinical needs.

[0047] Furthermore: the enhanced Transformer sub-model includes: an RMS regularization sub-module, a self-attention sub-module and a feedforward perception network sub-module connected in sequence;

[0048] The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting;

[0049] The self-attention submodule is used to mine the association relationship between different word tokens in different sub-semantic spaces and generate attention features;

[0050] The feedforward perception network submodule is used to fuse the attention features and output the fused features.

[0051] The beneficial effects of the above further scheme are: the stability of the input features is ensured through the RMS regularization sub-module, and the feature distribution of the attention layer output can be stabilized through the self-attention sub-module, ensuring that the feedforward perception network sub-module can efficiently process feature information, so that the enhanced Transformer sub-model can capture long-distance dependencies while avoiding information loss or interference due to feature mismatch between layers, ultimately improving the generalization ability and training effect of the model.

[0052] Furthermore: the expression of the enhanced Transformer sub-model is as follows:

[0053]

[0054] in, is the fusion feature output by the l-th layer enhanced Transformer sub-model, and are all learnable parameters of the linear mapping layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, is the input feature, RMSNorm is the RMS regularization, Concat is the concatenation and fusion of the feature matrix, and Softmax is the activation function.

[0055] The beneficial effects of the above further scheme are: through the mathematical expression of the enhanced Transformer sub-model, the structure of each module can be specifically represented. The enhanced Transformer sub-model introduces the RMS regularization layer at the beginning of the calculation to ensure the stability of the input features. The feedforward perception network sub-module uses the RMS regularization layer again, which can further stabilize the feature distribution of the attention layer output and optimize the calculation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a medical big language question answering method based on data desensitization;

[0057] Figure 2 Schematic diagram of word embedding. DETAILED DESCRIPTION

[0058] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0059] like Figure 1 As shown, a medical large language question answering method based on data desensitization includes the following steps:

[0060] S1: Obtain training data from the medical field corpus;

[0061] S2: Preprocess the training data and perform data desensitization using a data desensitization model to obtain processed training data;

[0062] S3: Building a medical large language question answering model based on the improved enhanced Transformer sub-model;

[0063] S4: Inputting the processed training data into the medical big language question answering model for training to obtain a trained medical big language question answering model;

[0064] S5: Use the trained medical big language question answering model to conduct medical question answering.

[0065] In one embodiment of the present invention, S1 obtains training data from the medical field corpus, including: Wiki Chinese Encyclopedia, Baidu Encyclopedia, C4 dataset, Chinese Wudao open source dataset, shibing624 medical field data, and actual medical dataset. After preprocessing and ChatGLM-6B word segmentation processing, the scale of the training data is approximately 63.4 billion word tokens.

[0066] In one embodiment of the present invention, preprocessing includes supervised fine-tuning, text deduplication, and word segmentation;

[0067] Supervised fine-tuning is used to fine-tune the parameters of training data based on professional datasets to improve the performance of the training data in medical question answering. In an embodiment of the present invention, the professional datasets used for supervised fine-tuning include: shibing624 medical field data, huatuoGPT supervised fine-tuning data, partial data from the DISC-MED dataset, and the ChatMed dataset. The loss function of supervised fine-tuning is autoregressive loss. The goal of supervised fine-tuning is to minimize the difference between the predicted reference answer and the predicted answer, so that the predicted answer is as consistent as possible with the diagnosis proposed by the professional doctor.

[0068] Text deduplication is used to remove identical texts to prevent duplicate texts from acting as noise and affecting the pre-training effect;

[0069] Word segmentation is used to segment the text, obtain word tokens, and obtain a word token dictionary. In the embodiment of the present invention, the question text is segmented, the entire question text is separated into single word tokens, and an end symbol token is added after each question text segmentation. <eos>(End of Sentence), separated from the next question text, and the word tokens after separation are statistically analyzed to generate a word token dictionary, which facilitates word embedding operations in the medical large language question answering model.

[0070] In one embodiment of the present invention, the training of a medical large language question-answering model requires a large amount of processed training data. To ensure that patient privacy information is effectively protected, medical data needs to be desensitized before training. However, common desensitization methods may result in incomplete desensitization, increasing the risk of privacy leakage. The present invention maximizes the level of privacy protection by establishing a data desensitization model; the data desensitization model is established as follows:

[0071] A1: Obtain medical corpus;

[0072] A2: Extract private entities from each data entry in the medical corpus through manual specification and rule-based recognition. Rule-based recognition is set by technicians based on usage requirements, such as gender, age, disease type, etc.

[0073] A3: Input data entries and privacy entities into the LLM large language model;

[0074] A4: Prompt and guide the LLM large language model to perform anonymization processing;

[0075] A5: Output the replacement text of the private entity through the LLM large language model;

[0076] A6: Train the LLM large language model using data entries, privacy entities, and replacement text, and evaluate its privacy protection performance;

[0077] A7: After completing the privacy protection performance evaluation, the data desensitization model is obtained.

[0078] In one embodiment of the present invention, the extracted data entry c is: "Zhang San, 45 years old, living in Xuhui District, Shanghai, went to Shanghai First People's Hospital for treatment on September 1, 2023. He was admitted to the intensive care unit due to persistent high fever and shortness of breath. The doctor diagnosed him with pneumonia, and his condition was relatively serious, requiring immediate intubation and the use of a ventilator."

[0079] Based on rule P, the privacy entity P(c) is identified as: ['Mr. Wang', '45 years old', 'Xuhui District, Shanghai', 'September 1, 2023', 'Shanghai First People's Hospital'];

[0080] After anonymization through the LLM large language model, the replacement text s is obtained: "Li Si, 50 years old, lives in Chaoyang District, Beijing. On October 5, 2025, he went to Beijing Union Hospital for treatment and was admitted to the intensive care unit due to persistent high fever and shortness of breath. The doctor diagnosed him with pneumonia, and his condition was relatively serious, requiring immediate intubation and use of a ventilator."

[0081] The data entry c, the privacy entity P(c), and the replacement text s are trained on a small LLM large language model to obtain a data desensitization model and evaluate its privacy protection performance. To assess whether the desensitized medical text still contains identifiable patient private information, such as name, ID number, medical records, etc., the privacy protection performance of the data desensitization model is evaluated. The desensitized text is inferred through a large language model to generate inferred private information. The privacy protection performance formula is as follows:

[0082] f t ,p t =f p (x t )

[0083] Where: p t The privacy score is used to quantify the privacy risk that still exists in the replaced text; t The text feedback generated for the model can point out clues in the anonymized text that may lead to privacy leaks.

[0084] The performance of privacy protection is evaluated through the following steps:

[0085] The specific steps for evaluating privacy protection performance are as follows:

[0086] The replacement text is concatenated with the privacy inference instruction and input into the LLM large language model to output the inference result of the private information.

[0087] Compare the inferred results with the actual private information and calculate the privacy score based on the comparison results;

[0088] Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

[0089] In one embodiment of the present invention, the replacement text is concatenated with privacy inference instructions, and an LLM large language model is used to generate inference results about private information. The privacy inference instructions are predefined template questions that can be used to evaluate the privacy protection of the replacement text. For example, a pre-set privacy inference instruction is: "Infer the user's name or identity information from this text." These inference results are compared with the actual private information y. If content matching y is found in the inference results, a privacy score is calculated based on the ranking:

[0090] p t =rank(y,{y′1,y′2,…,y′ K })

[0091] Among them, p t is the privacy score, rank is the score, y is the real private information, {y′1,y′2,…,y′ K } is the set of inference results, y′1, y′2 and y′ K They are all single inference results output by the LLM large language model. Rank refers to the real private information y in the inference results {y′1,y′2,…,y′ K }, which determines the privacy score p t The closer the position is to the front, the greater the privacy risk is. t If no match is found, then p t =K+1, indicating the lowest privacy risk, and failure to re-identify private information. K is an adjustable parameter that technicians can set according to specific application scenarios or needs. If K is small, the privacy evaluator will only infer a small amount of possible private information, with a limited search range, which may miss some privacy risks. If K is large, the privacy evaluator will explore the anonymized text more deeply, infer more possible private information, and be able to more comprehensively assess privacy risks.

[0092] In one embodiment of the present invention, the LLM large language model also generates feedback f t , detailed explanation of the reasons leading to the identification of private information, guiding the further improvement of the subsequent anonymization process, ensuring that the medical question-answering algorithm effectively protects privacy information during the training process and prevents the patient's sensitive information from being re-identified; the LLM large language model is based on the privacy-optimized meta-instructions, combined with feedback f t , adjust the replacement text, and the privacy-optimized meta-instructions will guide the LLM large language model on how to further process sensitive information, clearly indicating which information needs to be processed and how to process it, such as completely removing it, further blurring it, or replacing it with irrelevant information to eliminate privacy risks; the LLM large language model will use the privacy-optimized meta-instructions combined with feedback f t , and continuously fine-tune and iterate until the privacy score reaches the set maximum threshold K+1.

[0093] The present invention provides a medical large language question answering method based on data desensitization, and provides a medical large language question answering model, including an analysis module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module, and a data desensitization module connected in sequence;

[0094] Among them, the word segmentation module is used to segment the input question and obtain word tokens;

[0095] The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary and map them to a high-dimensional semantic feature space through the word embedding matrix to obtain the question feature matrix;

[0096] A rotation position encoding module is used to add the rotation position encoding matrix to the problem feature matrix to obtain a feature matrix that integrates position information;

[0097] The enhanced Transformer sub-model is used to process the feature matrix of the fused position information and output feature information;

[0098] The word prediction module is used to map feature information and infer and output question-answering results.

[0099] In one embodiment of the present invention, the question text "Why do I get cramps in my buttocks when I'm pregnant" is input and processed by the word segmentation module to obtain the following: ["pregnancy", "time", "buttocks", "cramps", "cramps", "is", "why"]; then the word embedding module is used to convert the word tokens into serial numbers according to a predefined dictionary, and then each token serial number is mapped to a high-dimensional semantic feature space through the word embedding matrix, such as Figure 2 As shown in the figure, it is a word embedding diagram. After subsequent learning, the cosine distance of word vectors with similar semantics in the feature space is small. Through the word segmentation module and word embedding module, a question text can be converted into a question feature matrix F k ∈R k×d , where k is the length of the question text and d is the dimension of the semantic feature space of word embedding.

[0100] In one embodiment of the present invention, the problem feature matrix is ​​processed using a rotation position encoding module; the problem feature matrix F k Add the rotation position encoding matrix p and get the feature matrix F of the fused position information P =F k +P, at the same time, the rotation position encoding module can represent the position information of different word tokens in the input question text, display the absolute position information of each word token in the input text, and perceive the relative position relationship between word tokens.

[0101] In one embodiment of the present invention, an enhanced Transformer module is used to process the feature matrix fused with position information. For complex application scenarios such as medical semantic analysis, the present invention improves the performance of the traditional Transformer model in handling long-distance dependencies and semantic understanding, optimizes the stability of the training process, and enhances the Transformer model's ability to capture multi-layer semantic spaces. An enhanced Transformer sub-model (Extended Transformer, E-Transformer) is proposed:

[0102] The enhanced Transformer sub-model includes: an RMS regularization sub-module, a self-attention sub-module, and a feedforward perception network sub-module connected in sequence;

[0103] The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting. It can increase the stability of the training process without changing the input feature dimension. Its expression is as follows:

[0104]

[0105] in, is the feature after RMS regularization processing, is the input feature, RMSNorm is RMS regularization;

[0106] The self-attention submodule is used to mine the association between different word tokens in different sub-semantic spaces and generate attention features. The self-attention submodule first converts the input features into three feature matrices of index, key, and value through a linear transformation layer, and then calculates the attention features through the self-attention module. Its expression is as follows:

[0107]

[0108] in, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, Concat is the concatenation and fusion of matrices, Linear is the linear transformation layer; the residual term is the connection term It can prevent the degradation of medical large-scale language question-answering models and improve the training effect of the models;

[0109] The feedforward perception network submodule is used to fuse the attention features and output the fused features. It can process and transform the long dependency information encoded by the attention mechanism. Its expression is as follows:

[0110]

[0111] in, is the fusion feature output by the l-th layer enhanced Transformer sub-model, and are all learnable parameters of the linear mapping layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the residual connection term, is the feature after RMS regularization processing, is the input feature, RMSNorm is RMS regularization, Concat is the concatenation and fusion of feature matrices, and Softmax is the activation function. Through the multi-layer enhanced Transformer sub-model, it is possible to learn long-range dependencies between texts and prevent model degradation caused by multi-step training. In the above parameters, the l in the upper right corner represents the l-th layer of enhanced Transformer sub-model, such as: is the output of the l-th layer enhanced Transformer sub-model, and the output after N layers of E-Transformer is

[0112] The present invention proposes an enhanced Transformer sub-model (Extended Transformer, E-Transformer). Compared with the ordinary Transformer model, E-Transformer introduces an RMS regularization layer to ensure the stability of input features, and then performs parallel calculations of multiple attention layers. After the attention layer, the feedforward perception network sub-module uses the RMS regularization layer again to further stabilize the feature distribution of the attention layer output, ensuring that the next module, the feedforward perception network sub-module, can efficiently process the semantic information generated by the previous order. E-Transformer optimizes the computational process of the Transformer model, allowing the Transformer model to capture long-range dependencies while avoiding information loss or interference due to feature mismatch between layers, ultimately improving the model's generalization ability and training effect.

[0113] In one embodiment of the present invention, the word prediction layer can convert the output feature information of the multi-layer E-Transformer encoding into Mapping is performed to infer the next word token of the current iteration, which is expressed as follows:

[0114]

[0115] Among them, P w is the predicted probability distribution of the next word token, is the learnable parameter matrix of the linear mapping layer, is the feature vector output after the p-th linear transformation layer E-Transformer, is the bias term associated with the p-th linear transformation layer E-Transformer.

[0116] The beneficial effects of the present invention are as follows: the present invention adopts an enhanced Transformer module to strengthen the reasoning ability of long-distance context, ensuring that the medical question-answering language model can output suggestions that are more in line with clinical needs. The present invention combines pre-training and supervised fine-tuning to process training data, thereby improving the generalization ability of the medical question-answering language model for medical professional knowledge and generating more accurate and professional suggestions. The present invention combines deep learning with traditional data to establish a data desensitization model, realizing the identification and conversion of sensitive information in the data, and improving the accuracy and efficiency of privacy protection.< / eos>

Claims

1. A medical large language question answering method based on data desensitization, characterized in that: The following steps are involved: S1: Obtain training data from the medical field corpus; S2: Preprocess the training data and perform data desensitization using a data desensitization model to obtain processed training data; The data desensitization model is established as follows: A1: Obtain medical corpus; A2: Extracting privacy entities from each data entry in the medical domain corpus through manual specification and rule-based recognition; A3: Input data entries and privacy entities into the LLM large language model; A4: Prompt and guide the LLM large language model to perform anonymization processing; A5: Output the replacement text of the private entity through the LLM large language model; A6: Train the LLM large language model using data entries, privacy entities, and replacement text, and evaluate its privacy protection performance; A7: After evaluating the privacy protection performance, a data desensitization model is obtained. S3: Building a medical large language question answering model based on the improved enhanced Transformer sub-model; The medical large language question answering model includes a word segmentation module, a word embedding module, a rotation position encoding module, several layers of enhanced Transformer sub-models, a word prediction module and a data desensitization module connected in sequence; The word segmentation module is used to segment the input question to obtain word tokens; The word embedding module is used to convert word tokens into serial numbers according to a predefined dictionary, and map them to a high-dimensional semantic feature space through a word embedding matrix to obtain a question feature matrix; The rotation position encoding module is used to add the rotation position encoding matrix and the problem feature matrix to obtain a feature matrix of fused position information; The enhanced Transformer sub-model is used to process the feature matrix of the fused position information and output the fused features; The word prediction module is used to map the fused features and infer and output the question-answering results; The enhanced Transformer sub-model includes: an RMS regularization sub-module, a self-attention sub-module and a feedforward perception network sub-module connected in sequence; The RMS regularization submodule is used to process the feature matrix of the input fusion position information to prevent overfitting; The self-attention submodule is used to mine the association relationship between different word tokens in different sub-semantic spaces and generate attention features; The self-attention submodule first converts the input features into three feature matrices of index, key, and value through a linear transformation layer, and then calculates the attention features through the self-attention module; The feedforward perception network submodule is used to fuse the attention features and output the fused features; S4: Inputting the processed training data into the medical big language question answering model for training to obtain a trained medical big language question answering model; S5: Use the trained medical big language question answering model to conduct medical question answering.

2. The medical large language question answering method based on data desensitization according to claim 1 is characterized in that: The preprocessing includes supervised fine-tuning, text deduplication and word segmentation; The supervised fine-tuning is used to fine-tune the parameters of the training data based on the professional dataset; The text deduplication is used to remove identical texts; The word segmentation process is used to segment the text to obtain word tokens, and to obtain a word token dictionary by statistics.

3. The medical large language question answering method based on data desensitization according to claim 1 is characterized in that: The specific steps for evaluating privacy protection performance are as follows: The replacement text is concatenated with the privacy inference instruction and input into the LLM large language model to output the inference result of the private information. The privacy reasoning instruction is a predefined template question used to evaluate the privacy protection of the replacement text; Compare the inferred results with the actual private information and calculate the privacy score based on the comparison results; Determine whether the privacy score is less than the set threshold. If so, adjust the LLM large language model and return to A3. Otherwise, complete the evaluation of privacy protection performance.

4. The medical large language question answering method based on data desensitization according to claim 3 is characterized in that: The privacy score is expressed as follows: ； in, Score for privacy, To score points, For real private information, is the set of inference results, 、 as well as Each is a single inference result output by the LLM large language model; K represents an adjustable parameter that is set according to specific application scenarios or requirements.

5. The medical large language question answering method based on data desensitization according to claim 1 is characterized in that: The mathematical expression of the enhanced Transformer sub-model is as follows: ； ； ； ； in, For the The fused features output by the layer-enhanced Transformer sub-model, , , and are all learnable parameters of the linear transformation layer, is the attention feature, is the index feature matrix, is the key feature matrix, is the value feature matrix, is the feature after RMS regularization processing, is the linear transformation layer, is the input feature, is RMS regularization, In order to concatenate and fuse the feature matrices, is the activation function.

Images