A privacy protection method and system for medical consultation scenarios

Abstract

The present invention discloses a privacy protection method and system for medical consultation scenarios, relating to the field of artificial intelligence technology. The method includes the following steps: training multiple doctor models based on a large language model, including several remote doctor models and a chief physician model; recording the consultation text and adaptively compressing it to obtain a compressed consultation text; segmenting the compressed consultation text into several non-overlapping partial texts, which are transmitted separately to several remote doctor models to obtain several diagnosis results; and concatenating the several diagnosis results to form a joint vector, which is input into the chief physician model to obtain the final consultation result. By adaptively compressing the user's consultation text and segmenting the consultation data into multiple independent modules, the present invention ensures that the remote doctor model during the consultation process cannot obtain all data, thereby achieving privacy protection in medical consultation scenarios while also ensuring the accuracy of the medical consultation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and healthcare technology, and in particular to a privacy protection method and system for medical consultation scenarios. Background Art

[0002] With the widespread application of large language models in the field of natural language processing, online medical consultations have developed rapidly. The use of large language models can help patients quickly obtain professional medical advice or diagnosis. However, in actual applications, since patients often need to upload text containing private information such as name, contact information, and medical history, the risk of privacy leakage continues to rise. Most existing medical large language models focus mainly on the accuracy of consultations, and lack a complete mechanism for protecting the privacy of user information. Large models may memorize and leak personal privacy information in the training data during the training or inference stage. In addition, if the complete user information is transmitted to a remote server or cloud model at one time, there is also a communication risk. Therefore, a comprehensive method that takes into account both data privacy protection and medical consultation accuracy is needed. The present invention is based on information entropy adaptive compression, combined with a collaborative diagnosis mechanism of multiple doctors and chief physicians, which can significantly reduce the risk of patient privacy leakage while ensuring the accuracy of consultations. Summary of the Invention

[0003] The purpose of the present invention is to solve the problems in the prior art.

[0004] The first invention, the technical solution adopted by the present invention to solve its technical problem is: to provide a privacy protection method for medical consultation scenarios, including the following steps:

[0005] Based on the large language model training, several remote doctor models and one chief doctor model are obtained; the remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; the chief doctor model integrates several diagnosis results and outputs a final diagnosis result;

[0006] Record the medical consultation text and perform adaptive compression to obtain a compressed medical consultation text;

[0007] The compressed medical consultation text is divided into several non-overlapping partial texts, which are transmitted to several remote doctor models respectively to obtain several diagnosis results;

[0008] Several diagnostic results are spliced ​​together to form a joint vector and input into the chief physician model to obtain the final consultation result.

[0009] Preferably, the multiple doctor models are obtained by training based on the large language model, specifically:

[0010] The training set is constructed using historical medical consultation data and simulated sensitive information. The doctor model and chief physician model are pre-trained and LoRA fine-tuned. The LoRA fine-tuning is expressed as:

[0011] W′ i =W i +B i A i ;

[0012] Among them, W i is the weight matrix in the pre-trained large language model, B i With A i is a low-rank matrix satisfying |B i |,|A i |<<|W i |.

[0013] Preferably, recording the medical consultation text and adaptively compressing it to obtain the compressed medical consultation text comprises the following steps:

[0014] Divide the medical consultation text into sensitive information and non-sensitive information;

[0015] Adaptively compress sensitive information based on the information entropy of the medical inquiry text and the information entropy of the sensitive information to obtain compressed sensitive information;

[0016] The compressed sensitive information is merged with the non-sensitive information to form a compressed medical consultation text.

[0017] Preferably, the medical consultation text is divided into sensitive information and non-sensitive information, specifically:

[0018] Natural language processing technology and preset regular expression rules are used to identify and extract sensitive information, including names, contact information and addresses.

[0019] Preferably, the adaptively compressing the sensitive information based on the information entropy of the medical inquiry text and the information entropy of the sensitive information to obtain the compressed sensitive information comprises the following steps:

[0020] The information entropy of the medical inquiry text and the information entropy of the sensitive information are calculated respectively, and then the entropy ratio R is calculated, which is expressed as:

[0021]

[0022] Among them, H(Q) represents the information entropy of the medical consultation text, H(I priv ) represents the information entropy of sensitive information;

[0023] The compression ratio ρ is determined according to the entropy ratio R, which is expressed as:

[0024]

[0025] Sensitive information is grouped and dimensionally reduced according to the compression ratio ρ to form compressed sensitive information, which is expressed as:

[0026]

[0027] Among them, I priv represents sensitive information, k represents the group index after grouping, t i The vector representation of the i-th token in the sensitive information; Indicates compressed sensitive information.

[0028] Preferably, the information entropy of the medical inquiry text and the information entropy of the sensitive information are calculated separately, and the information entropy is calculated using a sliding window or sentence-level segmentation calculation method.

[0029] In a second aspect, the present invention provides a privacy protection system for medical consultation scenarios, comprising:

[0030] The training module is based on the large language model training to obtain several remote doctor models and one chief doctor model; the remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; the chief doctor model integrates several diagnosis results and outputs a final diagnosis result;

[0031] Compression module, records the medical consultation text and performs adaptive compression to obtain compressed medical consultation text;

[0032] The segmentation module divides the compressed medical consultation text into several non-overlapping parts, which are transmitted to several remote doctor models to obtain several diagnosis results.

[0033] The fusion module splices several diagnostic results into a joint vector and inputs it into the chief physician model to obtain the final consultation result.

[0034] In a third aspect, the present invention further provides a computer device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0035] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of any of the above-mentioned methods when executed by a processor.

[0036] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program / instruction, which implements the steps of any of the above-mentioned methods when executed by a processor.

[0037] The present invention has the following beneficial effects:

[0038] (1) The present invention realizes data adaptive compression processing by efficiently detecting and extracting sensitive information in user medical consultation texts and compressing it using information entropy;

[0039] (2) The present invention divides the compressed medical consultation data and transmits them to multiple remote doctor models for independent processing. Finally, the chief physician model integrates the diagnosis results of each doctor, taking into account both privacy protection and the accuracy of medical consultation.

[0040] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The difference between the embodiments of the present invention and traditional medical consultation;

[0042] Figure 2 A diagram showing the steps of a method according to an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of a flow chart of an embodiment of the present invention;

[0044] Figure 4 2 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] This invention aims to resolve the contradiction between protecting user privacy and ensuring the accuracy of consultation in existing telemedicine consultation systems, and proposes a privacy protection method based on information entropy adaptive compression and multi-doctor-chief physician collaboration mechanism. Figure 1 The difference between the embodiment of the present invention and traditional medical consultation is shown in FIG. Figure 1 As shown in (a) in FIG, including searching the Internet, communicating between patients, online doctor consultation and searching medical books, most of the methods have the risk of privacy leakage except for patients searching medical books by themselves; and the present invention as Figure 1 As shown in (b), by detecting and compressing sensitive information in the user's medical consultation text, and dividing the processed medical consultation data and transmitting it to multiple remote doctor models for independent processing, the chief physician model finally integrates the diagnosis results of each doctor, taking into account both privacy protection and the accuracy of medical consultation.

[0046] For details, see Figure 2 and Figure 3 FIG. 1 is a diagram showing steps and a flow chart of a method according to an embodiment of the present invention, which includes the following steps:

[0047] S201, based on the large language model training, obtains several remote doctor models and one chief physician model; the remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; the chief physician model integrates the several diagnosis results and outputs a final diagnosis result;

[0048] S202, recording the medical consultation text and performing adaptive compression to obtain a compressed medical consultation text;

[0049] S203, dividing the compressed medical inquiry text into a plurality of non-overlapping partial texts, and transmitting them to a plurality of remote doctor models respectively to obtain a plurality of diagnosis results;

[0050] S204: Concatenate several diagnosis results into a joint vector and input it into the chief physician model to obtain the final consultation result.

[0051] Specifically, in S201, a training set is constructed using a large amount of historical medical consultation data and simulated sensitive information. The doctor model is pre-trained and LoRA (low-rank adaptation) fine-tuned based on the training set. The fine-tuning formula is:

[0052] W′ i =W i +B i A i ;

[0053] Among them, W i To pre-train the weight matrix in the large language model, the embodiment of the present invention adopts the Llama3 large language model; B i With A i is a low-rank matrix satisfying |B i |,|A i |<<|W i |, thereby reducing the computational and communication burden when updating the model. |B i |,|A i |<<|W i | Usually need to meet B i and A i With W i There is a difference of one or more orders of magnitude between them, |B i | and |A i About W i |1% to 5% (or even lower). For example, W is a 1024*1024 matrix, B is a 1024*8 matrix, and A is an 8*1024 matrix.

[0054] Specifically, the S202 includes the following steps:

[0055] S2021, when the user submits the medical consultation text, the system will preprocess the text. The purpose of preprocessing is to identify and extract sensitive information in the text. In this step, natural language processing technology is used to perform word segmentation, part-of-speech tagging, and named entity recognition on the medical consultation text to ensure that sensitive content such as personal information and medical history records can be accurately identified. The system will automatically identify and extract sensitive information in the text, such as personal name, address, phone number, and medical history records, to form a sensitive information set I priv , the rest of the information constitutes the non-sensitive information set I non_priv This step lays the foundation for subsequent privacy protection and information processing.

[0056] S2022, calculate information entropy. Calculate the overall medical consultation text Q and sensitive information I respectively priv The information entropy calculation step adopts a sliding window or sentence level segmentation calculation method to calculate the information entropy of Q and I priv Calculate the entropy value in sections to more accurately reflect the importance of sensitive information in a certain part of the text. Use the following formula:

[0057]

[0058] Among them, p() means finding the probability;

[0059] Thus we get H(Q) and H(I priv ), and calculate the entropy ratio R:

[0060]

[0061] S2023, perform adaptive compression. Determine the compression ratio ρ according to the entropy ratio R. When ρ=1, When ρ=2, Let ρ = 4. According to ρ, the sensitive information I priv Perform grouping and dimensionality reduction processing to form compressed sensitive information In the adaptive compression step, the dimensionality reduction processing of sensitive information adopts mean aggregation or other aggregation methods to merge consecutive ρ tokens into one representation, and through this method, effective compression of sensitive information is achieved.

[0062]

[0063] Among them, t i The vector representation of the i-th token in the sensitive information.

[0064] This compression process significantly reduces the identifiability of sensitive information, further enhancing data privacy.

[0065] S2024, compressed sensitive information Non-sensitive information non_priv Merge to form the final compressed consultation text I comp ,Right now:

[0066]

[0067] Specifically, in order to further strengthen privacy protection, the compressed medical consultation text is cut into multiple non-overlapping parts in S203. Each part does not contain complete sensitive information, so no remote doctor model can obtain complete sensitive data alone. The segmented data will be transmitted to different remote doctor models for independent diagnosis. In this way, even if the model is attacked, only limited information can be obtained, avoiding the leakage of sensitive information. comp Divide the text into two non-overlapping parts I comp and I comp2 , ensuring that any remote doctor model cannot obtain complete sensitive data when processing independently. Then, the segmented data I comp1 and I comp2 The data is transmitted to the large language models (remote doctor model 1 and remote doctor model 2) that have been fine-tuned by LoRA in advance. Remote doctor model 1 and remote doctor model 2 conduct consultation and diagnosis respectively, and output diagnosis results O1 and O2, which are expressed as:

[0068]

[0069] in, and Respectively represent processing I comp1 and I comp2 Output function of the remote doctor model.

[0070] Specifically, in S204, the diagnosis results O1 and O2 are concatenated to form a joint input vector [O1; O2], which is then input into the chief physician model that has also been fine-tuned by LoRA. The final consultation result O is output by fusing the diagnosis results of each doctor. final , its calculation expression is:

[0071]

[0072] in, Represents the fusion function of the chief physician model.

[0073] To verify the effectiveness of the embodiments of the present invention, experiments were conducted using the widely used medical question-answering datasets MedMCQA, PubMedQA, and MedQA (including the MedQA_1 Simplified Chinese subset, the MedQA_2 Traditional Chinese subset, and the MedQA_3 English subset). The experimental platform was a computing device equipped with an RTX 4090 GPU. The experimental results are shown in Tables 1 to 4.

[0074] Table 1 - Comparison of diagnostic effects of the embodiments of the present invention and other methods:

[0075]

[0076] As shown in Table 1, the proposed ACP2LLM model performed significantly well on the MedMCQA dataset, achieving an accuracy of 83.40%. This represents a 22.70% improvement over the next-best method, GPT-3.5-turbo (60.70%). In the MedQA_1 and MedQA_2 subsets, ACP2LLM achieved accuracies of 65.42% and 63.67%, respectively, significantly outperforming other general-purpose and specialized medical models, demonstrating the superior diagnostic accuracy of this method across diverse language and medical cultural contexts.

[0077] Table 2 - Comparison of Wilcoxon index between the embodiment of the present invention and other methods:

[0078]

[0079] The Wilcoxon significance test results shown in Table 2 show that the performance difference between ACP2LLM and other models is statistically significant. Specifically, compared with the GPT-3.5-turbo model, ACP2LLM achieves a significant difference (p < 0.05); compared with other models (LLaMa2, Ziya-LLaMa, HuatuoGPT, and MedChatZH), its performance improvement is even more statistically significant (p < 0.01), confirming the robust superiority of ACP2LLM in medical diagnosis tasks.

[0080] Table 3 - Comparison of answer content generation quality between the embodiment of the present invention and other methods:

[0081]

[0082] As shown in Table 3, ACP2LLM achieved optimal performance in terms of relevance, coherence, and helpfulness. Its relevance score reached 99.21, significantly higher than other models, demonstrating its deep understanding of medical consultation questions and the accuracy of its responses. It also performed well in terms of coherence (7.49) and helpfulness (22.58), demonstrating the clear advantages of this method in providing practical and user-friendly medical advice.

[0083] Table 4 - Diagnostic effects of different compression rates in the embodiments of the present invention:

[0084]

[0085] From the experimental data analysis in Table 4, we can see that ACP2LLM's performance is stable under different compression rates (low, medium, and high). At the highest privacy compression ratio (compression rate of 4), it can achieve an average accuracy of 72.92%. This reflects that the present invention achieves good adaptability and balance between privacy protection and information retention. In addition, through the "multi-doctor-one chief physician" collaborative mechanism, ACP2LLM significantly improves the accuracy of diagnostic decisions based on the opinions of multiple doctors, fully demonstrating the effectiveness of the mechanism design of the present invention.

[0086] See also Figure 4 FIG. 1 is a system structure diagram of an embodiment of the present invention, including:

[0087] Training module 401, based on the large language model training to obtain several remote doctor models and a chief physician model; the remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; the chief physician model integrates the multiple diagnosis results and outputs a final diagnosis result;

[0088] Compression module 402 records the medical consultation text and performs adaptive compression to obtain a compressed medical consultation text;

[0089] The segmentation module 403 segments the compressed medical inquiry text into a plurality of non-overlapping partial texts, and transmits them to a plurality of remote doctor models respectively to obtain a plurality of diagnosis results;

[0090] The fusion module 404 combines several diagnosis results into a joint vector and inputs it into the chief physician model to obtain the final consultation result.

[0091] This paper provides a privacy protection method based on adaptive information entropy compression and a multi-doctor-chief physician collaboration mechanism. This method combines an adaptive information entropy compression algorithm with a multi-doctor collaboration mechanism. By compressing sensitive information, segmenting data, and enabling independent diagnosis across multiple models, this method aims to improve the efficiency and accuracy of large-scale medical consultations while ensuring privacy protection.

[0092] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A privacy protection method for medical consultation scenarios, characterized by: The following steps are involved: Based on the large language model training, several remote doctor models and one chief doctor model were obtained; The remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; The chief physician model fuses several diagnosis results and outputs a final diagnosis result; Record the medical consultation text and perform adaptive compression to obtain a compressed medical consultation text; The compressed medical consultation text is divided into several non-overlapping partial texts, which are transmitted to several remote doctor models respectively to obtain several diagnosis results; Several diagnostic results are spliced ​​together to form a joint vector and input into the chief physician model to obtain the final consultation result.

2. The privacy protection method for medical consultation scenarios according to claim 1 is characterized in that: The large language model-based training yields several remote doctor models and one chief physician model, specifically: A training set is constructed using historical consultation data and simulated sensitive information. The remote doctor model and the chief doctor model are pre-trained and LoRA fine-tuned. The LoRA fine-tuning is expressed as: W′ i =W i +B i A i ; Among them, W i is the weight matrix in the pre-trained large language model, B i With A i is a low-rank matrix satisfying |B i |,|A i |<<|W i |.

3. The privacy protection method for medical consultation scenarios according to claim 1 is characterized in that: The step of recording the medical consultation text and adaptively compressing the text to obtain the compressed medical consultation text includes the following steps: Divide the medical consultation text into sensitive information and non-sensitive information; Adaptively compress sensitive information based on the information entropy of the medical inquiry text and the information entropy of the sensitive information to obtain compressed sensitive information; The compressed sensitive information is merged with the non-sensitive information to form a compressed medical consultation text.

4. The privacy protection method for medical consultation scenarios according to claim 3 is characterized in that: The medical consultation text is divided into sensitive information and non-sensitive information, specifically: Natural language processing technology and preset regular expression rules are used to identify and extract sensitive information, including names, contact information and addresses.

5. The privacy protection method for medical consultation scenarios according to claim 3 is characterized in that: The method of adaptively compressing the sensitive information based on the information entropy of the medical inquiry text and the information entropy of the sensitive information to obtain the compressed sensitive information includes the following steps: The information entropy of the medical inquiry text and the information entropy of the sensitive information are calculated respectively, and then the entropy ratio R is calculated, which is expressed as: Among them, H(Q) represents the information entropy of the medical consultation text, H(I priv ) represents the information entropy of sensitive information; The compression ratio ρ is determined according to the entropy ratio R, which is expressed as: Sensitive information is grouped and dimensionally reduced according to the compression ratio ρ to form compressed sensitive information, which is expressed as: Among them, I priv represents sensitive information, k represents the group index after grouping, t i The vector representation of the i-th token in the sensitive information; Indicates compressed sensitive information.

6. The privacy protection method for medical consultation scenarios according to claim 5 is characterized in that: The information entropy of the medical inquiry text and the information entropy of the sensitive information are calculated separately, and the information entropy is calculated using a sliding window or sentence-level segmentation calculation method.

7. A privacy protection system for medical consultation scenarios, characterized by: include: The training module uses the large language model to train several remote doctor models and one chief physician model; The remote doctor model performs a diagnosis based on the input text and outputs a diagnosis result; The chief physician model fuses several diagnosis results and outputs a final diagnosis result; Compression module, records the medical consultation text and performs adaptive compression to obtain compressed medical consultation text; The segmentation module divides the compressed medical consultation text into several non-overlapping parts, which are transmitted to several remote doctor models to obtain several diagnosis results. The fusion module splices several diagnostic results into a joint vector and inputs it into the chief physician model to obtain the final consultation result.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images