Patient large model classification method and system based on memory and retrieval augmentation
By introducing memory plugins and gating mechanisms into the patient classification of the large patient model, the problem of insufficient utilization of knowledge resources in existing technologies is solved, achieving higher classification accuracy and personalized treatment recommendations, and improving the effectiveness of patient management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI PROVINCIAL HOSPITAL
- Filing Date
- 2024-08-22
- Publication Date
- 2026-06-23
Smart Images
Figure CN119150066B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a patient large model classification method and system based on memory and retrieval enhancement. Background Technology
[0002] In the field of patient management, the main goal of accurate patient classification is to automatically assign patients to appropriate treatment or management pathways based on given patient information (including disease descriptions, examination results, etc.). However, current methods fail to fully utilize existing knowledge resources, and single classification models lack multi-dimensional information references when dealing with complex conditions. Summary of the Invention
[0003] Based on the technical problems existing in the background technology, this invention proposes a patient large model classification method and system based on memory and retrieval enhancement, which can dynamically adjust the knowledge instances used according to the specific condition of the patient, thereby significantly improving the accuracy of patient classification.
[0004] The patient classification method based on memory and retrieval enhancement proposed in this invention inputs the current patient information into the trained medical large model to output the classification results of the patient information.
[0005] The training process of the large medical model is as follows:
[0006] Step 1: Obtain information from different patients to construct a training dataset. For the i-th data point in the training dataset, retrieve the corresponding knowledge instance s using the retrieval module. n ;
[0007] Step 2: For each knowledge instance s n The latent representation of the knowledge instance is obtained by encoding through an encoder. The latent representation of the knowledge instance is mapped to the key representation and value representation through the key matrix and value matrix. The latent representation of the i-th data is used as the query. The weight of the knowledge instance corresponding to the i-th data is calculated based on the inner product between the key representation, value representation and query. The weight of the knowledge instance is then applied to the value representation to obtain the knowledge instance representation corresponding to the i-th data.
[0008] Step 3: Input the i-th data into the large language model, and use the gating mechanism to incorporate the knowledge instance representation corresponding to the i-th data into the information flow of the large language model, thereby outputting the classification result of the i-th data; construct a loss function based on the classification results of all data to adjust the trainable parameters in the medical large model.
[0009] Furthermore, in step two, the knowledge instance representation a corresponding to the i-th data is... i,S The calculation is as follows:
[0010]
[0011]
[0012] Where, p i,{n} This represents the weight of the knowledge instance corresponding to the i-th data point. This represents the key representation of the i-th data. h represents the value corresponding to the i-th data. i,{X} Let S represent the latent representation of the i-th data, * denote the vector inner product operation, and S n This represents the retrieved knowledge instance.
[0013] Furthermore, in step three, a gating mechanism is used to incorporate the knowledge instance representation corresponding to the i-th data into the information flow of the large language model, specifically as follows:
[0014] The i-th data is encoded through the Transformer layer in the large language model;
[0015] By incorporating knowledge instance representations into the standard Transformer decoding process using a gating mechanism, the information in the large language model and the incorporated knowledge instance representations are balanced by a gate reset mechanism based on the gating mechanism. The output of the last Transformer layer passes through a fully connected layer and a softmax classifier in sequence to output the classification result.
[0016] During the decoding process, the gating mechanism between the l-th and l+1-th Transformers calculates the reset gate using the following formula:
[0017]
[0018] in, This represents the l-th layer reset gate corresponding to the i-th data, and sigma represents the sigmoid function. This represents the output of the l-th layer Transformer. and These are the trainable parameters of the l-th layer Transformer, a i,S This represents the knowledge instance representation corresponding to the i-th data.
[0019] Furthermore, the information and the incorporated knowledge instances in the gate-balanced large language model based on the gating mechanism are specifically represented as follows:
[0020]
[0021] in, This is the output of the l-th layer Transformer; 1 is a matrix with the same dimensions as the reset gate, and all values are 1. This represents the input to the (l+1)th layer Transformer.
[0022] The patient classification system based on memory and retrieval enhancement inputs current patient information into a trained medical model to output the classification results of the patient information.
[0023] The training process of the large medical model is as follows:
[0024] Step 1: Obtain information from different patients to construct a training dataset. Then, retrieve the corresponding knowledge instance for the i-th data in the training dataset using the retrieval module.
[0025] Step 2: Encode each knowledge instance using an encoder to obtain its latent representation. Map the latent representation of the knowledge instance to the key representation and value representation using the key matrix and value matrix. Use the latent representation of the i-th data as the query. Calculate the weight of the knowledge instance corresponding to the i-th data based on the inner product of the key representation, value representation, and query. Apply the weight of the knowledge instance to the value representation to obtain the knowledge instance representation corresponding to the i-th data.
[0026] Step 3: Input the i-th data into the large language model, and use the gating mechanism to incorporate the knowledge instance representation corresponding to the i-th data into the information flow of the large language model, thereby outputting the classification result of the i-th data; construct a loss function based on the classification results of all data to adjust the trainable parameters in the medical large model.
[0027] Furthermore, in step three, a gating mechanism is used to incorporate the knowledge instance representation corresponding to the i-th data into the information flow of the large language model, specifically as follows:
[0028] The i-th data is encoded through the Transformer layer in the large language model;
[0029] By incorporating knowledge instance representations into the standard Transformer decoding process using a gating mechanism, the information in the large language model and the incorporated knowledge instance representations are balanced by a gate reset mechanism based on the gating mechanism. The output of the last Transformer layer passes through a fully connected layer and a softmax classifier in sequence to output the classification result.
[0030] During the decoding process, the gating mechanism between the l-th and l+1-th Transformers calculates the reset gate using the following formula:
[0031]
[0032] in, This represents the l-th level reset gate, and sigma represents the sigmoid function. This represents the output of the l-th layer Transformer. and These are the trainable parameters of the l-th layer Transformer, a i,S This represents the knowledge instance representation corresponding to the i-th data.
[0033] Furthermore, the information and the incorporated knowledge instances in the gate-balanced large language model based on the gating mechanism are specifically represented as follows:
[0034]
[0035] in, It is the output of the l-th layer Transformer and the input of the (l+1)-th layer Transformer; 1 is a matrix with the same dimensions as the reset gate, and all values are 1.
[0036] The advantages of the patient large model classification method and system based on memory and retrieval enhancement provided by this invention are as follows: The patient large model classification method and system based on memory and retrieval enhancement provided in the structure of this invention (1) improves the accuracy of patient classification: This embodiment extracts relevant knowledge instances from the knowledge base by using a memory plugin, and optimizes the knowledge encoding and modeling process of the memory plugin by combining contrastive learning, so that LLM can better utilize new knowledge. Compared with the traditional single classification model, this embodiment can dynamically adjust the knowledge instances used according to the specific condition of the patient, thereby significantly improving the accuracy of patient classification; (2) provides personalized treatment suggestions: This invention introduces a gating mechanism in LLM, which enables it to generate classification results by comprehensively weighting knowledge instances. This not only improves the performance of the medical large model, but also enables it to provide more personalized treatment suggestions according to the specific condition of the patient. Compared with the traditional method, this embodiment can better utilize multi-dimensional information reference, thereby providing more effective and accurate treatment plans when dealing with complex conditions. Attached Figure Description
[0037] Figure 1 This is a schematic diagram of the structure of the present invention;
[0038] Figure 2 This is a flowchart of the training process for a large medical model. Detailed Implementation
[0039] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0040] This embodiment's large-scale medical model is built upon a Large Language Model (LLM) and a memory plugin. This allows the trained model to extract and encode relevant knowledge instances from the knowledge base using the memory plugin, thus better utilizing new knowledge. Simultaneously, the LLM, combined with a gating mechanism, generates classification results by comprehensively weighting the knowledge instances. This embodiment not only improves the accuracy of patient classification but also provides personalized treatment suggestions based on the patient's specific condition, as detailed below.
[0041] like Figure 1 and 2 As shown, the patient large model classification method based on memory and retrieval enhancement proposed in this invention inputs the current patient information into the trained medical large model to output the classification results of the patient information;
[0042] The training process of the large medical model is as follows:
[0043] Step 1: Obtain information from different patients to construct a training dataset. Then, retrieve the corresponding knowledge instance for the i-th data in the training dataset using the retrieval module.
[0044] The retrieval module retrieves knowledge instances corresponding to the i-th data point from existing resources. Specifically, it uses existing toolkits to extract knowledge instances based on the input and a knowledge base (e.g., Wikipedia), where these knowledge instances guide the LLM in generating correct responses. This makes the classification results generated by the large medical model more consistent with existing classification rules, reducing misclassifications.
[0045] Step 2: For each knowledge instance s n The latent representation of the knowledge instance is obtained by encoding through an encoder. The latent representation of the knowledge instance is mapped to the key representation and value representation through the key matrix and value matrix. The latent representation of the i-th data is used as the query. The weight of the knowledge instance corresponding to the i-th data is calculated based on the inner product between the key representation, value representation and query. The weight of the knowledge instance is then applied to the value representation to obtain the knowledge instance representation corresponding to the i-th data.
[0046] To better utilize these knowledge instances from step one and incorporate them into the LLM, this embodiment proposes a memory plugin, which includes a knowledge encoder and a memory mechanism. First, a knowledge encoder (e.g., a Transformer encoder) is used to extract their latent representations. Specifically, for each knowledge instance... i，n The input is to the encoder, and its latent representation is computed through the final encoding layer:
[0047]
[0048] in, For knowledge instances si，n The potential representation, s i，n f is the knowledge instance corresponding to the i-th data. {KE} For encoder.
[0049] Based on memory mechanisms, knowledge instances s i，n Potential representation Further data processing: For each knowledge instance, the extracted latent representation is mapped to the key representation using a key matrix and a value matrix. Sum value representation Then, using the latent representation h of the input... i，{X} As a query, and calculate the weight of each knowledge instance based on their inner product:
[0050]
[0051] Where, p i，{n} This represents the weight of the knowledge instance corresponding to the i-th data point. The key indicates that, The value represents h. i，{X} represents the latent representation of the i-th data, and * represents the vector inner product operation.
[0052] It should be noted that h i，{X} The latent representation of the i-th data is the latent representation corresponding to the input, and The latent representation of a knowledge instance is the knowledge instance S retrieved by the retrieval module, with the input as the index. i，n The corresponding latent representations are obtained through the same process, which is achieved by an encoder, only the input for encoding is different.
[0053] Furthermore, when the i-th data is used as input, the retrieval module can retrieve one or more knowledge instances. Multiple knowledge representations corresponding to the same input can be distinguished by different values of n. That is, when the same input corresponds to 4 knowledge instances, n = 1, 2, 3, 4, which correspond to s respectively. i，1 s i，2 s i，3 s i，4 For each knowledge instance, simply follow steps two and three in sequence.
[0054] The obtained weight p i，{n} Applying this to the corresponding value representation, and obtaining the knowledge instance representation a using the following formula. i，S :
[0055]
[0056] Where sum represents the total.
[0057] In step two, different knowledge instances are assigned different weights based on the input. Therefore, knowledge instances that are crucial to modeling the input receive higher weights than those that are less important. The weighted knowledge instance representation contains key information for generating the input response and is incorporated into subsequent LLM processing, thereby improving the classification accuracy of the large-scale medical model.
[0058] Step 3: Input the i-th data into the large language model, and use the gating mechanism to incorporate the knowledge instance representation corresponding to the i-th data into the information flow of the large language model, thereby outputting the classification result of the i-th data; construct a loss function based on the classification results of all data to adjust the trainable parameters in the medical large model.
[0059] The hints provided by an LLM have a significant impact on its performance. Therefore, once knowledge instances are encoded and weighted, how the resulting knowledge representation is incorporated into the LLM plays a crucial role in guiding the LLM to generate responses. Intuitively, the contribution of knowledge information can vary across different contexts. For example, knowledge acquired from the memory module may be less important if the LLM has already learned it. Therefore, balancing the contribution of knowledge information when incorporating it into the LLM is essential.
[0060] This embodiment proposes using a gating module to control the information flow when incorporating knowledge information into the LLM. Specifically, the input text is first input into the LLM and encoded through the Transformer layers within the LLM. For each pair of connected Transformer layers in the LLM, a gating mechanism is used to incorporate the knowledge instance representation into the standard Transformer decoding process. The gating mechanism between the l-th and l+1-th Transformer layers calculates the reset gate using the following formula:
[0061]
[0062] in, This represents the l-th layer reset gate corresponding to the i-th data, and sigma represents the sigmoid function. This represents the output of the l-th layer Transformer. and These are the trainable parameters of the l-th layer Transformer, a i S represents the knowledge instance representation corresponding to the i-th data.
[0063] The information and knowledge instance representations in the gate-balanced large language model based on the gating mechanism are as follows:
[0064]
[0065] in, It is the output of the l-th layer Transformer and the input of the (l+1)-th layer Transformer; 1 is a matrix with the same dimensions as the reset gate, and all values are 1.
[0066] at last, The input is fed into the next Transformer layer, and the output of the final Transformer layer is obtained according to the standard decoding process. Following the conventions of standard Transformer decoders, the last hidden vector of the matrix is fed into a fully connected layer, and the output label is predicted by a softmax classifier, thus forming the output.
[0067] Through steps one through three, this embodiment aims to generate a response (e.g., an answer) based on a given input text (e.g., a question) by leveraging a list of knowledge instances obtained from existing resources (e.g., an information retrieval system). The overall method architecture comprises two parts: a memory plugin and a gated large model (LLM). The memory plugin extracts latent features of knowledge instances using a knowledge encoding process, and then uses a memory module to weight these encoded knowledge instances according to their contribution to solving the test example; the LLM prompting process uses the encoded and weighted knowledge to generate the output, resulting in a medical large model capable of accurately classifying the input.
[0068] Whether this embodiment is applicable to different disease grades or categories, such as liver disease and mental illness, and whether it can input training datasets for different diseases to obtain large-scale medical models for different diseases, this embodiment has the following advantages:
[0069] (1) Improved accuracy of patient classification: This embodiment extracts relevant knowledge instances from the knowledge base using a memory plugin and optimizes the knowledge encoding and modeling process of the memory plugin through contrastive learning, enabling LLM to better utilize new knowledge. Compared with traditional single classification models, this embodiment can dynamically adjust the knowledge instances used according to the specific condition of the patient, thereby significantly improving the accuracy of patient classification.
[0070] (2) Providing personalized treatment recommendations: This invention introduces a gating mechanism into LLM, enabling it to generate classification results by comprehensively weighted knowledge instances. This not only improves the performance of the large medical model but also allows it to provide more personalized treatment recommendations based on the patient's specific condition. Compared to traditional methods, this embodiment can better utilize multi-dimensional information references, thereby providing more effective and accurate treatment plans when dealing with complex conditions.
[0071] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A patient large-scale model classification method based on memory and retrieval enhancement, characterized in that, Based on a given input text, the goal is to generate an answer by utilizing a list of knowledge instances obtained from an information retrieval system, where current patient information is input into a trained large medical model to output a classification result of the patient information; The training process of the large medical model is as follows: Step 1: Obtain information from different patients to construct a training dataset. Then, for the first patient in the training dataset... Each piece of data is retrieved using the retrieval module to obtain corresponding knowledge instances; Step 2: Encode each knowledge instance using an encoder to obtain its latent representation. Map the latent representation of the knowledge instance to its key and value representations using a key matrix and a value matrix, respectively. The potential representation of each data point is used as a query, and the first data point is calculated based on the key representation, value representation, and the inner product between queries. The weight of each data point corresponds to a knowledge instance, and this weight is applied to the value representation to obtain the first data point. The knowledge instance representation corresponding to each data point; Step 3, place the first The first data point is input into the large language model, and the second data point is then processed using a gating mechanism. The knowledge instance representation corresponding to the first data point is incorporated into the information flow of the large language model, thereby outputting the first data point. The classification results of all data; a loss function is constructed based on the classification results of all data to adjust the trainable parameters in the large medical model; In step three, the gating mechanism is used to... The knowledge instance representation corresponding to each data point is incorporated into the information flow of the large language model, specifically as follows: No. Each piece of data is encoded through the Transformer layer in the large language model; By incorporating knowledge instance representations into the standard Transformer decoding process using a gating mechanism, the information in the large language model and the incorporated knowledge instance representations are balanced by a gate reset mechanism based on the gating mechanism. The output of the last Transformer layer passes through a fully connected layer and a softmax classifier in sequence to output the classification result. In the decoding process, for the first... Layer and first The gating mechanism between Transformer layers calculates the reset gate using the following formula: in, Indicates the first The corresponding data point is the first... Reset the door at each floor. This represents the sigmoid function. Indicates the first The output of the layer Transformer and It is the first Trainable parameters of a Transformer layer Indicates the first Each piece of data represents a knowledge instance.
2. The patient large-scale model classification method based on memory and retrieval enhancement according to claim 1, characterized in that, In step two, the first The knowledge instance representation of each data point The calculation is as follows: in, Indicates the first Each data point corresponds to a weight of a knowledge instance. Indicates the first The key represents each piece of data. Indicates the first The value corresponding to each data point represents, Indicates the first Data The potential representation, This represents the vector dot product operation. This represents the retrieved knowledge instance.
3. The patient large-scale model classification method based on memory and retrieval enhancement according to claim 1, characterized in that, The information and knowledge instance representations in the gate-balanced large language model based on the gating mechanism are as follows: in, It is the first The output of the Transformer layer; 1 is a matrix with the same dimensions as the reset gate, and all values are 1. Indicates the first The input to the Transformer layer.
4. A patient large-scale model classification system based on memory and retrieval enhancement, characterized in that: Based on a given input text, the goal is to generate an answer by utilizing a list of knowledge instances obtained from an information retrieval system, where current patient information is input into a trained large medical model to output a classification result of the patient information; The training process of the large medical model is as follows: Step 1: Obtain information from different patients to construct a training dataset. Then, for the first patient in the training dataset... Each piece of data is retrieved using the retrieval module to obtain corresponding knowledge instances; Step 2: Encode each knowledge instance using an encoder to obtain its latent representation. Map the latent representation of the knowledge instance to its key and value representations using a key matrix and a value matrix, respectively. The potential representation of each data point is used as a query, and the first data point is calculated based on the key representation, value representation, and the inner product between queries. The weight of each data point corresponds to a knowledge instance, and this weight is applied to the value representation to obtain the first data point. The knowledge instance representation corresponding to each data point; Step 3, place the first The first data point is input into the large language model, and the second data point is then processed using a gating mechanism. The knowledge instance representation corresponding to the first data point is incorporated into the information flow of the large language model, thereby outputting the first data point. The classification results of all data; a loss function is constructed based on the classification results of all data to adjust the trainable parameters in the large medical model; In step three, the gating mechanism is used to... The knowledge instance representation corresponding to each data point is incorporated into the information flow of the large language model, specifically as follows: No. Each piece of data is encoded through the Transformer layer in the large language model; By incorporating knowledge instance representations into the standard Transformer decoding process using a gating mechanism, the information in the large language model and the incorporated knowledge instance representations are balanced by a gate reset mechanism based on the gating mechanism. The output of the last Transformer layer passes through a fully connected layer and a softmax classifier in sequence to output the classification result. In the decoding process, for the first... Layer and first The gating mechanism between Transformer layers calculates the reset gate using the following formula: in, Indicates the first Reset the door at each floor. This represents the sigmoid function. Indicates the first The output of the layer Transformer and It is the first Trainable parameters of a Transformer layer Indicates the first Each piece of data represents a knowledge instance.
5. The patient large-scale model classification system based on memory and retrieval enhancement according to claim 4, characterized in that, The information and knowledge instance representations in the gate-balanced large language model based on the gating mechanism are as follows: in, It is the first The output of the Transformer layer; 1 is a matrix with the same dimensions as the reset gate, and all values are 1. Indicates the first The input to the Transformer layer.