Large language model disease screening method and system based on token perception

Abstract

The invention discloses a big language model disease screening method and system based on token perception. According to the method, a training sample set containing non-disease and disease samples is constructed, mutual information is utilized to construct a graph network, and multiple rounds of graph propagation are performed to strengthen useful information in tokens. And then, obtaining a final representation through graph aggregation, constructing a final training set, and training a probe to determine an editing direction so as to guide a large language model to generate a text conforming to medical standards. By utilizing the method, the performance of the large language model in screening of mental and psychological diseases can be effectively improved, and a reliable auxiliary tool is provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to a token-aware large language model disease screening method and also relates to a corresponding large language model disease screening system, belonging to the technical field of medical care informatics. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have shown great application potential in the medical field, especially in mental health services. For example, large language models can be used to analyze patients' self-narration or chat records during the diagnosis and treatment process, identify emotional clues and language features, and help doctors screen patients who may have mental illnesses such as depression more quickly. In addition, large language models can also be used as virtual diagnosis and treatment large language models to interact with patients with mental illnesses. For example, by analyzing patient input, large language models can generate appropriate responses to help patients sort out their emotions and provide supportive feedback, thereby achieving the effect of auxiliary diagnosis and treatment to intervene in mental illnesses. However, if the content generated by the large language model deviates from medical norms or treatment standards, it may have a serious impact on the patient's health, such as misleading patients to take inappropriate treatment methods or inducing dangerous behaviors in patients with mental illnesses.

[0003] Compared with supervised fine-tuning (SFT) and reinforcement learning based on human feedback (RLHF), inference-time intervention (ITI) technology has become a powerful means to solve this problem due to its advantages of high efficiency and low cost. ITI technology trains interpretable probes on positive and negative samples (such as depressed and non-depressed patient groups) to identify potential problems in the generation of medical content by large language models, and then intervenes in the internal representation of the large language model in real time during the inference process to ensure that the generated content meets medical standards and diagnosis and treatment needs. This reduces the dependence on large amounts of training data and computing resources, reduces application costs, and avoids introducing new risks, such as large language models generating non-professional advice that flatters patients.

[0004] However, existing ITI techniques use coarse-grained sentence-level analysis, ignoring important information of individual tokens, resulting in unsatisfactory intervention results. In addition, existing ITI techniques are difficult to adapt to the individualized and complex scenarios of mental illness, and fail to fully solve the problem of flexibility in content generation in dynamic scenarios. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a large language model disease screening method based on token perception.

[0006] Another technical problem to be solved by the present invention is to provide a large language model disease screening system based on token perception.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a token-aware large language model disease screening method is provided, comprising the following steps:

[0009] Construct a dataset containing non-disease samples + and disease samples - The training sample set S. Each sample s = {x1, ..., x n} consists of multiple tokens x i Composition, expressed as s = {o1, ..., o n}, where each o i Corresponds to a token x i ;

[0010] Construct a graph network based on mutual information for each training sample;

[0011] Perform r rounds of graph propagation based on the graph network. In each round of propagation, update the information of vertices and edges, thereby strengthening the useful information inherent in each token before graph aggregation;

[0012] Estimate the univariate entropy of token representations;

[0013] Perform graph aggregation on the enhanced tokens to obtain the final representation to construct the final training set;

[0014] Based on the final training set, the training probes are calculated And determine the editing direction; where the editing direction is applied with the same strength to the large language model's prediction of the next token when generating text.

[0015] Preferably, the graph network middle, represents the token representations generated from all large language models o k：n The abstract vertex set, ε = {e i，j |i, j∈[k, n]} denotes the edge set based on mutual information, which represents the mutual information between tokens;

[0016] The graph network of round r The formula is as follows:

[0017]

[0018] Among them, H(·) represents the univariate Shannon entropy, and H(·,·) represents the joint entropy;

[0019] In the rth round of propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the edge weight To weight neighbor vertices information; edge The update is done by calculating the two vertices Mutual information between achieved.

[0020] Preferably, the training probe is obtained based on the following formula: And determine the editing direction

[0021]

[0022] In the formula, Indicates that in the training set The expected value of the cross entropy loss on ; For token x i The final representation of; CE represents the cross entropy loss, y represents the true label; represents the i-th vertex after r rounds of propagation, n represents the total number of nodes, and k represents the number of selected neighbor nodes.

[0023] Preferably, the large language model disease screening method further comprises the following steps:

[0024] Train an automatic inconsistency estimator to assess the inconsistency of the predictions from the token representation m t ;

[0025] By quantifying the prediction uncertainty, we can get the uncertainty u of the quantified token. t ;

[0026] During the intervention phase, the estimated level of inconsistency m t and the uncertainty of the quantized token u t , for adaptive token intervention.

[0027] Preferably, by obtaining the head W of the large language model LM The predicted probability distribution σ(W LM o t ), and calculate the normalized entropy by the vocabulary size z to obtain the uncertainty u of the quantized token t :u t =H(σ(W LM o t )) / log(z);

[0028] Among them, H(·) represents the univariate Shannon entropy, o tRepresents the token representation generated for the tth time.

[0029] Preferably, a token x will be generated that will result in inaccurate medical advice. j is marked as 1, while other tokens not related to medical knowledge are marked as 0; for a given sample pair (s + ,s - ), + Zhong and s - Different tokens are considered inconsistent tokens and token x j The inconsistent labels y j The annotations are as follows:

[0030]

[0031] in, Represents the token-level anomaly dataset formed, o j Indicates inconsistent samples For each token x j Representation.

[0032] Preferably, during inference, the automatic inconsistency estimator E m Process the token representation o generated at the tth time t , and estimate the potential inconsistency, and get the estimated inconsistency m t :

[0033]

[0034] Preferably, different intervention intensities are distinguished for different tokens. satisfy:

[0035]

[0036] in, Indicates the generation of output consistent with actual medical knowledge and ethical standards; editorial direction Applied to each prediction of the next token with the same strength α; β represents the balancing factor.

[0037] According to a second aspect of an embodiment of the present invention, a token-aware large language model disease screening system is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the above-mentioned large language model disease screening method.

[0038] Compared with the prior art, the present invention has the following technical features:

[0039] 1) Token-level intervention mechanism: This invention performs fine-grained processing on text data related to mental illness, captures the contextual spatial relationship of all input information, perceives the subtle emotions and symptom expressions in the language, and performs adaptive intervention on the sequence output at each moment to improve the recognition accuracy of the large language model. Compared with traditional sentence-level processing, token-level intervention can dynamically adjust the representation of different words and phrases, making the large language model more sensitive and accurate when facing complex descriptions of mental illness.

[0040] 2) Mutual information-guided token aggregation: Through the mutual information (MI)-guided graph propagation mechanism, the present invention can establish spatial relationships for sharing contextual information between tokens, enhance the large language model's understanding of the contribution of different tokens in the description of patient symptoms, thereby optimizing the internal representation of the large language model and helping the large language model to better generate content that conforms to medical knowledge and diagnosis and treatment standards.

[0041] 3) Inconsistency-aware adaptive token intervention: The present invention uses an inconsistency-aware adaptive token intervention mechanism (MAI) to dynamically adjust the intervention intensity of the text sequence output at each moment according to the specific performance and prediction uncertainty of each token. This mechanism can help the large language model identify and correct potential errors or generated content that does not meet the requirements of mental illness diagnosis and treatment, ensuring that the final result is more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram showing a comparison between the token-aware large language model disease screening method provided by an embodiment of the present invention and the prior art;

[0043] Figure 2 This is a flow chart of a disease screening method based on a token-aware large language model in the first embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of a token-aware large language model disease screening system in the second embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] The technical concept in the embodiment of the present invention is to establish accurate and unbiased text sequence output by finely analyzing the correlation between each token in the entire sentence context space and its contribution in generating medical content, and significantly improve the performance of the large language model in the diagnosis and treatment of mental and psychological diseases. Furthermore, based on the inconsistent perception of each token, the intervention intensity is dynamically adjusted to achieve adaptive token intervention, ensuring that the content generated by the large language model meets clinical standards and medical needs.

[0047] Specifically, the large language model disease screening method provided by the embodiment of the present invention eliminates the adverse effects of directional deviation and the same intervention intensity by introducing mutual information-guided token aggregation (abbreviated as MIG) and inconsistency-aware adaptive token intervention (abbreviated as MAI). Specifically, MIG captures and utilizes the key interactions in the context space between tokens (i.e., words or phrases in the text) by constructing a graph network based on mutual information. In MIG, each token is regarded as a vertex in the graph, and the mutual information between tokens is used as an edge, representing the degree of information sharing between them. Through graph propagation, the large language model can strengthen the useful information inherent in each token before aggregation, thereby optimizing the internal representation of the large language model and helping the large language model to better generate content that conforms to medical knowledge and diagnosis and treatment standards. MAI dynamically adjusts the intervention intensity output for the text sequence at each moment according to the specific performance and prediction uncertainty of each token. MAI evaluates the degree of inconsistency predicted by the token representation by training an automatic inconsistency estimator, and perceives the abnormality of the token in combination with the uncertainty predicted by the large language model. During the intervention phase, adaptive token intervention is conducted based on the estimated degree of inconsistency and quantified token uncertainty to identify and correct potential errors or generated content that does not meet the requirements of mental and psychological disease diagnosis and treatment, ensuring that the final output is more reliable.

[0048] The two mechanisms of MIG and MAI work together to enable the large language model disease screening method provided by the embodiment of the present invention to capture key information more accurately and improve the recognition accuracy and reliability of the large language model in the diagnosis and treatment of mental and psychological diseases. Through MIG, the large language model can enhance the ability to understand medical information in the training samples; while MAI adjusts the intervention intensity through inconsistency perception to achieve token-level precision intervention.

[0049] First embodiment

[0050] Next, taking the screening of depression patients based on text content such as blogs and WeChat as an example, the specific implementation process of the large language model disease screening method provided by the first embodiment of the present invention is described.

[0051] When a large language model based on decoding generates content related to mental illness, it may be possible to use the head W of the large language model. LM Generate content that is inconsistent with medical knowledge and treatment standards. Studies have shown that there is an explainable correlation between these inaccurate contents and the internal representation of large language models. Therefore, ITI technology can guide large language models to generate content that is more in line with medical needs by sparsely intervening in the internal representation of large language models. Specifically in the scenario of mental and psychological disease screening, ensuring that the content generated by large language models is consistent with actual medical knowledge and ethical standards is crucial to patient safety.

[0052] The existing ITI technology is based on sentence-level analysis and includes the following steps:

[0053] S1: Use detection techniques to distinguish between content that matches medical knowledge and content that does not meet medical requirements in the internal representation of the large language model.

[0054] This step includes the following sub-steps:

[0055] S11) Construct a sample containing non-depressed + and depression samples - Sample set Each sample s={x1,...,x n} is composed of multiple tokens x i The sentence composed of , which represents the hth head from the multi-head self-attention (MHSA) in the lth layer decoder, is expressed as Each of these Corresponds to a token x i ;

[0056] S12) Use the representation of the last token As the whole sample s l，h Representatives and form a data set y represents the label of the sample (a negative y indicates that the sample is depressed; a positive y indicates that the sample is not depressed);

[0057] S13) By training probe P l，h To distinguish the direction of mental health. Find the parameter d that minimizes the cross entropy loss through the following formula: l，h :

[0058]

[0059] In formula (1), d l，h Indicates probe P l，h The learned parameters are also the editing direction of the content generated by the large language model. y labels each sample as depressed or non-depressed, and CE represents the cross entropy loss, which is used to measure the difference between the prediction of the probe and the true label y. Indicated in the dataset The expected value of the cross entropy loss on .

[0060] S2: During training, evaluate the accuracy of different heads.

[0061] Accuracy refers to the proportion of sample labels that the probe correctly predicts. On the validation set, the large language model tests the accuracy of different heads and then selects the sparse heads with the highest accuracy on the validation set for intervention. These heads perform best on the validation set, meaning they can most accurately distinguish between depressed and non-depressed samples and can be used for further analysis or intervention. The selection of sparse heads also helps reduce the complexity of the large language model and improve its efficiency and interpretability.

[0062] S3: In the intervention phase, the editing direction d l，h The same strength α is applied to each prediction of the next token by the large language model when generating text, thereby biasing the representation in the direction of mental health and guiding the large language model to generate outputs consistent with actual medical knowledge and ethical standards:

[0063]

[0064] in, Represents the token representation generated by the large language model after intervention. Analysis shows that the direction of deviation d l ，h and the same strength α was determined based on sentence-level analysis, weakening the psychological health output after the intervention.

[0065] Different from the existing ITI technology, see Figure 1 and Figure 2 , the large language model disease screening method provided by the embodiment of the present invention includes a training process and a testing process. In the training process, after the training data is input into the large language model, the representation after the multi-head self-attention module (MHSA) of each layer is extracted, and input into MIG for interpretable probe training to obtain the probe and editing direction. In the testing process, after the test data is input into the large language model, the representation after the multi-head self-attention module (MHSA) that needs intervention is obtained; the above representation is edited using MAI to obtain the edited representation; the edited representation is given as input to the subsequent feedforward layer (FFN). Finally, the prediction result is obtained through the large language model.

[0066] Specifically, the large language model disease screening method provided by the embodiment of the present invention includes the following steps:

[0067] Step 1: Construct a sample containing non-disease (e.g. non-depression) + and disease (e.g. depression) samples - The training sample set Each sample s={x1,...,x n} consists of multiple tokens x i Composition, expressed as s = {o1, ..., o n}, where each oi Corresponds to a token x i ;

[0068] Step 2: Build a graph network based on mutual information (MI) for each training sample

[0069] Specifically, graph networks middle, represents the token representations generated from all large language models o k：n The abstract vertex set, ε = {e i，j |i, j∈[k, n]} represents the edge set based on mutual information, which represents the mutual information between tokens. Initial graph In each vertex Represents the corresponding token representation o i , each edge express and The mutual information between them.

[0070] Step 3: Perform r rounds of graph propagation based on the graph network. In each round of propagation, update the information of vertices and edges to strengthen the useful information inherent in each token before graph aggregation.

[0071] The graph network of round r The formula is as follows:

[0072]

[0073] In formula (3), H(·) represents the univariate Shannon entropy, and H(·,·) represents the joint entropy.

[0074] In the rth round of propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the edge weight To weight neighbor vertices Information. The update is done by calculating the two vertices The mutual information between The univariate Shannon entropy of two vertices and joint entropy Calculated.

[0075] Because in each round, the update of the vertex is achieved by aggregating the information of its neighbor vertices, using the weight of the edge To weight neighbor vertices Therefore, the weight of the edge is used in the aggregation process to strengthen the useful information inherent in each token. Therefore, neighboring tokens with higher mutual information with the current token will have a greater impact, thereby strengthening the useful information of the current token. That is, the mutual information between tokens is used to adjust the representation of tokens, making the representation of each token richer and more useful.

[0076] Moreover, Shannon entropy is a measure of the uncertainty of a random variable, and univariate entropy can be used to quantify the amount of information carried by a single token. The higher the entropy, the greater the uncertainty of the token and the more information it carries. In text processing, this means that it is possible to identify which tokens have strong semantic associations. Therefore, by calculating the mutual information between tokens, the token combination that contributes most to the meaning of the sentence can be found; by identifying and strengthening tokens with high mutual information, the large language model can then pay more attention to information that is critical to understanding a sentence or document. This helps the large language model filter out noise or irrelevant information when processing text, thereby improving the quality and relevance of information. The representation of tokens can be optimized through feature selection and feature weighting guided by mutual information. This enables the large language model to pay more attention to features that have a direct impact on the prediction task during training. It can be seen that in an embodiment of the present invention, mutual information identifies and quantifies the degree of information sharing between tokens, updates tokens based on the degree of sharing, and improves the quality and relevance of information.

[0077] Moreover, in adaptive token intervention (e.g., MAI), mutual information can be used to dynamically adjust the weights of tokens to reflect their importance in a specific context. This approach allows large language models to flexibly adjust their focus when faced with different tasks or datasets.

[0078] Here, the univariate entropy of the token representation is estimated When , we discretize the continuous representation into bins to calculate the probability distribution according to the method introduced by Steuer et al. in the paper "The mutual information: detecting and evaluating dependencies between variables" (Bioinformatics. 2002: 18 Suppl 2: S231-40.), thereby estimating the univariate entropy of the token representation.

[0079] Step 4: Perform graph aggregation on the enhanced tokens to obtain the final representation It contains richer and more discriminative information to construct the final training set

[0080] Step 5: Based on the final training set The training probe is calculated based on the following formula And determine the editing direction

[0081]

[0082] In the above formula, Indicates that in the training set The expected value of the cross entropy loss on ; the final representation CE stands for cross entropy loss, which is used to measure the probe The difference between the prediction and the true label y; represents the i-th vertex after r rounds of propagation, n represents the total number of nodes, and k represents the number of selected neighbor nodes.

[0083] It can be seen from formula (4) that it is similar to formula (1) in the existing ITI technology, but the training set used in the embodiment of the present invention is different, and the optimized final representation is used.

[0084] Typically in graph networks, node features are propagated and aggregated through the graph structure to update the representation of each node; by minimizing the cross entropy loss, the large language model learns how to predict the node's label based on the node's local graph structure information.

[0085] As is known to all, sentence strength and token granularity are two different methods for understanding and generating text in natural language processing. The embodiment of the present invention effectively alleviates the problem of direction deviation by combining the advantages of sentence strength and token granularity analysis, and realizes more accurate direction exploration. Specifically, first, by considering the strength of the entire sentence, the large language model can capture the overall emotional tendency or theme of the sentence, which helps to understand the text at a macro level and ensure that the output of the large language model is consistent with the overall intention of the sentence. Sentence strength, as an overall guiding signal, helps the large language model to maintain the correct alignment direction when processing text, especially in sentiment analysis tasks. Secondly, using token granularity analysis, the large language model can understand the text at a finer level, that is, analyze each word's contribution to the overall meaning of the sentence word by word, which helps to identify and strengthen those words that have a significant impact on the sentiment of the sentence. Finally, through mutual information enhancement, the large language model can more accurately determine the contribution of each token to the direction of the sentence, thereby realizing the alignment direction at the micro level. This not only helps the large language model to more accurately reflect the information of each word when generating text, but also improves the relevance and accuracy of the text. Combining the analysis of these three aspects, the embodiment of the present invention can fully understand the text at different levels and achieve more accurate text processing and generation.

[0086] It can be seen that the embodiment of the present invention combines sentence strength and token granularity to enable the large language model to understand text at different levels. Among them, sentence strength provides macro guidance, while token granularity provides micro details. The combination of the two can more comprehensively achieve the alignment direction.

[0087] Step 6: Train an automatic inconsistency estimator to assess the degree of inconsistency predicted by token representations.

[0088] To estimate the token representation more directly (the final representation ) to construct a token-level anomaly dataset based on inconsistencies in the generation of mental health treatment recommendations. It is assumed that specific error tokens will result in content that is inconsistent with medical treatment standards. - (For example, an incorrect diagnosis suggestion may mislead the patient). This will result in the generation of a token x that is inaccurate for medical advice. j is marked as 1, while other tokens not related to medical knowledge are marked as 0. For a given sample pair (s + ,s - ), + Zhong and s - Different tokens are considered inconsistent tokens and token x j The inconsistent labels y j The annotations are as follows:

[0089]

[0090] in, Represents the token-level anomaly dataset formed, where o j Indicates inconsistent samples For each token x j Based on Able to train an automatic inconsistency estimator E m , which is a large language model with logistic regression parameterized by θ, is used to assess the inconsistency of the predictions made by the token representations. At inference time, E m Process the token representation o generated at the tth time t , and estimate the potential inconsistency, and get the estimated inconsistency m t :

[0091]

[0092] Step 7: Quantify the uncertainty of the prediction to obtain the quantified token uncertainty.

[0093] The degree of inconsistency of the estimated representation obtained in the previous step does not cover the analysis of prediction uncertainty. Prediction uncertainty has been shown to be associated with the occurrence of inconsistencies in large language models. Intuitively, higher uncertainty indicates a greater potential for inconsistency. Therefore, by obtaining the head W of the large language model LM The predicted probability distribution σ(W LM o t ), and calculate the normalized entropy by the vocabulary size z to get the uncertainty u of the quantized token t , to aid in inconsistency awareness:

[0094] u t =H(σ(W LM o t )) / log(z) (7)

[0095] Step 8: In the intervention phase, based on the estimated level of inconsistency m in step 6 t and the quantized token uncertainty u obtained in step 7 t , for adaptive token intervention.

[0096] In one embodiment of the present invention, m t and u t A weighted sum is performed and a balance factor β is used as the final inconsistency measure. This allows for differentiating intervention strengths for different tokens. The adaptive token intervention process in the embodiment of the present invention can be expressed as:

[0097]

[0098] in, Indicates the generation of output consistent with actual medical knowledge and ethical standards; editorial direction Applied to each prediction with the same strength α; β represents the balancing factor.

[0099] In summary, the present invention achieves token-aware intervention in the reasoning of large language models by combining MIG and MAI. MIG accurately identifies knowledge direction by enhancing detection, while MAI performs adaptive intervention based on token inconsistency. This method breaks through the limitations of traditional sentence-level methods and significantly improves the performance of mental illness screening.

[0100] Specifically, MAI dynamically adjusts the intervention intensity of the text sequence output according to the specific performance and prediction uncertainty of each token, helping the model to identify and correct potential errors or content that does not meet the diagnosis and treatment requirements, and ensuring the reliability of the output results. In order to solve the problem of directional deviation that may occur in sentence-level probes in existing ITI technology, the present invention uses all informative tokens to detect medical information to generate content that is more in line with the diagnosis and treatment needs of mental and psychological diseases. Unlike the simple averaging method in the prior art, the MIG mechanism in the present invention can capture key interactions in the context space between tokens and enhance the ability to understand medical information. MIG uses mutual information to perform global context space probability analysis, emphasizes information sharing between tokens, promotes the contribution of each token in content generation, and enhances feature integration through graph propagation, thereby improving the performance of the model in the application of medical knowledge.

[0101] Based on MIG, MAI further quantifies uncertainty through inconsistency assessment and prediction capabilities, perceives token anomalies, and achieves precise token-level intervention. In practical applications, the deviation of large language models is usually reflected in the prediction of token representation and generated content. MAI perceives token anomalies by combining inconsistency assessment with an estimator trained on a token-level inconsistent dataset and quantifying uncertainty using the model's prediction capabilities. These two methods complement each other and jointly promote the construction of reasonable text sequences, providing reliable support for the intervention of large language models.

[0102] In order to verify the effect of the present invention, the inventor conducted an experiment, and the experimental data set was selected from the depression detection data on the Depression Reddit social media. The experimental results show that after adopting the large language model disease screening method provided by the present invention, the recognition accuracy of depression increased from 64.5% to 67.9%. This result confirms the effectiveness of the present invention in the diagnosis of mental and psychological diseases, especially in dealing with the complex and subtle emotions and symptom expressions of patients, the large language model can capture key information more accurately. Compared with the traditional sentence-level prediction method, the large language model disease screening method can perform a more detailed analysis of vocabulary and phrases, thereby improving the performance of the large language model when processing depression patient data and reducing the occurrence of misjudgment and missed judgment. This improvement in accuracy not only enhances the large language model's ability to understand mental and psychological diseases, but also demonstrates its potential in actual clinical applications. Specifically, the present invention can provide doctors with more reliable diagnostic aids, help promote early screening and personalized treatment, and thus provide patients with more timely and effective help.

[0103] Second embodiment

[0104] Based on the above token-aware large language model disease screening method, the embodiment of the present invention further provides a token-aware large language model disease screening system. Figure 3 As shown, the system includes a processor and a memory. The memory is coupled to the processor and is used to store a program. When the program is executed by the processor, the processor implements the large language model disease screening method in the above embodiment.

[0105] Among them, the processor is used to control the overall operation of the system to complete all or part of the steps of the above-mentioned token-aware large language model disease screening method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support operations in the system, which data may include, for example, instructions for any application or method operating on the system, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, etc.

[0106] In an exemplary embodiment, the system can be implemented by a computer chip or entity, or by a product with a certain function, for executing the above-mentioned token-aware large language model disease screening method, and achieving the same technical effect as the above-mentioned method. A typical embodiment is a computer, a server. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] In another exemplary embodiment, the present invention further provides a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the large language model disease screening method in any of the above embodiments. For example, the computer-readable storage medium may be the above-mentioned memory including program instructions, and the above-mentioned program instructions may be executed by a processor of the system to complete the above-mentioned token-aware large language model disease screening method, and achieve the technical effect consistent with the above-mentioned method.

[0108] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined, and the order of the steps can be changed, all within the protection scope of the present invention.

[0109] The above is a detailed description of the token-aware large language model disease screening method and system provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear the corresponding legal liability.

Claims

1. A token-aware large language model disease screening method, characterized in that The steps include: Construct a dataset containing non-disease samples + and disease samples - The training sample set Each sample s={x1,...,x n } consists of multiple tokens x i Composition, expressed as s = {o1, ..., o n }, where each o i Corresponds to a token x i ; Construct a graph network based on mutual information for each training sample; Perform r rounds of graph propagation based on the graph network. In each round of propagation, update the information of vertices and edges, thereby strengthening the useful information inherent in each token before graph aggregation; Perform graph aggregation on the enhanced tokens to obtain the final representation to construct the final training set; Based on the final training set, the training probes are calculated and determine the editing direction; wherein the editing direction is applied with the same strength to the prediction of the next token by the large language model when generating text.

2. The large language model disease screening method according to claim 1, characterized in that: The graph network middle, represents the token representations generated from all large language models o k：n The abstract vertex set, ε = {e i，j |i, j∈[k, n]} denotes the edge set based on mutual information, which represents the mutual information between tokens; The graph network of round r The formula is as follows: Among them, H(·) represents the univariate Shannon entropy, and H(·,·) represents the joint entropy.

3. The large language model disease screening method according to claim 2, characterized in that: In the rth round of propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the edge weight To weight neighbor vertices information; edge The update is done by calculating the two vertices Mutual information between achieved.

4. The large language model disease screening method according to claim 1, characterized in that The training probe is obtained based on the following formula And determine the editing direction in, Indicates that in the training set The expected value of the cross entropy loss on ; For token x i The final representation of; CE represents the cross entropy loss, y represents the true label; represents the i-th vertex after r rounds of propagation, n represents the total number of nodes, and k represents the number of selected neighbor nodes.

5. The large language model disease screening method according to claim 1, characterized in that The following steps are also included: Train an automatic inconsistency estimator to assess the inconsistency of the predictions from the token representation m t ; By quantifying the prediction uncertainty, we can get the uncertainty u of the quantified token. t ; During the intervention phase, the estimated level of inconsistency m t and the uncertainty of the quantized token u t , for adaptive token intervention.

6. The large language model disease screening method according to claim 5, characterized in that: By obtaining the head W of the large language model LM The predicted probability distribution σ(W LM o t ), and calculate the normalized entropy by the vocabulary size z to obtain the uncertainty u of the quantized token t : at t =H(σ(W LM ·about t )) / log(z) Among them, H(·) represents the univariate Shannon entropy, o t Represents the token representation generated for the tth time.

7. The large language model disease screening method according to claim 6, characterized in that: Will result in the generation of tokens x that are inaccurate medical advice j is marked as 1, while other tokens not related to medical knowledge are marked as 0; for a given sample pair (s + ,s - ), + Zhong and s - Different tokens are considered inconsistent tokens and token x j The inconsistent labels y j The annotations are as follows: in, Represents the token-level anomaly dataset formed, o f Inconsistent samples For each token x f Representation.

8. The large language model disease screening method according to claim 7, characterized in that: At inference time, the automatic inconsistency estimator E m Process the token representation o generated at the tth time t , and estimate the potential inconsistency, and obtain the estimated inconsistency m t :

9. The large language model disease screening method according to claim 8, characterized in that Differentiate intervention intensity for different tokens Satisfy the following formula: in, Indicates the generation of output consistent with actual medical knowledge and ethical standards; editorial direction Applied to each prediction of the next token with the same strength α; β represents the balancing factor.

10. A token-aware large language model disease screening system, characterized by It comprises a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the large language model disease screening method described in any one of claims 1 to 9.

Images