Token-aware based large language model disease screening method and system

Abstract

This invention discloses a token-aware large language model disease screening method and system. This method constructs a training sample set containing both disease and non-disease samples, builds a graph network using mutual information, and performs multiple rounds of graph propagation to enhance the useful information in the tokens. Subsequently, a final representation is obtained through graph aggregation, and a final training set is constructed. Probes are trained to determine editing directions to guide the large language model in generating text that meets medical standards. This invention can effectively improve the performance of large language models in screening for psychiatric and psychological disorders, providing a reliable auxiliary tool for clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to a token-aware-based large language model disease screening method and a corresponding large language model disease screening system, and belongs to the technical field of medical informatics. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have shown great application potential in the medical field, especially in mental health services. For example, large language models can be used to analyze patients' self-reports or chat records during treatment, identify emotional cues and language features, and help doctors quickly screen patients who may have depression and other mental health disorders. In addition, large language models can also interact with mental health patients as virtual diagnosis and treatment models. For example, by analyzing patient input, large language models can generate appropriate responses to help patients sort out their emotions and provide supportive feedback, thereby achieving the effect of assisting diagnosis and treatment to intervene in mental health disorders. However, if the content generated by the large language model deviates from medical standards or treatment standards, it may have a serious impact on the health of patients, such as misleading patients to take inappropriate treatment methods, or causing dangerous behaviors in mental health patients.

[0003] Compared with supervised fine-tuning (SFT) and reinforcement learning based on human feedback (RLHF), inference-time intervention (ITI) technology is a powerful means to solve this problem due to its efficiency and low cost. ITI technology trains interpretable probes on positive and negative samples (such as depression and non-depression patient groups) to identify potential problem directions of large language models when generating medical content, and then intervenes in the internal representation of the large language model in real time during inference to ensure that the generated content meets medical standards and treatment needs. This reduces the dependence on large amounts of training data and computing resources, reduces application costs, and avoids introducing new risks such as large language models generating non-professional recommendations for flattering patients.

[0004] However, existing ITI technology uses coarse-grained sentence-level analysis, ignoring important information about individual tokens, resulting in suboptimal intervention results. Moreover, existing ITI technology is difficult to adapt to individualized and complex scenarios of mental health disorders, and fails to fully address the flexibility of content generation in dynamic scenarios. SUMMARY

[0005] The primary technical problem to be solved by the present application is to provide a token-aware-based large language model disease screening method.

[0006] Another technical problem to be solved by the present application is to provide a token-aware-based large language model disease screening system.

[0007] To achieve the above technical purposes, the present application adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present application, a token-aware-based large language model disease screening method is provided, comprising the following steps:

[0009] A training sample set S containing non-disease samples s + and disease samples s - is constructed. Each sample s={x1,...,x n} is composed of multiple tokens x i , represented as s={o1,...,o n}, wherein each o i corresponds to a token x i ;

[0010] A graph network based on mutual information is constructed for each training sample;

[0011] r rounds of graph propagation are performed based on the graph network, and in each round of propagation, the information of the vertices and edges is updated, thereby strengthening the useful information inherent in each token before graph aggregation;

[0012] The univariate entropy of the token representation is estimated;

[0013] The enhanced tokens are graph aggregated to obtain a final representation to construct a final training set;

[0014] Based on the final training set, a training probe is calculated and an edit direction is determined; wherein the edit direction is applied to the prediction of the next token by the large language model when generating text with the same strength.

[0015] Preferably, in the graph network , the vertex set abstracted from all token representations o k：n generated by the large language model is represented as ε={e i，j |i,j∈[k,n]} represents a mutual information-based edge set representing the mutual information between tokens;

[0016] The rth round of graph network is formulated as follows:

[0017]

[0018] wherein H(·) represents the univariate Shannon entropy, and H(·,·) represents the joint entropy;

[0019] In the rth round of propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the weight of the edge To weight neighbor vertices Information The update is done by calculating the two vertices Mutual information between achieved.

[0020] Preferably, the training probe is obtained based on the following formula: and determine the editing direction

[0021]

[0022] Where, Indicates that in the training set The expected value of the cross entropy loss on ; For token x i The final representation; CE represents the cross entropy loss, y represents the true label; represents the i-th vertex after r rounds of propagation, n represents the total number of nodes, and k represents the number of selected neighbor nodes.

[0023] Preferably, the large language model disease screening method further comprises the following steps:

[0024] Train an automatic inconsistency estimator to evaluate the degree of inconsistency m predicted by the token representation t ;

[0025] By quantifying the prediction uncertainty, we can get the uncertainty u of the quantified token. t ;

[0026] During the intervention phase, the degree of inconsistency m expressed based on the estimated t and the uncertainty of the quantized token u t , for adaptive token intervention.

[0027] Preferably, by obtaining the head W of the large language model LM The predicted probability distribution σ(W LM o t ), and calculate the normalized entropy by the vocabulary size z to obtain the uncertainty u of the quantized token t :u t =H(σ(W LM o t )) / log(z);

[0028] Among them, H(·) represents the single variable Shannon entropy, o tdenotes the token representation generated at the t-th time.

[0029] wherein preferably, tokens x that would lead to the generation of inaccurate medical advice j are labeled 1, while other tokens that are not related to medical knowledge are labeled 0; for a given pair of samples (s + , s - ), tokens in s + that are different from s - are considered as inconsistent tokens, and the inconsistent label y j for token x j is annotated as follows:

[0030]

[0031] wherein, denotes the token-level anomaly dataset formed, o j denotes the inconsistent samples in o j .

[0032] wherein preferably, at inference time, the automatic inconsistency estimator E m processes the token representation o t generated at the t-th time and estimates the potential inconsistency degree, resulting in an estimated inconsistency degree m t :

[0033]

[0034] wherein preferably, different tokens are distinguished by intervention strength satisfies:

[0035]

[0036] wherein, denotes generating an output consistent with actual medical knowledge and ethical standards; the editing direction is applied to the prediction of the next token at each time with the same strength a; and b represents a balance factor.

[0037] According to a second aspect of an embodiment of the present application, there is provided a token-aware large language model disease screening system, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, when the computer program is executed by the processor, the processor implements the large language model disease screening method described above.

[0038] Compared with the prior art, the present application has the following technical characteristics:

[0039] 1) Token-level intervention mechanism: The present application improves the recognition accuracy of large language models by capturing the contextual spatial relationship of all input information, perceiving subtle emotional and symptom expressions in language, and outputting adaptive intervention for each moment sequence. Compared with traditional sentence-level processing, token-level intervention can dynamically adjust the representation of different words and phrases, making large language models more sensitive and accurate when facing complex mental and psychological disease descriptions.

[0040] 2) Mutual information guided token aggregation: Through the mutual information (MI) guided graph propagation mechanism, the present application can establish a spatial relationship of context information sharing between tokens, enhance the understanding of the contribution of different tokens in patient symptom description by large language models, and optimize the internal representation of large language models to help large language models generate content that meets medical knowledge and diagnosis and treatment standards.

[0041] 3) Adaptive token intervention with inconsistency perception: The present application uses adaptive token intervention mechanism (MAI) with inconsistency perception to dynamically adjust the intervention intensity for each moment text sequence output according to the specific performance and prediction uncertainty of each token. This mechanism can help large language models identify and correct potential errors or generated content that does not meet the requirements of mental and psychological disease diagnosis and treatment, ensuring that the final result is more reliable. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The comparison diagram of the token perception-based large language model disease screening method provided by the embodiment of the present application and the prior art is shown in the figure.

[0043] Figure 2 The flowchart of the token perception-based large language model disease screening method in the first embodiment of the present application is shown in the figure.

[0044] Figure 3 The structure diagram of the token perception-based large language model disease screening system in the second embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0045] The technical content of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0046] The technical concept in the embodiment of the present application is to analyze the association relationship between each token in the entire sentence context space and its contribution in generating medical content, establish an accurate and unbiased text sequence output, and significantly improve the performance of large language models in mental and psychological disease diagnosis and treatment. Furthermore, based on the inconsistency perception of each token, the intervention intensity is dynamically adjusted to realize adaptive token intervention, ensuring that the generated content of large language models meets the clinical standards and medical needs.

[0047] Specifically, the large language model disease screening method provided by the embodiments of the present application introduces mutual information guided token aggregation (referred to as MIG) and inconsistent perception adaptive token intervention (referred to as MAI) to eliminate the adverse effects of direction deviation and the same intervention intensity, respectively. Specifically, MIG captures and utilizes the key interactions between tokens (i.e., words or phrases in the text) in the context space by constructing a graph network based on mutual information. In MIG, each token is regarded as a vertex in the graph, and the mutual information between tokens is regarded as an edge, representing the degree of information sharing between them. Through graph propagation, the large language model can strengthen the useful information inherent in each token before aggregation, thereby optimizing the internal representation of the large language model and helping the large language model to better generate content that conforms to medical knowledge and diagnosis and treatment standards. MAI dynamically adjusts the intervention intensity for the output of the text sequence at each moment according to the specific performance and prediction uncertainty of each token. MAI estimates the degree of inconsistency represented by the token through the training of an automatic inconsistency estimator, and combines the uncertainty predicted by the large language model to perceive the abnormality of the token. In the intervention phase, based on the estimated inconsistency degree and the quantified token uncertainty, adaptive token intervention is performed to identify and correct potential errors or generated content that does not meet the requirements of mental and psychological disease diagnosis and treatment, ensuring that the final output is more reliable.

[0048] The two mechanisms of MIG and MAI work together to enable the large language model disease screening method provided by the embodiments of the present application to more accurately capture key information and improve the recognition accuracy and reliability of the large language model in mental and psychological disease diagnosis and treatment. Through MIG, the large language model can strengthen the understanding of medical information in the training sample; and MAI adjusts the intervention intensity through inconsistency perception to achieve precise token-level intervention.

[0049] First embodiment

[0050] Next, taking the screening of depression patients based on blog, WeChat and other text content as an example, the specific implementation process of the large language model disease screening method provided by the first embodiment of the present application is described.

[0051] When the large language model based on decoding generates mental and psychological disease related content, it may pass through the head W LM Generating content that does not conform to medical knowledge and diagnosis and treatment standards, existing research has shown that there is an interpretable correlation between these inaccurate content and the internal representation of the large language model. Therefore, the ITI technology can intervene sparsely on the internal representation of the large language model to guide it to generate content that conforms more to medical needs. In the context of mental and psychological disease screening, ensuring that the content generated by the large language model is consistent with the actual medical knowledge and ethical standards is crucial for patient safety.

[0052] Existing ITI techniques are implemented based on sentence-level analysis, including the following steps:

[0053] S1: Use probing techniques to distinguish between content in the internal representation of large language models that matches medical knowledge and content that does not meet medical requirements.

[0054] This step includes the following sub-steps:

[0055] S11) Construct a sample set containing non-depression samples s + and depression samples s - Each sample s = {x1,..., x n} is a sentence composed of multiple tokens x i , which represents the hth head of the multi-head self-attention (MHSA) from the lth layer decoder, denoted as where each corresponds to a token x i ;

[0056] S12) Use the representation of the last token as a representative of the entire sample s l，h and form a dataset y represents the label of the sample (y is negative if the sample is depression; is positive if the sample is non-depression);

[0057] S13) Train the probe P l，h to distinguish between mental health directions. Find the parameters d l，h that minimize the cross-entropy loss by:

[0058]

[0059] In equation (1), d l，h represents the parameters learned by the probe P l，h , which is the editing direction of the content generated by the large language model. y labels each sample as depression or non-depression, and CE represents the cross-entropy loss, which measures the difference between the probe's prediction and the true label y. represents the expected value of the cross-entropy loss on the dataset .

[0060] S2: During training, evaluate the accuracy of different heads.

[0061] ​The accuracy rate refers to the proportion of probes that correctly predict the labels of samples. On the validation set, the large language model tests the accuracy rates of different heads, and then selects the sparse head with the highest accuracy rate on the validation set for intervention. These heads perform best on the validation set, meaning they can most accurately distinguish between depressed and non-depressed samples, and can be used for further analysis or intervention. The selection of sparse heads also helps to reduce the complexity of the large language model, improving the efficiency and interpretability of the large language model.

[0062] S3: In the intervention phase, the editing direction d l，h is applied to each prediction of the next token made by the large language model when generating text with the same strength a, so as to bias the representation towards the mental health direction and guide the large language model to generate output consistent with actual medical knowledge and ethical standards:

[0063]

[0064] wherein, represents the token representation generated by the intervened large language model. Analysis shows that the deviated direction d l ，h and the same strength a are determined based on sentence-level analysis, which weakens the mental health output after intervention.

[0065] Unlike existing ITI techniques, see Figure 1 and Figure 2 , the large language model disease screening method provided by the embodiments of the present application includes a training process and a testing process. In the training process, after the training data is input into the large language model, the representation after each layer of multi-head self-attention module (MHSA) is extracted, input into the MIG for probe training for interpretability, and the probe and the editing direction are obtained. In the testing process, after the test data is input into the large language model, the representation after the multi-head self-attention module (MHSA) that needs to be intervened is obtained; the above representation is edited using the MAI to obtain the edited representation; the edited representation is input into the subsequent feedforward layer (FFN). Finally, the prediction result is obtained through the large language model.

[0066] Specifically, the large language model disease screening method provided by the embodiments of the present application includes the following steps:

[0067] Step 1: Construct a training sample set containing non-disease (e.g. non-depression) samples s + and disease (e.g. depression) samples s - Each sample s = {x1,..., x n} is composed of multiple tokens x i , represented as s = {o1,..., o n}, where each o​i Corresponds to a token x i ;

[0068] Step 2: Build a graph network based on mutual information (MI) for each training sample

[0069] Specifically, graph networks middle, represents the token representations generated from all large language models o k：n The abstract vertex set, ε={e i，j |i, j∈[k, n]} represents the edge set based on mutual information, which represents the mutual information between tokens. Initial graph In each vertex Represents the corresponding token representation o i , each edge express and The mutual information between them.

[0070] Step 3: Perform r rounds of graph propagation based on the graph network. In each round of propagation, the vertex and edge information is updated to strengthen the useful information inherent in each token before the graph is aggregated.

[0071] The graph network of round r The formula is as follows:

[0072]

[0073] In formula (3), H(·) represents the univariate Shannon entropy, and H(·,·) represents the joint entropy.

[0074] In the rth round of propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the weight of the edge To weight neighbor vertices Information. The update is done by calculating the two vertices The mutual information between The univariate Shannon entropy of two vertices and joint entropy Calculated.

[0075] Because in each round the update of the vertex is achieved by aggregating the information of its neighbor vertices, the weight of the edge is used To weight neighbor vertices information inherent in each token. Therefore, neighbor tokens with higher mutual information with the current token will have a greater impact, thereby reinforcing the useful information of the current token. That is, the representation of tokens is adjusted using mutual information between tokens, so that the representation of each token is more rich and useful.

[0076] Moreover, Shannon entropy is a measure of uncertainty of a random variable, and the entropy of a single variable can be used to quantify the amount of information carried by a single token. The higher the entropy, the greater the uncertainty of the token and the more information it carries. In text processing, this means that which tokens have strong semantic associations can be identified. Therefore, by calculating the mutual information between tokens, the token combination that contributes most to the meaning of the sentence can be found; by identifying and reinforcing tokens with high mutual information, large language models can then pay more attention to information that is crucial to understanding sentences or documents. This helps large language models filter out noise or irrelevant information when processing text, thereby improving the quality and relevance of information. Through mutual information-guided feature selection and feature weighting, the representation of tokens can be optimized. This allows large language models to pay more attention to features that have a direct impact on the prediction task during the training process. As can be seen, in the embodiments of the present invention, mutual information identifies and quantifies the degree of information sharing between tokens, and updates tokens based on the degree of sharing, thereby improving the quality and relevance of information.

[0077] Moreover, in adaptive token intervention (e.g., MAI), mutual information can be used to dynamically adjust the weights of tokens to reflect their importance in a particular context. This approach allows large language models to flexibly adjust their focus when facing different tasks or datasets.

[0078] Here, the single-variable entropy of the token representation is estimated When the single-variable entropy of the token representation is estimated

[0079] Step four: graph aggregation on the enhanced tokens to obtain the final representation which contains more rich and discriminative information, and builds the final training set

[0080] Step five: based on the final training set The training probe is calculated based on the following formula and determine the editing direction

[0081]

[0082] In the above formula, represents the expected value of the cross-entropy loss on the training set ; finally represents CE represents the cross-entropy loss, which is used to measure the difference between the prediction of the probe and the true label y; ; represents the i-th vertex after r rounds of propagation, n represents the total number of nodes, and k represents the number of selected neighbor nodes.

[0083] As can be seen from formula (4), it is similar to formula (1) in the existing ITI technology, but the training set used in the embodiment of the present application is different, and the final representation used is optimized.

[0084] Generally in graph networks, node features are propagated and aggregated through graph structure to update the representation of each node; by minimizing the cross-entropy loss, the large language model learns how to predict the label of the node according to the local graph structure information of the node.

[0085] As is known to all, sentence strength and token granularity are two different methods for understanding and generating text in natural language processing. The embodiment of the present application effectively reduces the problem of direction deviation by combining the advantages of sentence strength and token granularity analysis, and realizes more accurate direction exploration. Specifically, first, by considering the strength of the entire sentence, the large language model can capture the overall sentiment tendency or theme of the sentence, which helps to understand the text at a macro level and ensures that the output of the large language model is consistent with the overall intention of the sentence. As a whole guiding signal, sentence strength helps the large language model maintain the correct alignment direction when processing text, especially in sentiment analysis tasks. Second, by using token granularity analysis, the large language model can understand the text at a finer level, i.e. analyzing the contribution of each word to the overall meaning of the sentence word by word, which helps to identify and strengthen those words that have a significant impact on the sentiment of the sentence. Finally, by mutual information enhancement, the large language model can more accurately determine the contribution of each token to the direction of the sentence, thereby realizing alignment direction at a micro level. This not only helps the large language model to more accurately reflect the information of each word when generating text, but also improves the relevance and accuracy of the text. By comprehensively analyzing these three aspects, the embodiment of the present application can comprehensively understand the text at different levels, and realize more accurate text processing and generation.

[0086] ​Therefore, the embodiment of the present application combines sentence strength and token granularity, so that the large language model can understand the text at different levels. Among them, the sentence strength provides macro guidance, and the token granularity provides micro details, and the combination of the two can more comprehensively realize the alignment direction.

[0087] Step six: training an automatic inconsistency estimator for evaluating the inconsistency degree predicted by the token representation.

[0088] In order to more directly estimate the inconsistency of the token representation (final representation ) in the generation of mental health diagnosis and treatment recommendations, a token-level abnormal data set is constructed. It is assumed that a specific error token will lead to content s - that does not conform to medical diagnosis and treatment standards (for example, a wrong diagnosis recommendation may mislead the patient). The token x j that leads to the generation of inaccurate medical recommendations is marked as 1, and other tokens unrelated to medical knowledge are marked as 0. For a given sample pair (s + , s - ), the tokens different from s + in s - are regarded as inconsistent tokens, and the inconsistency label y j of the token x j is annotated as follows:

[0089]

[0090] Wherein, represents the token-level abnormal data set formed, wherein o j represents the representation of each token x j in the inconsistent sample . Based on , an automatic inconsistency estimator E m can be trained, which is a logistic regression large language model parameterized as θ, for evaluating the inconsistency degree predicted by the token representation. In inference, E m processes the token representation o t generated at the t-th time, and estimates the potential inconsistency degree to obtain the estimated inconsistency degree m t of the token representation:

[0091]

[0092] Step seven: obtaining quantified token uncertainty through prediction uncertainty quantification.

[0093] The inconsistency degree of the estimated representation obtained in the previous step cannot cover the analysis of prediction uncertainty. However, prediction uncertainty has been proven to be related to the occurrence of inconsistency of large language models. Intuitively, higher uncertainty represents greater potential inconsistency. Therefore, by obtaining the probability distribution σ(W LM predicted by the head of the large language model W LM ·o t , and calculating the normalized entropy by the size of the vocabulary z, the quantified token uncertainty u t is obtained to assist in inconsistency perception:

[0094] u t = H(σ(W LM ·o t )) / log(z) (7)

[0095] Step eight: In the intervention stage, based on the inconsistency degree m t of the estimated representation in step six and the quantified token uncertainty u t obtained in step seven, adaptive token intervention is performed.

[0096] In an embodiment of the present application, m t and u t are directly weighted and summed, and a balance factor β is used as the final inconsistency measure. This can distinguish the intervention intensity for different tokens The adaptive token intervention process in the embodiment of the present application can be represented as:

[0097]

[0098] wherein, represents generating output consistent with actual medical knowledge and ethical standards; the editing direction is applied to each prediction with the same strength α; β represents the balance factor.

[0099] In summary, the present application realizes token-aware intervention of large language models during reasoning by combining MIG and MAI technologies. MIG accurately identifies the knowledge direction through enhanced detection, while MAI performs adaptive intervention based on token inconsistency. This method breaks through the limitations of traditional sentence-level methods and significantly improves the performance of mental illness screening.

[0100] Specifically, MAI dynamically adjusts the intervention intensity of the text sequence output according to the specific performance and predicted uncertainty of each token, helping the model to identify and correct potential errors or content that does not meet the requirements of diagnosis and treatment, and ensuring the reliability of the output results. In order to solve the problem of direction deviation that may occur in the existing ITI technology, the present application uses all information tokens to detect medical information to generate content that is more in line with the needs of mental illness diagnosis and treatment. Unlike the simple average method in the prior art, the MIG mechanism in the present application can capture the key interaction in the context space between tokens, and strengthen the medical information understanding ability. MIG performs global context space probability analysis by applying mutual information, emphasizes information sharing between tokens, promotes the contribution of each token in content generation, and enhances feature integration through graph propagation to improve the performance of the model in medical knowledge application.

[0101] Based on MIG, MAI further quantifies uncertainty through inconsistency evaluation and prediction ability to perceive the abnormality of tokens and achieve precise token-level intervention. In practical applications, the bias of large language models is usually reflected in the related predictions of token representation and generated content. MAI perceives the abnormality of tokens by combining the estimator trained based on the token-level inconsistency dataset for inconsistency evaluation and quantifying uncertainty using model prediction ability. These two methods complement each other and work together to build reasonable text sequences and provide reliable support for the intervention of large language models.

[0102] In order to verify the effect of the present application, the inventors conducted experiments, and the experimental data set was selected from depression detection data on Depression Reddit social media. The experimental results show that after using the large language model disease screening method provided by the present application, the recognition accuracy of depression has increased from 64.5% to 67.9%. This result confirms the effectiveness of the present application in the diagnosis of mental and psychological diseases, especially in dealing with the complex and subtle emotional and symptom expressions of patients, the large language model can more accurately capture key information. Compared with traditional sentence-level prediction methods, the large language model disease screening method can analyze words and phrases more carefully, thereby improving the performance of the large language model in processing depression patient data and reducing the occurrence of false positives and false negatives. This improvement in accuracy not only enhances the understanding ability of the large language model for mental and psychological diseases, but also demonstrates its potential in actual clinical applications. Specifically, the present application can provide doctors with a more reliable diagnostic aid tool to help promote early screening and personalized treatment, thereby providing more timely and effective help for patients.

[0103] Second embodiment

[0104] On the basis of the above-mentioned token-aware large language model disease screening method, the embodiment of the present application further provides a token-aware large language model disease screening system. As shown in the above-mentioned token-aware large language model disease screening method, the system comprises a processor and a memory. The memory is coupled to the processor and is used to store programs. When the programs are executed by the processor, the processor implements the large language model disease screening method in the above-mentioned embodiment. Figure 3

[0105] The processor is used to control the overall operation of the system to complete all or part of the steps of the above-mentioned token-aware large language model disease screening method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable logic gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support the operation of the system. These data may, for example, include instructions for operating any application or method on the system, as well as application-related data. The memory can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, etc.

[0106] In one exemplary embodiment, the system can be specifically realized by a computer chip or an entity, or by a product with certain functions, for executing the above-mentioned token-aware large language model disease screening method and achieving the technical effects consistent with the above-mentioned method. A typical embodiment is a computer, a server. Specifically, the computer may, for example, be a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] In another exemplary embodiment, the present application also provides a computer-readable storage medium comprising program instructions, which are executed by a processor to implement the steps of the large language model disease screening method in any one of the above-mentioned embodiments. For example, the computer-readable storage medium may, for example, be the above-mentioned memory comprising program instructions, which can be executed by the processor of the system to complete the above-mentioned token-aware large language model disease screening method and achieve the technical effects consistent with the above-mentioned method.

[0108] ​It should be noted that the above embodiments are only illustrative. The technical solutions of various embodiments can be combined, and the order of various steps can be changed, all within the protection scope of the present application.

[0109] The token-aware-based large language model disease screening method and system provided by the present application are described in detail above. Any obvious modification made by a person skilled in the art without departing from the essential content of the present application will constitute an infringement of the patent right of the present application and will bear the corresponding legal responsibility.

Claims

1. A large language model disease screening method based on token perception, characterized by The steps include: Construct a non-disease sample and disease samples The training sample set ; Each sample Multiple tokens Composition, expressed as , where each Corresponds to a token ; Build a graph network based on mutual information for each training sample; Based on graph network Round graph propagation, in which vertex and edge information is updated in each round of propagation, thereby strengthening the useful information inherent in each token before the graph is aggregated; Perform graph aggregation on the enhanced tokens to obtain the final representation to construct the final training set; Based on the final training set, the training probe is calculated , and determine the editing direction; wherein the editing direction is applied with the same strength as the prediction of the next token by the large language model when generating text; Train an automatic inconsistency estimator to assess the degree of inconsistency between predictions from token representations ; By quantifying the prediction uncertainty, we can get the uncertainty of the token ; During the intervention phase, the degree of inconsistency based on the estimates and the uncertainty of the quantified token , conduct adaptive token intervention, in, The graph network middle, Represents token representations generated from all large language models The abstract vertex set, represents the mutual information-based edge set, which represents the mutual information between tokens; The first Wheel Graph Network The formula is as follows: in, represents the univariate Shannon entropy, represents the joint entropy; In the In round propagation, the vertex The update is done by aggregating its neighbor vertices The information is realized by using the weight of the edge To weight neighbor vertices Information The update is done by calculating the two vertices Mutual information between achieved; By obtaining the head of the large language model Predicted probability distribution , and by vocabulary size Calculate the normalized entropy to obtain the uncertainty of the quantified token : in, represents the univariate Shannon entropy, Indicates the The token representation generated by this time; At inference time, the automatic inconsistency estimator Processing The generated token represents , and estimate the potential inconsistency, and get the estimated inconsistency : 。 2. The large language model disease screening method according to claim 1, characterized in that The training probe is obtained based on the following formula , and determine the editing direction : in, Indicates that in the training set The expected value of the cross entropy loss on ; For token The final expression of represents the cross entropy loss, represents the true label; Indicates passing The first round of transmission vertex, Indicates the total number of nodes, Indicates the number of selected neighbor nodes.

3. The large language model disease screening method according to claim 1, wherein: Tokens that would result in inaccurate medical advice is marked as 1, while other tokens not related to medical knowledge are marked as 0; for a given sample pair ,Will Zhongyu Different tokens are considered inconsistent tokens and the tokens Inconsistent labels The markings are as follows: in, Represents the token-level anomaly dataset formed, Inconsistent samples Each token in Representation.

4. The large language model disease screening method according to claim 1, characterized in that Differentiate intervention intensity for different tokens , satisfying the following formula: ； in, Indicates the generation of output consistent with practical medical knowledge and ethical standards; editorial direction With the same intensity Applied to each prediction of the next token; Represents the balance factor.

5. A large language model disease screening system based on token perception, characterized by It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the large language model disease screening method described in any one of claims 1 to 4.

Images