LLM report interpretation method and system based on synthetic data

Abstract

The application relates to a synthetic data-based LLM report interpretation method and system, and belongs to the technical field of natural language processing and large language model application, in particular to a synthetic data-based LLM report interpretation method and system. The application aims to solve the problems of insufficient accuracy, universality and user adaptability in the report interpretation of the prior art. The application first generates professional question and answer pairs through synthetic data to enrich the knowledge base of the system; secondly, the cache mechanism is used to store the answered question and answer pairs, so that repeated calculation is avoided, and the query response speed is improved; finally, the context-enhanced generation model is used to generate answers that are more in line with the characteristics of the report field when processing user queries, so that the accuracy and professionalism of the interpretation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and large language model (LLM) application, in particular to a LLM report interpretation method and system based on synthetic data. BACKGROUND

[0002] With the rapid development of genomics, medical testing and clinical reports, the content of related test reports becomes increasingly complex. The content of the report often involves a large number of professional terms, complex structures and multi-dimensional results, such as chart data, diagnosis results and suggestions, and the structures are different. When facing these reports, the traditional manual interpretation method not only takes a long time, but also is prone to errors due to human negligence or understanding bias. In addition, some traditional report interpretation techniques usually rely on artificial pre-defined rules or templates. These methods are often simple in interpreting reports and difficult to cover all potential complex information in the report. Especially for high-dimensional data, chart analysis and their relevance in genetic testing reports and medical testing reports, traditional methods cannot deeply understand and extract relevant information. Therefore, it is particularly important to develop an intelligent interpretation method that can automatically process such complex reports and adapt to different field requirements. How to accurately and quickly extract effective information from these reports has become a major challenge in current technology.

[0003] Although current large language models (LLM) perform well in text generation and semantic understanding, they still face many challenges when dealing with highly specialized reports. These challenges mainly include the lack of domain-specific knowledge, the insufficient accuracy of report interpretation, and the limited accuracy of model-generated answers when facing ambiguous or complex queries. In order to improve the accuracy and efficiency of report interpretation, in recent years, retrieval-augmented generation (RAG) technology has been widely applied in text generation tasks. RAG technology combines information retrieval and generation models, and by retrieving relevant context information and combining it with the generation model, it can significantly improve the accuracy of the generated results. However, in practical applications, RAG technology still has certain limitations when it comes to report interpretation, especially in terms of improving report interpretation efficiency and answer accuracy. Traditional RAG methods do not fully utilize domain-specific context information and caching mechanisms, resulting in the need for a large amount of computing resources during querying and insufficient understanding of the professional nature of the report.

[0004] Existing report interpretation techniques have problems of insufficient accuracy and incomplete information coverage when dealing with professional reports in various fields, especially when the data source is complex or there are ambiguous queries, traditional methods are difficult to meet the precise needs of users. Although large language models (LLM) have strong generation capabilities, they are limited by pre-training data and context association capabilities, and their performance may not be sufficient when facing new fields or specific reports.

[0005] To this end, the present application proposes an intelligent report interpretation method based on RAG technology combined with synthetic data and question and answer caching mechanism to better cope with this challenge. SUMMARY

[0006] The purpose of the present application is to solve the problems of insufficient accuracy, comprehensiveness and user adaptability in report interpretation in the prior art, and to propose an LLM report interpretation method and system based on synthetic data.

[0007] The specific process of an LLM report interpretation method based on synthetic data is as follows:

[0008] Step S0, using a large language model to generate a synthetic database;

[0009] Step S1, processing each question and answer data in the synthetic database obtained in step S0 and storing it in a question and answer cache;

[0010] Step S2, receiving a user uploaded PDF file, using PyMuPDF to parse the PDF file to obtain text data;

[0011] Step S3, segmenting the text data obtained in step S2 to obtain each segment of text data; processing each segment of text data to generate a vector; indexing the vector and storing it in a temporary vector database;

[0012] Step S4, the user inputs a query text, and obtains the user question Q through a large language model;

[0013] Step S5, searching for the most similar (Q q +A q ) in the question and answer cache established in step S1 through the user question Q;

[0014] Obtain the qth question and answer pair data (Q q ,A q ) in the synthetic database based on (Q q +A q ), let the qth question and answer pair data C q =(Q q ,A q );

[0015] Where (Q q +A q ) represents the qth new text in the new text set; (Q q ,A q ) represents the qth question and answer pair data in the synthetic database, Q q represents the qth question and answer pair data (Q q ,A qA q represents the qth question and answer pair data (Q q ,A q ) in the synthetic database; 1≤q≤M.

[0016] Step S6, write instruction prompt words, use large language model to judge whether the most similar question and answer pair data C q =(Q q ,A q ) can directly answer, if yes, proceed to step S7, otherwise proceed to step S8;

[0017] Step S7, the answer A q in the most similar question and answer pair data C q =(Q q ,A q ) is the answer A q of the user query text, and the current question and answer is ended;

[0018] Step S8, the user question Q and the temporary vector database of step S3 obtain corresponding text block data C t ;

[0019] Step S9, use the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q of step S4, and the large language model obtains a response A;

[0020] Step S10,

[0021] The user judges whether the response A obtained by the large language model in step S9 is reasonable for the user question Q;

[0022] If yes, the reply is qualified;

[0023] If not, the reply is unqualified, and the user provides a reasonable reply, which is added to the question and answer cache library of step S1 to form new synthetic data.

[0024] A LLM report interpretation system based on synthetic data comprises:

[0025] A data synthesis module, a document parsing module, a cache library retrieval module, a user query module, a document retrieval module, a question and answer determination module, a response generation module, and a response determination module.

[0026] The beneficial effects of the present application are:

[0027] The purpose of the present application is to provide a method and system for enhancing the performance of LLM report interpretation based on synthetic data, to solve the problems of insufficient accuracy, comprehensiveness and user adaptability in report interpretation in the prior art. The purpose of the present application is to provide an intelligent report interpretation method based on RAG technology, especially in the fields of genetic testing reports and medical testing reports, aiming to improve the accuracy and efficiency of report interpretation. By introducing a cache mechanism and combining professional structured data synthesis question and answer pairs, the accuracy and response speed of report interpretation can be significantly improved. Specifically, the present application aims to achieve this goal in the following ways: first, generate professional question and answer pairs through synthetic data to enrich the system's knowledge base; second, store the answered question and answer pairs through the cache mechanism to avoid repeated calculations and improve query response speed; finally, use the context-enhanced generation model to generate answers that are more consistent with the characteristics of the report field when processing user queries, thereby improving the accuracy and professionalism of interpretation.

[0028] By combining the question and answer cache mechanism, synthetic data and dynamic context learning ability, the report interpretation ability of large language models can be effectively improved. The present application proposes a method based on synthetic data, which generates pseudo question and answer pairs from structured databases and texts, and uses sliding window chunking, vectorization indexing and query rewriting techniques to realize the complete process from user query to accurate report interpretation. This method aims to improve the accuracy, real-time performance and field adaptability of report interpretation, and provides efficient and intelligent services for users.

[0029] With the help of context examples, the answers generated by the system will be more consistent with the professional field style of the report, improving the accuracy of interpretation. This feature is particularly suitable for highly specialized fields such as genetic testing reports and medical testing reports, which can help users quickly and accurately understand the key content of the report; in addition, the method of the present application can continuously optimize the quality of answers according to user feedback, so that the system has the ability of dynamic updating and optimization, thereby continuously improving the service quality and adapting to user needs in the long term; through this self-enhancing mechanism, the system not only improves the response speed and accuracy of single query, but also forms a more accurate and efficient interpretation platform in the process of accumulating question and answer pairs, adapting to the needs of more professional fields. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flowchart of a method for enhancing the performance of LLM report interpretation based on synthetic data of the present application;

[0031] Figure 2 A flowchart of the data synthesis of the question and answer cache library of the present application;

[0032] Figure 3 A question and answer data vectorization and case indexing diagram of the present application;

[0033] Figure 4 Indexing the graph for document vectorization of the present application. DETAILED DESCRIPTION

[0034] Specific implementation one: combined Figure 1 , Figure 2 This embodiment explains the specific process of a LLM report interpretation method based on synthetic data:

[0035] Step S0, using a large language model to generate a synthetic database;

[0036] Step S1, as shown in Figure 3 , process each question and answer data in the synthetic database obtained in step S0, and store it in the question and answer cache;

[0037] Step S2, receiving a user uploaded PDF file, using PyMuPDF to parse the PDF file to obtain text data;

[0038] Step S3, as shown in Figure 4 , segment the text data obtained in step S2 to obtain each segment of text data; process each segment of text data to generate a vector; index the vector and store it in a temporary vector database;

[0039] Step S4, the user inputs the query text, and obtains the user question Q through the large language model;

[0040] Step S5, search the most similar (Q q +A q ) (cosine similarity) in the question and answer cache established in step S1 for the user question Q;

[0041] Obtain the qth question and answer pair data (Q q +A q ) in the synthetic database based on (Q q +A q ), let the qth question and answer pair data C q =(Q q ,A q );

[0042] Where (Q q +A q ) represents the qth new text in the new text set; (Q q ,A q ) represents the qth question and answer pair data in the synthetic database, Q q represents the question in the qth question and answer pair data (Q q ,A q ) in the synthetic database, and A qrepresents the answer in the qth question and answer pair data (Q q ,A q ) in the synthesis database; 1≤q≤M.

[0043] Step S6, write instruction prompt word, "{retrieved most similar question and answer pair data C q} can directly answer the user question {user query}?", use large language model to determine whether the most similar question and answer pair data C q =(Q q ,A q ) can directly answer, if yes, proceed to step S7, otherwise proceed to step S8;

[0044] Step S7, the answer A q in the most similar question and answer pair data C q =(Q q ,A q ) is the answer A q to the user query text, and the current question and answer is ended;

[0045] Step S8, the user question Q and the temporary vector database in step S3 get corresponding text block data C t (C t contains K D j );

[0046] Step S9, use the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q in step S4, and the large language model obtains the response A;

[0047] Step S10,

[0048] The user judges whether the response A obtained by the large language model in step S9 is reasonable for the user question Q;

[0049] If yes, the reply is qualified;

[0050] If not, the reply is unqualified, and the user provides a reasonable reply, which is added to the question and answer cache in step S1 to form new synthesis data.

[0051] Specific implementation method two: the difference between this implementation method and the specific implementation method one is that the large language model is used in step S0 to generate the synthesis database,

[0052] The specific process is as follows:

[0053] Step S0-1,

[0054] From the existing medical structured database (including but not limited to open source data extracted from OncoKB tumor database,

[0055] Collecting structured data in structured storage of structured data stored in MySQL, excel or JSON format) in OncoKB tumor database has structured data (table) and unstructured data, structured data is directly collected, unstructured data is converted into structured data, examples of final data formats used, structured attributes and their attribute values, such as {gene: ABL1, gene description: a tyrosine kinase, is frequently altered by chromosomal translocations in leukemia.}, and {gene: ABL1, variation: BCR-ABL1 Fusion, cancer type: B-Lymphoblastic Leukemia / Lymphoma, drug used: Dasatinib};

[0056] Step S0-2, texturize structured data in structured database according to clauses (for example, structured data in step S0-1 is directly texturized into: structured data (table) one line per clause, and "gene: ABL1, gene description: a tyrosine kinase, is frequently altered by chromosomal translocations in leukemia." is one clause);

[0057] Step S0-3, based on each piece of data texturized in S0-2, write instruction prompt words, use large language model to synthesize question and answer data, and obtain preliminary synthesized question and answer data;

[0058] The specific process is as follows:

[0059] Design and write instruction prompt words, "I hope you act as a question and answer data construction expert, please you only according to the data I provide to construct five pieces of question and answer data, and the generated sentences should meet the real dialogue scene; the following is the data to be constructed gene: {data in database}, gene description: {data in database}",

[0060] Use the instruction prompt words to synthesize question and answer data, and obtain the preliminary question and answer pair;

[0061] {Data in database} is the data texturized in S0-2 (such as "gene: ABL1, gene description: a tyrosine kinase, is frequently altered by chromosomal translocations in leukemia.");

[0062] Step S0-4: Write instruction prompts, such as "Please judge whether the synthesized question-and-answer data is reasonable". Use a large language model to perform quality screening on the initially synthesized question-and-answer data. If it is unqualified, repeat step S0-3 until it is qualified or the number of iterations reaches the upper limit, and obtain the synthesized database {(Q1,A1),(Q2,A2),…,(Q i A i ),…,(Q M A M )};

[0063] in,

[0064] (Q1,A1) represents the first question-answer pair in the synthetic database, where Q1 represents the question in the first question-answer pair (Q1,A1) in the synthetic database, and A1 represents the answer in the first question-answer pair (Q1,A1) in the synthetic database.

[0065] (Q2,A2) represents the second question-answer pair in the synthetic database, where Q2 represents the question in the second question-answer pair (Q2,A2) in the synthetic database, and A2 represents the answer in the second question-answer pair (Q2,A2) in the synthetic database.

[0066] (Q i A i () represents the i-th question-answer pair in the synthetic database, and Qi represents the i-th question-answer pair in the synthetic database (Q). i A i The question in ) is A i Represents the i-th question-answer pair data (Q) in the synthetic database. i A i The answer in )

[0067] (Q M A M ) represents the Mth question-answer pair in the synthetic database, Q M This represents the Mth question-answer pair data (Q) in the synthetic database. M A M The question in ) is A M This represents the Mth question-answer pair data (Q) in the synthetic database. M A M The answer in )

[0068] The other steps and parameters are the same as in Specific Implementation Method 1.

[0069] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that, in step S1, as follows... Figure 3 As shown, each question-and-answer data in the synthetic database obtained in step S0 is processed and stored in the question-and-answer cache library;

[0070] The specific process is as follows:

[0071] Step S1-1: Combine the database {(Q1,A1),…,(Q i A i ),…,(Q M A M Formally, {(Q1,A1),…(Q n A n Each question-answer pair in {(Q1+A1),...}) is concatenated into a text, resulting in a new text set {(Q1+A1),...,(Q1+A1),...}) i +A i ),…,(Q M +A M )};

[0072] in,

[0073] (Q1+A1) represents the first new text in the new text set, (Q i +A i ) represents the i-th new text in the new text set, (Q M +A M ) represents the Mth new text in the new text set;

[0074] Step S1-2: Encode each new text in the new text set obtained in step S1-1 into a vector using a vector encoder (using BGE1.5-large-zh);

[0075] Step S1-3: Index the vectors obtained in step S1-2 and store them in the vector database as a question-and-answer cache (using Chroma).

[0076] Other steps and parameters are the same as in specific implementation method one or two.

[0077] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that, in step S3, as follows... Figure 4 As shown, the text data obtained in step S2 is segmented to obtain each segment of text data; each segment of text data is processed to generate a vector; an index is created on the vectors and stored in a temporary vector database; the specific process is as follows:

[0078] Step S3-1: Segment the text data obtained in step S2 to obtain each segment of text data; the specific process is as follows:

[0079] Step S3-2: Use a vector encoder (using bge1.5-large-zh) to encode each piece of text data obtained in step S3-1 into a vector;

[0080] Step S3-3, index the vector of step S3-2 and store it into a temporary vector database (using Chroma).

[0081] The other steps and parameters are the same as one of the first to third embodiments.

[0082] Embodiment five: different from one of the first to fourth embodiments, the step S3-1, the text data obtained in step S2 is segmented to obtain each segment of text data; the specific process is as follows:

[0083] We use the method of sliding window, such as setting the window size to 512 characters, and setting each step to 256;

[0084] The text data obtained in step S2 is divided into N segments by the sliding window method, each segment has a size of 512 characters, and the characters between adjacent segments overlap by 256, obtaining a text data set D = {D1, D2, …, DN}; j ,…,D N};

[0085] Each segment of text data is represented as D j , j = 1, 2, …, N;

[0086] Wherein, D1 represents the first segment of text data, D2 represents the second segment of text data, Dj represents the jth segment of text data, and DN represents the Nth segment of text data, j = 1, 2, …, N. j N

[0087] The other steps and parameters are the same as one of the first to fourth embodiments.

[0088] Embodiment six: different from one of the first to fifth embodiments, in step S4, the user inputs the query text, and obtains the user question Q through the large language model; the specific process is as follows:

[0089] Step S4-1, re regular expression processing is performed on the query text input by the user (including but not limited to eliminating disordered punctuation marks), to obtain the re regular expression processed user query text;

[0090] Step S4-2, write instruction prompt words, “please standardize the re regular expression processed user query text, so that the re regular expression processed user query text conforms to the problem in the genetic detection field”, use the large language model to standardize the re regular expression processed user query text, to obtain the standardized user question Q; standardization refers to eliminating ambiguity, so that it is more consistent with the query of the vertical field to which the report belongs.

[0091] ​​Other steps and parameters are the same as one of embodiments one to six.

[0092] Embodiment seven: different from one of embodiments one to six, the user question Q in step S8 and the temporary vector database in step S3 correspond to the text block data C t (C t ) containing K D j ) in step S5;

[0093] The specific process is:

[0094] Encode the user question Q into a vector using the vector encoder (use bge1.5-large-zh);

[0095] Retrieve the K most similar vectors corresponding to the user question Q in the temporary vector database in step S3;

[0096] Based on the K most similar vectors, the corresponding text block data C t (C t ) containing K D j ) is obtained;

[0097] Other steps and parameters are the same as one of embodiments one to six.

[0098] Embodiment eight: different from one of embodiments one to seven, the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q in step S4 are used to obtain the response A by a large language model; the specific process is:

[0099] Use the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q in step S4 to write instruction prompts, and use a large language model to obtain the response A;

[0100] Instruction prompts:

[0101] "Here are the relevant fragments retrieved,

[0102] {C t}

[0103] Here is an example of interpreting the question and answer, you can refer to the content, format and style, etc.:

[0104] {Q q}{A q}

[0105] Please answer the following questions:

[0106] {Q}”

[0107] Other steps and parameters are the same as one of embodiments one to seven.

[0108] Embodiment nine: an LLM report interpretation system based on synthetic data, the system comprising:

[0109] a data synthesis module, a document parsing module, a cache library retrieval module, a user query module, a document retrieval module, a question and answer determination module, a response generation module, and a response determination module.

[0110] Embodiment ten: different from embodiment nine, the data synthesis module is used to generate a synthetic database using a large language model, process each question and answer data in the synthetic database, and store it to a question and answer cache library;

[0111] The document parsing module is used to receive a user-uploaded PDF file, parse the PDF file using PyMuPDF, and obtain text data;

[0112] The cache library retrieval module is used to segment the text data obtained in step S2 to obtain each segment of text data, process each segment of text data to generate a vector, and index the vector and store it in a temporary vector database;

[0113] The user query module is used for the user to input a query text and obtain a user question Q through a large language model;

[0114] The document retrieval module is used to retrieve the most similar (Q q +A q ) (cosine similarity) in the established question and answer cache library based on the user question, and obtain the qth question and answer pair data (Q q +A q ) in the synthetic database based on (Q q ,A q ), and let the qth question and answer pair data C q = (Q q ,A q ).

[0115] The question and answer determination module is used to write an instruction prompt word, “{the retrieved most similar question and answer pair data C q} can it directly answer the user question {the user's query}?” and use a large language model to determine whether the most similar question and answer pair data C q = (Q q ,A q ) can directly answer, and if so, the most similar question and answer pair data C q = Q q ,Aq ) in step S3 q is the answer A to the user query text q , and the current question and answer session ends; otherwise, the user question Q and the temporary vector database in step S3 obtain the corresponding text block data C t (C t contains K D j );

[0116] The response generation module is used to obtain the response A from the large language model using the question and answer pair data C q , the text block data C t , and the user question Q.

[0117] The response determination module is used to determine whether the response A obtained by the large language model is reasonable for the user question Q; if so, the response is qualified; if not, the response is unqualified, and the user provides a reasonable answer, which is added to the synthetic database in step S0.

[0118] The other steps and parameters are the same as in embodiment eight.

[0119] Embodiment:

[0120] The present application will be further illustrated in conjunction with the accompanying drawings and specific embodiments, and it should be understood that the following illustrative embodiments of the present application are only used to explain the present application and do not limit the scope of the present application.

[0121] The present application relates to two different models, the first type is a dense vector retrieval model, which can encode text vectors, and the current open source model BGE1.5-large-zh is selected. The second type is a large language model (LLM), which can receive instructions and questions and generate responses, and the open source model Qwen2-72B-Instruct model is selected in the present application.

[0122] The prompts used in the present application are only examples for using the large model to achieve the effect, and should be understood in a broad sense as a series of prompts, and should not be understood as a limitation on the present application.

[0123] Embodiment 1

[0124] The present embodiment provides a method for enhancing the performance of LLM report interpretation based on synthetic data, which can be applied to the interpretation of genetic test reports, and can also be applied to the interpretation of test reports or body reports in the process of disease diagnosis and treatment, such as Figure 1 As shown, the method comprises the following steps:

[0125] Step S0 / As shown in Figure 2 , a synthetic database is generated using a large language model, and the specific process is as follows:

[0126] Step S0-1,

[0127] Collect structured data from existing medical structured databases (including but not limited to open source data extracted from OncoKB tumor database,

[0128] which is structured stored in MySQL, excel or JSON format) (OncoKB tumor database has structured data (table) and unstructured data, structured data is directly collected, unstructured data is converted into structured data, the final data format example, structured attributes and their attribute values, such as {gene: ABL1, gene description: a tyrosine kinase, is frequently altered by chromosomal translocations in leukemia.}, and {gene: ABL1, variation: BCR-ABL1 Fusion, cancer type: B-Lymphoblastic Leukemia / Lymphoma, drug used: Dasatinib});

[0129] Step S0-2, texturize structured data in structured database according to each article (for example, structured data in step S0-1 is directly texturized into: structured data (table) one article per line, "gene: ABL1, gene description: a tyrosine kinase, is frequently altered by chromosomal translocations in leukemia." is an article);

[0130] Step S0-3, based on each article of S0-2 texturization, write instruction prompt words, use large language model to synthesize question and answer data, and get preliminary synthesized question and answer data;

[0131] The specific process is:

[0132] Design and write instruction prompt words, "I hope you act as a question and answer data construction expert, please you only according to the data I provide to construct five question and answer data, the requirement is that the generated sentence conforms to the real dialogue scene; the following is the data to be constructed gene: {database data}, gene description: {database data}",

[0133] Use the instruction prompt words to synthesize question and answer data, and get the preliminary question and answer pair;

[0134] {Data in the database} is the data after text conversion of S0-2 (such as "Gene: ABL1, Gene description: atyrosine kinase, is frequently altered by chromosomal translocations in leukemia.");

[0135] Step S0-4, write instruction prompt words such as "Please judge whether the synthesized question and answer data is reasonable", use large language model to perform quality screening on the preliminary synthesized question and answer data, and if unqualified, repeat step S0-3 until qualified or the number of iteration reaches the upper limit, to obtain the synthesized database {(Q1, A1), (Q2, A2), …, (Q i ,A i ),…,(Q M ,A M )};

[0136] Among them,

[0137] (Q1, A1) represents the first question and answer pair data in the synthesized database, Q1 represents the question in the first question and answer pair data (Q1, A1) in the synthesized database, and A1 represents the answer in the first question and answer pair data (Q1, A1) in the synthesized database;

[0138] (Q2, A2) represents the second question and answer pair data in the synthesized database, Q2 represents the question in the second question and answer pair data (Q2, A2) in the synthesized database, and A2 represents the answer in the second question and answer pair data (Q2, A2) in the synthesized database;

[0139] (Q i ,A i ) represents the i-th question and answer pair data in the synthesized database, Q i represents the question in the i-th question and answer pair data (Q i ,A i ) in the synthesized database, and A i represents the answer in the i-th question and answer pair data (Q i ,A i ) in the synthesized database;

[0140] (Q M ,A M ) represents the M-th question and answer pair data in the synthesized database, Q M represents the question in the M-th question and answer pair data (Q M ,A M ) in the synthesized database, and A M represents the answer in the M-th question and answer pair data (Q M ,A M ) in the synthesized database.

[0141] Step S1, as shown in Figure 2 , each question and answer data in the synthesis database obtained in step S0 is processed and stored into the question and answer cache;

[0142] The specific process is as follows:

[0143] Step S1-1, each question and answer pair in the synthesis database {(Q1, A1), …, (Q i , A i ), …, (Q M , A M )} (formally, {(Q1, A1), … (Q n , A n ), …}) is connected into a text to obtain a new text set {(Q1+A1), …, (Q i +A i ), …, (Q M +A M )};

[0144] Among them,

[0145] (Q1+A1) represents the first new text in the new text set, (Q i +A i ) represents the i-th new text in the new text set, and (Q M +A M ) represents the M-th new text in the new text set;

[0146] Step S1-2, each new text in the new text set obtained in step S1-1 is encoded into a vector using a vector encoder (using BGE1.5-large-zh);

[0147] Step S1-3, index the vector obtained in step S1-2 and store it into the vector database as the question and answer cache (using Chroma).

[0148] Step S2, receiving a PDF file uploaded by a user, parsing the PDF file using PyMuPDF to obtain text data;

[0149] Step S3, as shown in Figure 4 , the text data obtained in step S2 is segmented to obtain each text data; each text data is processed to generate a vector; and the vector is indexed and stored into a temporary vector database;

[0150] The specific process is as follows:

[0151] Step S3-1, the text data obtained in step S2 is segmented to obtain each text data; the specific process is as follows:

[0152] We use the method of sliding window, such as setting the window size to 512 characters, and setting each step to 256;

[0153] The text data obtained in step S2 is divided into N segments by the sliding window method, each segment has a size of 512 characters, and the characters between adjacent segments overlap by 256, obtaining a text data set D = {D1, D2, …, DN}; j ,…,D N};

[0154] Each piece of text data is represented as D j , j = 1, 2, …, N;

[0155] Wherein, D1 represents the first piece of text data, D2 represents the second piece of text data, Dj represents the jth piece of text data, and DN represents the Nth piece of text data, j = 1, 2, …, N; j N

[0156] Step S3-2, encode each piece of text data obtained in step S3-1 into a vector using a vector encoder (using bge1.5-large-zh);

[0157] Step S3-3, index the vector of step S3-2 and store it in a temporary vector database (using Chroma).

[0158] Step S4, the user inputs the query text, and obtains the user question Q through the large language model;

[0159] The specific process is as follows:

[0160] Step S4-1, perform re regular expression processing (including but not limited to eliminating disordered punctuation marks) on the user input query text, to obtain the re regular expression processed user query text;

[0161] Step S4-2, write an instruction prompt word, “Please standardize the re regular expression processed user query text, so that the re regular expression processed user query text conforms to the problem in the genetic detection field”, use the large language model to standardize the re regular expression processed user query text, to obtain the standardized user question Q; standardization refers to eliminating ambiguity, so that it is more consistent with the query of the vertical field to which the report belongs.

[0162] Step S5, search the most similar (Q q +A q ) (cosine similarity) in the question and answer cache library established in step S1 from the user question Q;

[0163] Based on (Q q +A​​q )th question-answer pair data (Q q , A q ) in the synthesis database is obtained q = (Q q , A q ) ;

[0164] wherein (Q q + A q ) represents the qth new text in the new text set; (Q q , A q ) represents the qth question-answer pair data in the synthesis database, Q q represents the question in the qth question-answer pair data (Q q , A q ) in the synthesis database, and A q represents the answer in the qth question-answer pair data (Q q , A q ) in the synthesis database; 1 ≤ q ≤ M.

[0165] Step S6, an instruction prompt word is written, “{the most similar question-answer pair data C q retrieved} can directly answer the user question {the user query}?”, a large language model is used to determine whether the most similar question-answer pair data C q = (Q q , A q ) can directly answer, if yes, step S7 is performed, otherwise step S8 is performed;

[0166] Step S7, the answer A q in the most similar question-answer pair data C q = (Q q , A q ) is the answer A q to the user query text, and the current question and answer is ended;

[0167] Step S8, the user question Q and the temporary vector database in step S3 obtain corresponding text block data C t (C t contains K D j ); the specific process is as follows:

[0168] The user question Q is encoded into a vector using a vector encoder (bge1.5-large-zh is used);

[0169] The K most similar vectors corresponding to the user question Q are retrieved from the temporary vector database in step S3;

[0170] The corresponding text block data C t(C t K D j );

[0171] Step S9, using the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q in step S4, the large language model obtains a response A; the specific process is as follows:

[0172] Using the question and answer pair data C q obtained in step S5, the text block data C t obtained in step S8, and the user question Q in step S4, the instruction prompt word is written, and the large language model obtains a response A;

[0173] Instruction prompt word:

[0174] "Here are the relevant fragments retrieved,

[0175] {C t}

[0176] Here is an example of interpreting a question and answer, you can refer to the content, format and style, etc.:

[0177] {Q q}{A q}

[0178] Please answer the following questions:

[0179] {Q}”

[0180] Step S10,

[0181] The user judges whether the response A obtained by the large language model in step S9 is reasonable for the user question Q;

[0182] If yes, the reply is qualified;

[0183] If not, the reply is unqualified, and the user provides a reasonable reply to form new synthetic data and add it to the synthetic database in step S0.

[0184] The embodiment proposes a method for enhancing the performance of large language model (LLM) report interpretation based on synthetic data, which is suitable for the interpretation of genetic test reports and other medical test reports. By using a professional field structured database to generate pseudo question and answer data, and combining a dense vector retrieval model to vectorize the text, a question and answer cache library is constructed. When the user queries, the system first optimizes the query through standardization processing, and then finds the most relevant question and answer data through vector retrieval, generates a response combined with the report content, and improves the accuracy and efficiency of interpretation. Finally, the system will optimize the question and answer data according to the user feedback, dynamically update the cache library to continuously improve the performance.

[0185] Embodiment 2

[0186] The embodiment provides a system for interpreting the performance of a synthetic data enhanced LLM (large language model) report, which can be applied to the interpretation of genetic testing reports, and can also be applied to the interpretation of testing reports or interbody reports in the disease diagnosis and treatment process. It includes the following modules:

[0187] (1) The data synthesis module is used to generate a synthetic database using a large language model, process each question and answer data in the synthetic database, and store it in the question and answer cache library;

[0188] The data processing unit collects information from (such as a MySQL database) and unstructured text (such as medical reports, academic articles, etc.). By connecting the database using the SQLAlchemy library, using the pandas library to organize table data, and converting it into a format suitable for question and answer.

[0189] The question and answer synthesis unit: according to the information extracted from the data, through the prompt word (Prompt) technology, it uses a large language model (such as Qwen2-72B-Instruct or GPT series) to synthesize pseudo question and answer pairs.

[0190] The quality screening unit: through the large language model, the synthesized question and answer are evaluated for quality, and the question and answer pairs that meet the standards are screened out. If the question and answer do not meet the expectations, the system will re-synthesize. The instruction example used is: "Please judge whether the following question and answer are reasonable."

[0191] The vectorization module: the screened question and answer pairs are processed for vectorization and stored in the vector database. Using the sentence-transformers library to encode the question and answer pairs into vectors (we use BGE1.5-large-zh), using vector databases such as Chroma or FAISS for storage and management, facilitating subsequent fast retrieval.

[0192] The document parsing module is used to receive user-uploaded PDF files, and uses PyMuPDF to parse the PDF files to obtain text data;

[0193] The PDF parsing unit, when the user uploads a PDF report, the system uses the PyMuPDF (also known as fitz) library to extract the text in the report, ensuring accurate parsing.

[0194] The text cleaning unit uses regular expressions (re) for cleaning, removing these redundant parts, and retaining parts related to the report content.

[0195] Chunking unit: In order to better search the report, the text is divided into multiple smaller pieces. The size of each piece is usually 512 characters, and the text is cut by a sliding window method with a step size of 256 characters. Each piece of text is vectorized separately to facilitate subsequent search and interpretation.

[0196] (3) Cache library search module, which uses LangChain framework and searches the user's query through the vector database to find the most relevant question and answer data. This module ensures that the system can efficiently and quickly respond to user queries.

[0197] The cache library search module is used to segment the text data obtained in step S2 to obtain each piece of text data; process each piece of text data to generate a vector; index the vector and store it in a temporary vector database;

[0198] The user query module is used for the user to input the query text, and the large language model is used to obtain the user's question Q;

[0199] The document search module is used to search the most similar (Q q +A q ) in the established question and answer cache library based on the user's question Q; q +A q ) to obtain the qth question and answer pair data (Q q ,A q ) in the synthesis database, and let the qth question and answer pair data C q =(Q q ,A q );

[0200] Vectorization unit: With the support of the data synthesis module, all question and answer pairs have been converted into vector form and stored in the vector database. Each question and answer pair is encoded as a fixed-length vector representing its semantic features.

[0201] Similarity unit: When the user submits a query, the system first vectorizes the query text and searches through the vector database (such as FAISS or Chroma). The search process is based on cosine similarity or other similarity algorithms to find the most similar question and answer pair. We use LangChain to implement the database connection and encoding query process. (4)

[0203] The question and answer determination module is used to write instruction prompts, "{the most similar question and answer pair data C q} can directly answer the user's question {the user's query}?", and uses a large language model to determine the most similar question and answer pair data C q =(Q q ,Aq Can this be answered directly? If so, then the most similar question-and-answer pair is data C. q =(Q q A q Answer A in ) q That is, the answer A to the user's query text. q And end this Q&A session; otherwise, the user's question Q and the temporary vector database from step S3 will obtain the corresponding text block data C. t (C t It contains K D j );

[0204] Text vectorization unit: Each piece of text in the report (such as segments split by the PDF parsing module) is vectorized. Using the sentence-transformers library, the BGE1.5 large zh model is used to encode each text block as a vector and store it in a temporary vector database.

[0205] Relevant text retrieval unit: When a user queries, the system searches a temporary vector database for the most relevant text fragments. This retrieval process also relies on vector similarity calculation to ensure that the returned fragments are highly relevant to the query.

[0206] (5) The response generation module is used to utilize question-and-answer pair data C q Text block data C t Given the user's question Q, the large language model receives the response A;

[0207] The answer generation unit, whose integrated context is fed into a large language model (such as Qwen2-72B-Instruct), generates the final answer using the model's generative capabilities. The generated answer takes all provided contextual information into account to ensure accuracy.

[0208] Response Output Unit: The final generated answer will be returned to the user. If the query is simple, the answer may only come from the question-and-answer pairs retrieved from the cache; if the query is complex, the answer will incorporate more report fragments and question-and-answer examples.

[0209] Query Standardization Unit: User-submitted queries may be ambiguous or non-standard. The system will standardize the queries using a large language model. This includes correcting incorrect punctuation and eliminating semantic ambiguity, making the queries more explicit.

[0210] Context integration: In addition to standardized queries, the system also integrates queries with other relevant information such as report segments, Q&A examples, so that the generator module can generate more accurate answers. The instructions used are as follows: "Below is an example of interpreting Q&A, you can refer to its content, format and style, etc.: {example Q&A pair}. Below is a report-related segment: {retrieved report-related content}"

[0211] (8) Response judgment module for user to judge whether the response A obtained by large language model is reasonable for user question Q; if yes, the reply is qualified; if not, the reply is unqualified, the user provides a reasonable reply, and a new synthesized data is added to the synthesized database of step S0.

[0212] Through the cooperation of multiple modules, the system realizes a closed loop from data synthesis, document parsing, query processing to user feedback. Through continuous learning and optimization, it can provide high-quality report interpretation services for users.

[0213] The present application can also have other various embodiments, and those skilled in the art can make various corresponding changes and modifications according to the present application without departing from the spirit and essence of the present application. However, these corresponding changes and modifications should all belong to the protection scope of the claims attached to the present application.

Claims

1. A synthetic data-based LLM report interpretation method, characterized in that: The method specifically comprises the following steps: Step S0, generating a synthetic database using a large language model; Step S1, processing each question and answer data in the synthetic database obtained in step S0 and storing the question and answer data in a question and answer cache library; the specific process is as follows: Step S1-1, connect each question-answer pair in the synthetic database {(Q1, A1),..., (Q i , A i ),..., (Q M , A M )} into a text to obtain a new text set {(Q1+A1),..., (Q i + A i ),..., (Q M + A M )}; wherein, (Q1+A1) represents the first new text in the new text set, (Q i +A i ) represents the i-th new text in the new text set, (Q M +A M ) represents the M-th new text in the new text set; Step S1-2, encoding each new text in the new text set obtained in step S1-1 into a vector using a vector encoder; Step S1-3, indexing the vector obtained in step S1-2 and storing the vector in a vector database as the question and answer cache library; Step S2, receiving a PDF file uploaded by a user, parsing the PDF file using PyMuPDF to obtain text data; Step S3, segmenting the text data obtained in step S2 to obtain each segment of text data; processing each segment of text data to generate a vector; and indexing the vector and storing the vector in a temporary vector database; Step S4, inputting a query text by a user, and obtaining a user question Q by using a large language model; Step S5, the user question Q is searched in the question and answer cache library established in step S1 for the most similar (Q q + A q ); Based on (Q q ) +A q ) obtain the qth question and answer pair data (Q q ,A q ) in the synthesis database, let the qth question and answer pair data C q =(Q q ,A q ); wherein (Q q , A q ) represents the qth new text in the new text set; (Q q , A q ) represents the qth question and answer pair data in the synthetic database, Q q represents the question in the qth question and answer pair data (Q q , A q ) in the synthetic database, A q represents the answer in the qth question and answer pair data (Q q , A q ) in the synthetic database; 1≤q≤M; Step S6, write instruction prompt words, use large language model to judge the most similar question and answer pair data C q = (Q q , A q ) can be answered directly, if yes, proceed to step S7, otherwise proceed to step S8; Step S7, the answer A in the most similar question-answer pair data C q = (Q q , A q ) is the answer A of the user query text, and the current question-answer is ended. q q , and the current question-answer is ended.​ Step S8, the user question Q and the temporary vector database of step S3 get the corresponding text block data C t ; Step S9, the question and answer pair data C obtained in step S5 is used q , the text block data C obtained in step S8 t and the user question Q in step S4, and the large language model obtains the response A; Step S10, Step S9, determining whether the response A obtained by the large language model is reasonable for the user question Q; If yes, the response is qualified; If not, the response is unqualified, and a reasonable response is provided by the user to form new synthetic data and add the new synthetic data to the question and answer cache library in step S1.

2. The synthetic data-based LLM report interpretation method of claim 1, wherein: In step S0, the synthetic database is generated using a large language model; the specific process is as follows: Step S0-1, collecting structured data from an existing structured database; Step S0-2, texturing the structured data in the structured database; Step S0-3, based on each data textured in S0-2, writing an instruction prompt word, and using a large language model to synthesize question and answer data to obtain preliminary synthesized question and answer data; Step S0-4, write instruction prompt words, use large language model to screen the quality of the preliminary synthesized question and answer data, and repeat step S0-3 if unqualified, until qualified or the number of iterations reaches the upper limit, to obtain the synthesized database {(Q1, A1), (Q2, A2), …, (Q i , A i ) , …, (Q M , A M )}; wherein, (Q1, A1) represents the first question and answer pair data in the synthetic database, Q1 represents the question in the first question and answer pair data (Q1, A1) in the synthetic database, and A1 represents the answer in the first question and answer pair data (Q1, A1) in the synthetic database; (Q2, A2) represents the second question and answer pair data in the synthetic database, Q2 represents the question in the second question and answer pair data (Q2, A2) in the synthetic database, and A2 represents the answer in the second question and answer pair data (Q2, A2) in the synthetic database; (Q i ,A i ) represents the i-th question and answer pair data in the synthesized database, Q i represents the question in the i-th question and answer pair data (Q i ,A i ) in the synthesized database, A i represents the answer in the i-th question and answer pair data (Q i ,A i ) in the synthesized database; (Q M ,A M ) denotes the Mth question and answer pair data in the synthesis database, Q M denotes the question in the Mth question and answer pair data (Q M ,A M ) in the synthesis database, A M denotes the answer in the Mth question and answer pair data (Q M ,A M ) in the synthesis database.

3. The synthetic data-based LLM report interpretation method of claim 2, wherein: In step S3, the text data obtained in step S2 is segmented to obtain each segment of text data; each segment of text data is processed to generate a vector; and the vector is indexed and stored in a temporary vector database; The specific process is as follows: Step S3-1, segmenting the text data obtained in step S2 to obtain each segment of text data; Step S3-2, encoding each segment of text data obtained in step S3-1 into a vector using a vector encoder; Step S3-3, indexing the vector in step S3-2 and storing the vector in a temporary vector database.

4. The synthetic data-based LLM report interpretation method of claim 3, wherein: In step S3-1, the text data obtained in step S2 is segmented to obtain each segment of text data; the specific process is as follows: The text data obtained in step S2 is divided into N segments by using a sliding window method, each segment has a size of 512 characters, and the characters between adjacent segments overlap by 256, to obtain a text data set D = {D1, D2, …, D j N N}. Each piece of text data is represented as D j , j = 1, 2, …, N; wherein D1 represents the first piece of text data, D2 represents the second piece of text data, D j represents the jth piece of text data, D N represents the Nth piece of text data, j = 1, 2,..., N.

5. The synthetic data-based LLM report interpretation method of claim 4, wherein: In step S4, the user inputs a query text, and the user question Q is obtained by using a large language model; the specific process is as follows: Step S4-1, re regular expression processing is performed on the user input query text to obtain re regular expression processed user query text; Step S4-2, an instruction prompt word is written, and a large language model is used to standardize the re regular expression processed user query text to obtain a standardized user question Q.

6. The synthetic data-based LLM report interpretation method of claim 5, wherein: The user question Q in the step S8 and the temporary vector database of the step S3 obtain corresponding text block data C t ; the specific process is: The user question Q is encoded into a vector using a vector encoder; In step S3, the K most similar vectors corresponding to the user question Q are retrieved from the temporary vector database; Based on the most similar K vectors, the corresponding text block data C is obtained t .

7. The synthetic data-based LLM report interpretation method of claim 6, wherein: The question and answer pair data C obtained in step S5 is used in the step S9 q The text block data C obtained in step S8 t And the user question Q of step S4, the large language model obtains the response A; The specific process is as follows: The question and answer pair data C obtained by step S5 q The text block data C obtained by step S8 t The instruction prompt word is written with the user question Q of step S4, and the response A is obtained using a large language model.

8. A synthetic data-based LLM report interpretation system, characterized by: The system comprises a data synthesis module, a document parsing module, a cache library retrieval module, a user query module, a document retrieval module, a question and answer determination module, a response generation module, and a response determination module. The data synthesis module is used to generate a synthetic database using a large language model, process each question and answer data in the synthetic database, and store it in the question and answer cache library; the specific process is as follows: Connect each of the question-answer pairs in the synthetic database {(Q1, A1),..., (Q i ,A i ),..., (Q M ,A M )} into one text to obtain a new text set {(Q1+A1),..., (Q i +A i ),..., (Q M +A M )}; Wherein, (Q1+A1) represents the first new text in the new text set, (Q i +A i ) represents the i-th new text in the new text set, (Q M +A M ) represents the M-th new text in the new text set; Each new text in the new text set is encoded into a vector using a vector encoder; The vector is indexed and stored in the vector database as a question and answer cache library; The document parsing module is used to receive a user uploaded PDF file, parse the PDF file using PyMuPDF, and obtain text data; The cache library retrieval module is used to segment the obtained text data to obtain each text data, process each text data to generate a vector, and index the vector and store it in the temporary vector database; The user query module is used for user to input query text, and obtain user question Q through large language model; The document retrieval module is used to search for the most similar (Q) answers to user questions in the established question-and-answer cache. q +A q ); based on (Q) q +A q Retrieve the q-th question-answer pair data from the synthetic database (Q q A q Let C be the data of the q-th question-answer pair. q =(Q q A q ); The question and answer determination module is used to write instruction prompt words, and a large language model is used to determine the most similar question and answer pair data C q = (Q q , A q ) whether it can be directly answered, if yes, the answer A q in the most similar question and answer pair data C q = (Q q , A q ) is the answer A q of the user query text, and the current question and answer is ended; otherwise, the user question Q and the corresponding text block data C t are obtained from the temporary vector database. The response generation module is configured to utilize the question-answer pair data C q , the text block data C t and the user question Q to obtain the response A by using a large language model. The response determination module is used for user to determine whether the response A obtained by large language model is reasonable for user question Q; if yes, reply qualified; if not, reply unqualified, user provides reasonable reply, constitutes new synthesis data and adds to synthesis database.

Images