Ultrasonic examination result question and answer science popularization method and system based on large language model

Abstract

The invention discloses an ultrasonic examination result question and answer science popularization method and system based on a large language model, and relates to the technical field of medical health informatization and natural language process.The ultrasonic examination result question and answer science popularization method based on the large language model mainly comprises the steps that data enhancement and data cleaning are conducted on initial training data, and a training data set is obtained; according to the general fine tuning instruction set, obtaining an ultrasonic field instruction data set by using a GPT-4 API tool; according to the training data set, pre-training the large language model by using a low-rank adaptation method to obtain a pre-trained large language model; and performing instruction fine tuning on the pre-trained large language model to obtain an ultrasonic examination result question-answer large model, and answering questions of the user. According to the ultrasonic examination result question and answer science popularization method and system based on the large language model, accurate, friendly and personalized question and answer service can be provided for the patient in combination with the ultrasonic examination report.

Description

Technical Field

[0001] The present invention relates to the fields of medical health informationization and natural language processing technology, and more specifically, to a method and system for popularizing ultrasound examination result questions and answers based on a large language model. Background Art

[0002] With the rapid development of medical imaging technology, ultrasound examinations have become a common and important diagnostic method in modern medicine. However, ultrasound examination reports often contain a large amount of medical terminology and specialized content. While these terms are easy for medical professionals to understand, ordinary patients, due to their lack of medical knowledge, find it difficult to accurately understand the report content, especially those related to lesions, conclusions, and health risks. For example, common terms in ultrasound reports such as "echoic enhancement," "hypoechoic areas," "cystic structures," and "solid lesions" are relatively unfamiliar to ordinary patients and can easily lead to misunderstandings. The health consultation services currently provided by medical institutions are often limited by the availability of medical staff, making it difficult to provide adequate explanations and answers to every patient's questions.

[0003] Patients often want to easily understand ultrasound examination reports and access relevant medical knowledge. However, current solutions are limited. Doctors have limited time to explain and struggle to address all patient questions. Furthermore, while some medical information platforms offer relevant knowledge, most systems rely on static knowledge bases and lack interactivity and personalized response capabilities. Traditional question-and-answer systems exhibit limitations when handling complex medical terminology and personalized questions, failing to fully meet patient needs. Patients often seek answers on their own through search engines or social media, but the reliability and accuracy of this information cannot be guaranteed.

[0004] Recent advances in natural language processing technology have been significant, particularly with the emergence of large language models (such as OpenAI's GPT series), which have revolutionized language understanding and generation. These models, pre-trained on massive amounts of text data, are able to capture the semantics and contextual relationships in language. However, these models are often trained on general corpora and lack specialized fine-tuning for the medical field, particularly ultrasound examination reports. This makes them unable to efficiently and professionally answer specific questions about ultrasound reports.

[0005] Currently, research on AI-based medical information interpretation and popularization focuses on the following areas: Some research is focused on generating report text from imaging data using deep learning models. For example, these methods use convolutional neural networks (CNNs) to extract image features and recurrent neural network variants, such as long short-term memory networks, to generate diagnostic descriptions. For example, the CNN-RNN architecture designed by Shin et al. can predict annotated label information from chest X-ray images, including the severity of the condition and the organs affected. However, these approaches are primarily designed to assist physicians in generating reports, rather than providing professional interpretation for patients.

[0006] Some research has developed rule-based or machine learning-based medical question-answering systems. For example, these systems utilize knowledge graphs or retrieval technologies to answer medical questions. However, these systems are often limited to common diseases and lack the ability to accurately interpret personalized reports. In recent years, there has been a growing effort to fine-tune language models for the medical field, such as BioBERT and ClinicalBERT, which are trained on medical data from PubMed and MIMIC. However, these models are primarily used in specialized scenarios, such as assisting doctors with scientific research analysis, rather than serving ordinary patients.

[0007] Pre-trained language models have recently been increasingly applied to electronic health record (EHR) analysis, medical literature summarization, and medical question-answering systems. For example, researchers like Rasmy adapted the BERT framework to the structured EHR domain, improving the accuracy of two disease prediction tasks and facilitating disease prediction using small, local training datasets. Furthermore, Wang et al. constructed a reliable large-scale Chinese medical language model fine-tuned with Chinese medical knowledge, improving the performance of general-purpose large-scale models in medical question-answering.

[0008] However, there is currently no mature large language model for automated question answering and popular science explanation of ultrasound examination reports. The main reasons are: 1. Insufficient domain knowledge and lack of professional depth: Although general large language models are trained with massive amounts of data, their training corpus lacks medical data, especially professional knowledge such as ultrasound medical terminology, pathological characteristics, and diagnostic criteria. This makes it difficult to ensure the accuracy and professionalism of answers when processing the professional content of ultrasound reports.

[0009] 2. Lack of patient-friendliness and overly professional expressions: The general model is not trained to meet the scientific needs of ordinary patients. Its answers tend to be professionally expressed and contain a large number of medical terms, which are difficult for ordinary patients to understand. It cannot meet patients' urgent needs for "accurate and easy-to-understand" scientific explanations.

[0010] 3. Lack of context: The interpretation of ultrasound reports often requires a comprehensive analysis based on the patient's age, medical history, examination purpose, and other contextual information. However, general models lack the ability to understand and reason about individual patient situations, making it difficult to provide highly personalized answers.

[0011] Therefore, there is an urgent need for a Q&A system for ultrasound examination results based on a large language model. Through further training and instruction fine-tuning, the system can understand the professional content in the ultrasound examination report and provide accurate and easy-to-understand popular science answers to questions raised by patients, helping patients better understand their own health status and thus improving their medical service experience. Summary of the Invention

[0012] The purpose of the present invention is to provide a method and system for popularizing ultrasound examination result questions and answers based on a large language model, which can provide patients with accurate, friendly and personalized question and answer services in combination with ultrasound examination reports.

[0013] The present invention provides a popular science method for ultrasound examination result question and answer based on a large language model, comprising the following steps: S1: performing data enhancement on initial training data to obtain a high-quality data set in the ultrasound field; S2: performing data cleaning on the high-quality data set in the ultrasound field to obtain a training data set; S3: using a GPT-4 API tool according to a general fine-tuning instruction set to obtain an ultrasound field instruction data set; S4: using a low-rank adaptation method according to the training data set to pre-train the large language model to obtain a pre-trained large language model; S5: performing instruction fine-tuning on the pre-trained large language model to obtain a large model for ultrasound examination result question and answer; S6: using the large model for ultrasound examination result question and answer to answer user questions.

[0014] Furthermore, the above-mentioned initial training data includes ultrasound examination report samples, and the ultrasound examination report samples include image annotations and text descriptions; data enhancement is performed on the initial training data, including converting medical synonyms and professional terminology expansion of case descriptions, and converting colloquial descriptions into professional medical terminology expressions.

[0015] Furthermore, the above-mentioned high-quality datasets in the ultrasound field are cleaned, including privacy filtering, dirty data processing, and low-quality data screening.

[0016] Furthermore, step S3 specifically includes: adding refined instructions to the Prompt template based on the general fine-tuning instruction set, and using the GPT-4 API tool to generate an ultrasound field instruction dataset in a self-guided instruction manner.

[0017] Furthermore, step S4 specifically includes: inserting a LoRA adapter into a key layer of the large language model, setting the rank value of the LoRA adapter to a first rank value, setting the scaling factor of the LoRA adapter to a first scaling factor, setting the random inactivation ratio of the LoRA adapter to a first random inactivation ratio, optimizing only low-rank parameters, freezing other model weights, so that the model maintains its original performance and adapts to new tasks; pre-training the large language model in an autoregressive form according to the training data set, saving the pre-trained LoRA weights, and obtaining a pre-trained large language model.

[0018] Furthermore, the large language model is ChatGLM3, the first rank value is 32, the first scaling factor is 16, and the first random deactivation ratio is 0.05.

[0019] Furthermore, step S5 specifically includes: setting the rank value of the LoRA adapter to the second rank value, setting the scaling factor of the LoRA adapter to the second scaling factor, setting the random deactivation ratio of the LoRA adapter to the second random deactivation ratio, optimizing only the inserted LoRA layer, and keeping the basic model weight unchanged; loading the pre-trained model, initializing based on the saved LoRA weights, and freezing the master weights.

[0020] Furthermore, the second rank value is 16, the second scaling factor is 8, the second random inactivation ratio is 0.1, the preset learning rate is 1e-4, and the preset round number is 5.

[0021] The present invention also provides an ultrasound examination result question and answer science popularization system based on a large language model, and the system includes the following modules: a data enhancement module, configured to: perform data enhancement on the initial training data to obtain a high-quality data set in the ultrasound field; a data cleaning module, configured to: perform data cleaning on the high-quality data set in the ultrasound field to obtain a training data set; an ultrasound field instruction data generation module, configured to: obtain an ultrasound field instruction data set based on a general fine-tuning instruction set using the GPT-4 API tool; a pre-training module, configured to: pre-train the large language model based on the training data set using a low-rank adaptation method to obtain a pre-trained large language model; an instruction fine-tuning module, configured to: perform instruction fine-tuning on the pre-trained large language model to obtain a large model for ultrasound examination result question and answer; a user question module, configured to use the large model for ultrasound examination result question and answer to answer user questions.

[0022] The implementation of the ultrasound examination result question-answering popular science method and system based on a large language model provided by the present invention has the following beneficial effects: By constructing a high-quality dataset specifically for ultrasound and performing refined instruction fine-tuning, this paper develops a large language model with a deep understanding of professional terminology and medical background knowledge. This model not only efficiently processes the specialized content in complex ultrasound examination reports and provides scientific, practical, and easy-to-understand explanations, but also enhances its practical application value and demonstrates broad application potential and promotion prospects. This invention uses deep learning technology to pre-train and fine-tune the Large Language Model (LLM), enabling it to specifically target the medical terminology and diagnostic information in ultrasound examination reports. This not only provides accurate and personalized question-and-answer services, helping patients accurately understand ultrasound examination reports, improving medical service efficiency and personal health management capabilities, but also generates scientific answers based on the user's specific situation and provides relevant medical knowledge, thereby effectively improving patients' health literacy and promoting their active participation in health management. In summary, the present invention combines the content of ultrasound examination reports with the medical knowledge base to further train and fine-tune the large language model, so that it can master the professional terminology and diagnostic logic in the field of ultrasound, automatically identify the professional terminology in the report and generate easy-to-understand popular science answers. By combining the professional knowledge of ultrasound examination reports with natural language generation technology, it provides patients with accurate, friendly, and personalized question-and-answer services, helping patients understand the content of the report and obtain relevant popular science knowledge. BRIEF DESCRIPTION OF THE DRAWINGS The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 This is a flow chart of the ultrasound examination result question-answering popular science method based on a large language model provided by the present invention; Figure 2 It is a flow chart of model construction provided by the present invention; Figure 3 This is a prompt schematic diagram of generating instruction fine-tuning data provided by the present invention; Figure 4 Schematic diagram of the Lora method provided by the present invention; Figure 5 This is a schematic diagram of the pre-training script parameters provided by the present invention; Figure 6 This is a schematic diagram of the instruction fine-tuning script parameters provided by the present invention; Figure 7 This is a schematic diagram of user questions and model answers provided by the present invention. DETAILED DESCRIPTION

[0023] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0024] Figure 1A schematic diagram of a method for popularizing ultrasound examination results through question-and-answer based on a large language model is shown. In this embodiment, the method for popularizing ultrasound examination results through question-and-answer based on a large language model includes the following steps: S1: Perform data augmentation on the initial training data to obtain a high-quality dataset in the ultrasound field; In an exemplary embodiment, the initial training data includes ultrasound examination report samples, which include image annotations and text descriptions; data enhancement is performed on the initial training data, including converting medical synonyms and expanding professional terms in case descriptions, and converting colloquial descriptions into professional medical terminology expressions; As an exemplary embodiment, in step S1, a high-quality ultrasound dataset is constructed: a large number of ultrasound examination report samples covering various disease states of various organs such as the heart, liver, and kidneys are collected, and the samples include detailed image annotations and text descriptions written by doctors. Case discussions, medical education materials, and research reports are obtained from public medical databases to enrich the model knowledge system. This dataset construction method is innovative and different from existing general corpora or datasets of small amounts of medical text. S2: Clean the high-quality ultrasound dataset to obtain a training dataset; In an exemplary embodiment, data cleaning is performed on high-quality datasets in the ultrasound field, including privacy filtering, dirty data processing, and low-quality data screening; As an exemplary embodiment, in step S2, comprehensive data cleaning is performed, using regular expression matching, medical term recognition, named entity recognition and other technologies combined with manual review to remove all private information that may reveal personal identity and ensure the legal use of data, that is, privacy filtering; by formulating detailed rules, correcting or deleting erroneous records to ensure data accuracy, that is, dirty data processing; using automated scripts combined with manual review to remove incomplete or unclear data points and improve the overall data quality, that is, low-quality data screening; in addition, doctors and medical experts are organized to review data set samples to ensure that medical knowledge is accurate and credible; S3: Based on the general fine-tuning instruction set, use the GPT-4 API tool to obtain the ultrasound field instruction dataset; In an exemplary embodiment, step S3 specifically includes: adding refined instructions to the Prompt template based on the general fine-tuning instruction set, and using the GPT-4 API tool to generate an ultrasound field instruction dataset in a self-guided instruction manner; As an exemplary embodiment, in step S3, GPT-4API or other advanced tools are used to generate a dataset of questions and answers covering common user demand scenarios in a self-instruction manner. Through a carefully designed and iteratively optimized Prompt template, a rich question-answer set covering various organs and lesions is generated. At the same time, based on the existing general fine-tuning instruction set (such as GPT4-Alpaca, BELLE), through knowledge injection and data enhancement technology, general instructions are combined with professional terms in the ultrasound field and common user questions to enhance the model's anti-forgetting ability, improve its practicality and accuracy in practical applications, and generate ultrasound field instruction data. It should be noted that in step S3, API tools are used to generate self-instruction instruction data specific to the ultrasound field (self-instruct When using ultrasound data, we add refined instructions such as "Please think about the questions patients may have about ultrasound reports from their perspective" and "Ensure that the generated answers are easy to understand" to the prompt template. This combines ultrasound field professional terminology and user common questions to expand the model's learning scope and improve its practicality and accuracy in practical applications. It also includes a general set of fine-tuning instructions, such as GPT4-Alpaca and BELLE. S4: Based on the training data set, the large language model is pre-trained using a low-rank adaptation method to obtain a pre-trained large language model; In an exemplary embodiment, step S4 specifically includes: inserting a LoRA adapter in a key layer of the large language model, setting the rank value of the LoRA adapter to a first rank value, setting the scaling factor of the LoRA adapter to a first scaling factor, setting the random deactivation ratio of the LoRA adapter to a first random deactivation ratio, optimizing only low-rank parameters, freezing other model weights, so that the model maintains its original performance and adapts to new tasks; pre-training the large language model in an autoregressive form according to the training data set, saving the pre-trained LoRA weights, and obtaining a pre-trained large language model; In an exemplary embodiment, the large language model is ChatGLM3, the first rank value is 32, the first scaling factor is 16, and the first random deactivation ratio is 0.05; As an exemplary embodiment, the model is pre-trained in step S4. The specific process is: select a high-performance open source large language model (such as LLaMA3, ChatGLM3, Qwen2) as the basic model, and combine it with the low-rank adaptation method (LoRA) to achieve fast and effective parameter adjustment; the LoRA method diagram is shown in the figure. Figure 4 As shown, the pre-training script parameters are as follows Figure 5As shown in the figure, a LoRA adapter is inserted into the key layers of the model (such as the q_proj and v_proj matrices of the Self-Attention module), and the parameters are set as follows: rank value (r) = 32, scaling factor (α) = 16, Dropout = 0.05. Only low-rank parameters are optimized, and other model weights are frozen to ensure that the model maintains its original performance and adapts to new tasks. The cleaned dataset is loaded and the model is pre-trained in an autoregressive form, so that the model can effectively master the professional terminology and basic knowledge in the field of ultrasound, and has the initial ability to process ultrasound examination reports, which significantly improves the model's professionalism in the medical field and lays a solid foundation for the effectiveness of subsequent instruction fine-tuning. In addition, distributed training technology (such as DeepSpeed) is used to accelerate the training process and save the pre-trained LoRA weights. S5: Fine-tune the pre-trained large language model to obtain a large model for ultrasound examination result question answering. In an exemplary embodiment, step S5 specifically includes: setting the rank value of the LoRA adapter to a second rank value, setting the scaling factor of the LoRA adapter to a second scaling factor, setting the random deactivation ratio of the LoRA adapter to a second random deactivation ratio, optimizing only the inserted LoRA layer, and keeping the basic model weight unchanged; loading the pre-trained model, initializing based on the saved LoRA weights, and freezing the master weights; using cross entropy loss as the loss function, setting the learning rate to a preset learning rate, using a cosine annealing scheduling strategy, performing a preset round of training, and saving the final fine-tuning weights after completion to obtain a large model of ultrasound examination result question and answer; In an exemplary embodiment, the second rank value is 16, the second scaling factor is 8, the second random dropout ratio is 0.1; the preset learning rate is 1e-4; the preset round number is 5; As an exemplary embodiment, in step S5, the pre-trained large language model is fine-tuned, specifically including: setting LoRA parameters, rank value (r=16), appropriate scaling factor (α=8) and dropout ratio (0.1) to reduce computing requirements and ensure that the model adapts to complex user question-answering scenarios, optimizing only the inserted LoRA layer, keeping the basic model weights unchanged, and ensuring fine-tuning efficiency; loading the pre-trained model, initializing it based on the saved LoRA weights, and freezing the master weights; using cross entropy loss as the loss function, setting the learning rate to 1e-4, using the cosine annealing scheduling strategy, and training for 5 epochs, and saving the final fine-tuned weights after completion; S6: Use the ultrasound examination result question-answering model to answer user questions; As an exemplary embodiment, in step S6, a user interaction interface is constructed, and the llm-factory framework is used to load a pre-trained and fine-tuned large language model in the ultrasound field to provide users with a friendly interactive interface; users can ask questions about the ultrasound report through the text input box, and the system automatically parses the content and extracts key information for processing, and quickly gives scientific, reliable and easy-to-understand answers, which include all important findings and corresponding medical explanations.

[0025] This embodiment provides a popular science system for ultrasound examination result question-and-answer based on a large language model, comprising the following modules: a data enhancement module, configured to perform data enhancement on initial training data to obtain a high-quality ultrasound dataset; a data cleaning module, configured to perform data cleaning on the high-quality ultrasound dataset to obtain a training dataset; an ultrasound instruction data generation module, configured to obtain an ultrasound instruction dataset using a GPT-4 API tool based on a general fine-tuning instruction set; a pre-training module, configured to pre-train a large language model based on the training dataset using a low-rank adaptation method to obtain a pre-trained large language model; an instruction fine-tuning module, configured to perform instruction fine-tuning on the pre-trained large language model to obtain a large model for ultrasound examination result question-and-answer; and a user questioning module, configured to answer user questions using the large model for ultrasound examination result question-and-answer. It should be noted that the system uses deep learning technology to pre-train large language models in professional fields and fine-tune instructions to achieve accurate interpretation and popularization of ultrasound examination reports; it not only helps patients better understand their own health conditions and improves the popularity of medical information and patient experience, but also demonstrates broad application potential and promotion prospects with its innovative data set construction, instruction data generation and model training methods, which helps to improve medical service efficiency and personal health management capabilities.

[0026] In some embodiments, the above-mentioned ultrasound examination result question-answering popular science method based on a large language model can also be implemented in the following manner. Figure 2 As shown, the method mainly includes the following steps: Step 1: Construction and preprocessing of a high-quality ultrasound dataset. To enable the large language model to accurately and professionally serve the interpretation of ultrasound examination results, this embodiment innovatively constructs a high-quality dataset specifically for the ultrasound field. The dataset construction method and features are as follows: (1) Diversified and professional data sources: The present invention not only collects a large number of real ultrasound examination report samples from hospitals and ultrasound institutions as initial training data, but also covers different disease states of multiple organs such as the heart, liver, and kidneys, and contains detailed image annotations and professional text descriptions written by doctors, ensuring the authenticity and professionalism of the data; this is essentially different from the general corpus or data sets containing only a small amount of medical text in the existing technology, laying a solid foundation for the model to learn ultrasound professional knowledge; in addition, in order to further enrich the knowledge system of the model, this embodiment also adds relevant case discussions and authoritative medical education materials obtained from public medical databases, as well as high-quality medical popular science articles to ensure the comprehensiveness and authority of the data set; (2) Data augmentation technology improves data diversity: To increase the richness and diversity of the dataset, this embodiment innovatively uses data augmentation technology in the medical field, such as converting case descriptions into medical synonyms and expanding professional terms. For example, colloquial descriptions are converted into more professional medical terminology, enabling the model to adapt to a wider range of question-answering scenarios and improving the model's generalization ability. Step 2: Data cleaning is a key step in ensuring data quality and legitimacy. It removes unnecessary noise and privacy information, making the dataset cleaner and more reliable, thereby improving the effectiveness of model training. Comprehensive data cleaning is performed on the raw data obtained in Step 1, including privacy filtering to remove all information that may reveal personal identity to ensure the legitimacy of data use; dirty data processing to correct or delete erroneous records to ensure data accuracy; and low-quality data is screened out, removing incomplete or ambiguous data points to improve the overall data quality. Step 3: Refined generation and expansion of ultrasound-specific command data. Generating high-quality ultrasound-specific command data is intended to enable the model to learn how to accurately and user-friendly respond to specific types of user queries, enhancing its practicality and accuracy in practical applications. This embodiment uses the following methods to refine and expand ultrasound-specific command data: (1) Self-Instruct instruction generation based on GPT-4 API: The present invention innovatively uses GPT-4 API in combination with the preferred Prompt template (such as Figure 3The Prompt template has been optimized for multiple iterations and can guide the GPT-4 API to generate instruction data covering common ultrasound examination questions and high-quality popular science answers, ensuring the professionalism and practicality of the generated instruction data. For example, it generates input-output pairs for common terms and patient questions in ultrasound reports, such as: Input: "What does hypoechoic mean?" Output: "Hypoechoic refers to the performance of weak reflected echoes in a certain area of the ultrasound image, which may indicate a cyst or other lesion, but needs to be judged in combination with the specific examination report." This instruction data generation method based on the GPT-4 API can build a high-quality, large-scale ultrasound instruction dataset at low cost and high efficiency. (2) Injection and expansion of medical domain knowledge into general instruction datasets: To improve the versatility and anti-forgetting capabilities of the model, and to enable the model to maintain good general conversational capabilities while learning ultrasound detection domain knowledge, this paper uses existing general fine-tuning instruction sets (such as GPT4-Alpaca and BELLE) as a basis. Through knowledge injection and data augmentation technology, the general instruction dataset is combined with ultrasound domain professional terminology and user common questions to expand and generate a richer ultrasound domain instruction dataset. This data augmentation method that integrates general instructions and domain instructions effectively improves the comprehensive performance of the model in both professional and general scenarios.

[0027] Step 4: Professional pre-training of the large language model in the ultrasound field. The goal of the pre-training stage is to provide the model with sufficient medical background knowledge so that it can initially understand and process ultrasound examination reports, while also laying a solid foundation for the next step of fine-tuning instructions. Targeting the specific application scenario of ultrasound popularization, this embodiment proposes professional pre-training of the large language model in the ultrasound field: (1) Selection of high-performance open-source large language models: To reduce training costs and ensure model performance, this embodiment selects the excellent open-source large language models LLaMA3, ChatGLM3, and Qwen2 as basic models; these models excel in general language capabilities and performance, providing a strong foundation for subsequent domain pre-training and instruction fine-tuning; (2) Domain knowledge driven pre-training method: This embodiment uses the high-quality ultrasound domain dataset prepared in steps 1 and 3 to perform domain pre-training on the selected basic model based on the LoRA (Low-Rank Adaptation) low-rank adaptation technology. The schematic diagram of the LoRA method is shown in the figure. Figure 4 As shown; the pre-training script parameters are as follows Figure 5As shown in the figure, through domain pre-training, the model can effectively grasp the professional terminology and basic knowledge of ultrasound, and has the initial ability to process ultrasound examination reports. This significantly improves the model's professionalism in the medical field and lays a solid foundation for the effectiveness of subsequent instruction fine-tuning. This is fundamentally different from the existing methods of directly using a general large language model or only fine-tuning a small amount of medical text. Step 5: Targeted improvement of patient-friendly science popularization capabilities based on instruction fine-tuning. Instruction fine-tuning is intended to make the model more accurately adapt to the specific application scenario of ultrasound examination result questions and answers. Through targeted training, its performance in ultrasound science popularization tasks is targeted and improved, ensuring that the generated answers are both professional and easy to understand, truly meeting the science popularization needs of patients. In this embodiment, during the instruction fine-tuning stage, the model is further optimized to enable it to generate more natural, accurate, and patient-friendly science popularization answers: (1) Efficient LoRA instruction fine-tuning strategy: This invention innovatively uses the model pre-trained in step 4, utilizes high-quality ultrasound field instruction fine-tuning data, and adopts LoRA low-rank adaptation technology to perform efficient instruction fine-tuning on the pre-trained model. The instruction fine-tuning script parameters are as follows: Figure 6 As shown in the figure, the LoRA fine-tuning settings fully consider computing resource constraints and model performance requirements, achieving a balance between efficiency and effectiveness. Through instruction fine-tuning, the model can learn to understand the user's question intentions in ultrasound science popularization scenarios and generate high-quality science popularization answers that meet the instruction requirements, significantly improving the model's practicality in the field of ultrasound science popularization. (2) Targeted optimization of patient-friendly science popularization capabilities: In the instruction fine-tuning stage, this embodiment focuses on improving the model's patient-friendly science popularization capabilities. Through carefully designed instruction data, the model is guided to learn how to translate professional medical knowledge into easy-to-understand language close to the patient's understanding level, ensuring that the answers generated by the model can truly help patients understand ultrasound reports and eliminate patients' comprehension barriers and anxiety. Step 6: User questions. To improve the user experience, this embodiment designs a user-friendly human-computer interaction interface and integrates a trained large language model in the ultrasound field to build a complete ultrasound examination result question-answering system. The user questions and model answers are shown in the following figure. Figure 7 As shown in the figure, users can easily ask any questions about their ultrasound reports through a simple text input box, or directly upload the ultrasound report content. The system will automatically parse the content, extract key information for subsequent processing, and quickly and accurately give professional and easy-to-understand responses. In addition to directly answering users' questions, the system will also provide an easy-to-understand summary of the results, which includes all important findings and corresponding medical explanations, further enhancing the user experience. This embodiment also uses the llm-factory framework to ensure the stability, efficiency and scalability of the system operation.

[0028] In some embodiments, the above-mentioned ultrasound examination result question-and-answer popular science method based on a large language model can also be implemented in the following manner.

[0029] This example obtains real ultrasound examination reports covering organs such as the heart, liver, and kidneys from hospitals and ultrasound institutions, as well as relevant case discussions and educational materials extracted from open medical databases. The dataset is prepared and preprocessed using optical character recognition (OCR) technology to unify the image and text data formats, and data augmentation technology is used to expand question-and-answer scenarios. The dataset is cleaned to remove privacy information and erroneous records, and low-quality data is deleted to ensure data legitimacy and reliability. Ultrasound-specific question and answer instruction data is generated using the GPT-4 API, and a fine-tuned instruction set covering ultrasound examination terminology and frequently asked questions is constructed to enhance the model's practicality and accuracy. High-performance open-source large language models (such as LLaMA3 and ChatGLM3) are selected as the base model and pre-trained using LoRA technology. Low-rank parameter optimization is used to master basic ultrasound knowledge. The model instructions are fine-tuned based on the generated instruction dataset to further optimize the quality of the model's answers, making the generated content both professional and easy to understand. The present invention allows users to input relevant questions, and the system parses and generates targeted popular science answers. This system addresses the difficulty of understanding the professional terminology in ultrasound examination reports, providing patients with efficient and personalized medical science services.

[0030] As an exemplary embodiment, the above-mentioned ultrasound examination result question-and-answer popular science method based on a large language model can also be implemented in the following manner. This embodiment is implemented through computer software technology and designs a series of steps from data collection, model training to user interaction to ensure that the system can provide services to users efficiently and accurately. The ultrasound examination result question-and-answer popular science method based on a large language model in this embodiment includes: Step 1: Obtain a large number of real ultrasound examination reports from hospitals and ultrasound institutions. These reports cover examination results of multiple organs such as the heart, liver, and kidneys, and include detailed information such as image annotations, text analysis written by doctors, and diagnostic conclusions; 1. Integration of public medical databases and popular science resources: We systematically crawl and organize ultrasound-related case discussions, medical educational materials, professional research reports, and high-quality popular medical articles from authoritative open medical databases (such as MedlinePlus and MayoClinic) and popular medical science websites (such as DXY.com and UpToDate). This data volume exceeds 50,000 entries, effectively enriching the breadth and depth of the dataset. Case discussion data includes professional content such as physicians' analysis of ultrasound reports for typical cases, diagnostic strategies, and key points for differential diagnosis. Educational material data includes medical knowledge such as the basic principles of ultrasound examinations, explanations of common terminology, and ultrasound imaging characteristics of diseases. Popular science article data includes patient-oriented content such as ultrasound report interpretations, disease education, and health education.

[0031] 2. OCR-based text processing of image information: Considering that some ultrasound examination reports contain image information, such as annotations and structural labels of ultrasound examination images, in order to make full use of image information, we innovatively use high-precision OCR scanning tools (such as PaddleOCR, EasyOCR, etc.) to scan images and convert them into text. For example, arrows, annotation boxes, text annotations, etc. in the image are converted into text descriptions, which effectively integrates image information and improves the multimodal information content of the dataset.

[0032] Step 2: Implementation of a high-quality data cleaning process. To ensure the high quality of the dataset, a rigorous, multi-step data cleaning process was implemented: Multiple rounds of privacy information filtering were performed on ultrasound examination report samples using various techniques, such as regular expression matching, medical term recognition, and named entity recognition, combined with manual review. This filtering completely removed all types of personal privacy information, including patient name, age, medical record number, contact information, and address, ensuring that data use was completely legal and compliant. Multi-dimensional dirty data processing was performed. Detailed dirty data cleaning rules were developed to address various dirty data issues in the dataset, such as spelling errors, misuse of medical terminology, incomplete sentences, illogicality, and inconsistencies between image annotations and text descriptions. A team of medical experts was organized to conduct sample review and professional correction of the data to ensure its medical accuracy and professionalism. Multi-standard low-quality data screening was implemented. For example, criteria for low-quality data screening were established, such as unclear imaging data, obvious errors in medical interpretation, missing key information in the report content, and reports that did not conform to semantic logic or had no clinical significance. Using a combination of automated scripts and manual review, the dataset was subjected to multiple rounds of low-quality data screening to ensure the high quality and reliability of the final dataset.

[0033] Step 3: Combine the GPT-4 API to generate ultrasound-specific instruction data to cover common user demand scenarios; use the GPT-4 API to generate a series of ultrasound-specific question and answer instruction data (self-instruction method); 1. Prompt template optimization: In order to guide the GPT-4 API to generate high-quality instruction data, the present invention innovatively designs and continuously iteratively optimizes the Prompt template (such as Figure 3 As shown); the initial Prompt template focuses on guiding the GPT-4 API to generate common ultrasound examination questions and popular science answers, such as "Please generate popular science questions and answers about the term 'XXX' in ultrasound examination reports"; the iteratively optimized Prompt template places more emphasis on the quality, diversity and effectiveness of instruction data. For example, more refined instructions such as "Please generate high-quality, patient-friendly, accurate, and easy-to-understand popular science questions and answers about ultrasound examination reports", "Please think from the patient's perspective and consider the questions that patients may have about ultrasound reports", "Please ensure that the generated answers are easy to understand and avoid using overly professional medical terms", and "Please generate different types of popular science questions and answers for ultrasound examinations, such as term explanation, etiology analysis, risk assessment, and treatment recommendation" are added to the Prompt to guide the GPT-4 API to generate instruction data that better meets the requirements of the present invention; through multiple iterative optimizations, the final Prompt template can effectively improve the quality and relevance of instruction data generated by the GPT-4 API; 2. Self-Instruct instruction set generation based on the GPT-4 API: Using the final preferred Prompt template and the high-quality ultrasound dataset constructed in step 1, the GPT-4 API is used to generate Self-Instruct instruction data on a large scale and in batches. The specific generation process is as follows: the report text, popular science articles, medical education materials, etc. in the ultrasound dataset are input into the GPT-4 API. The Prompt template serves as an instruction, guiding the GPT-4 API to simulate patient questions based on the input text content and generate popular science answers that meet the instruction requirements. For example, if a description of an ultrasound report on fatty liver is input, the GPT-4 API will generate questions that patients may ask, such as "How is fatty liver described in the ultrasound report?" and "Is fatty liver serious? Does it require treatment?" based on the Prompt template, and generate corresponding popular science answers based on the input text content. Through this Self-Instruct method, this embodiment constructs a high-quality, large-scale ultrasound instruction dataset at low cost and high efficiency, with a data volume of more than 20,000, which meets the data requirements of the instruction fine-tuning stage. 3. Injection of medical domain knowledge and instruction expansion into general instruction datasets: To improve the versatility and anti-forgetting ability of the model and further expand the scale and diversity of instruction data, this embodiment also integrates and expands existing general fine-tuning instruction sets (such as GPT4-Alpaca and BELLE) with professional knowledge in the ultrasound field. The specific method is as follows: from the general instruction dataset, filter out instruction data related to the medical and health field, such as "healthy diet recommendations" and "disease prevention knowledge"; use the medical knowledge base and GPT-4 The API combines the filtered general command data with professional terminology in the ultrasound field and frequently asked user questions to rewrite and expand the commands. For example, the general command "Please give me some healthy eating advice" is rewritten as "What should patients with fatty liver pay attention to in their diet? Please provide popular science information based on the ultrasound examination report." The general command "Introduce the prevention knowledge of hypertension" is rewritten as "What should hypertensive patients pay attention to when undergoing ultrasound examinations? Can ultrasound detect hypertension?" etc. Through this knowledge injection and command expansion method, based on the general command dataset, more than 10,000 high-quality ultrasound field command data are expanded and generated, further enriching the diversity and knowledge coverage of the command dataset, and effectively improving the comprehensive performance of the model in professional fields and general scenarios.

[0034] Step 4: Pre-training the large language model in the ultrasound field. This step aims to use high-quality ultrasound datasets to pre-train the selected basic large language model with domain knowledge, enabling it to initially master the specialized terminology and knowledge in ultrasound medicine, laying the foundation for subsequent instruction fine-tuning. The specific implementation details are as follows: 1. Basic model selection and LoRA loading: Taking into account factors such as model performance, open source protocol, community activity, and training and deployment costs, this embodiment prefers high-performance open source large language models such as LLaMA3-8B, ChatGLM3-6B, and Qwen2-7B as basic models; these models have excellent performance in general language capabilities and downstream task performance, high community activity, easy access and use, and friendly open source protocols, allowing commercial use; compared with closed source models, open source models have higher controllability and customizability, and are more suitable for the research and application requirements of this invention; load the pre-trained weights of the selected basic model and initialize LoRA (Low-Rank Adaptation) low-rank adapter, the rank (r) of LoRA is set to 32 to ensure that the LoRA adapter has sufficient low-rank representation capability to effectively capture and learn ultrasound domain knowledge; the scaling factor (α) is set to 16 to balance the stability of training and the ability of the model. The larger the α value, the stronger the effect of the LoRA adapter, and vice versa; Dropout is set to 0.05 to avoid overfitting and improve the generalization ability of the model; the LoRA adapter is inserted into the key layers of the model Transformer layer, such as the q_proj and v_proj matrices of the Self-Attention module; the schematic diagram of the LoRA method is shown in the figure. Figure 4 As shown; 2. LoRA pre-training based on ultrasound dataset: Load the high-quality ultrasound dataset cleaned in steps 1 and 2, and perform domain pre-training on the basic model loaded with the LoRA adapter using the causal language modeling method. The pre-training script parameters are as follows: Figure 5 As shown in the figure; the goal of pre-training is to enable the model to learn and master the professional terminology, knowledge representation and language patterns in the field of ultrasound, so that it has the initial ability to understand and generate text in the field of ultrasound; during the pre-training process, all the original weight parameters of the base model are frozen, and only the low-rank parameters in the LoRA adapter are trained, which significantly reduces the computational cost and time cost of pre-training; the AdamW optimizer is used, the learning rate is set to 2e-5, the BatchSize is set to 128, and the training Epoch is set to 3 rounds; distributed training frameworks such as DeepSpeed are used to accelerate the training process; after the pre-training is completed, the pre-trained LoRA weights are saved for use in the subsequent instruction fine-tuning stage.

[0035] Step 5: Model instruction fine-tuning. This step aims to use a high-quality ultrasound instruction dataset to fine-tune the pre-trained model, enabling it to understand and follow user instructions and generate high-quality, patient-friendly ultrasound science Q&A. The specific implementation details are as follows: 1. LoRA efficient instruction fine-tuning strategy: Based on the model pre-trained in step 4, load the high-quality ultrasound field instruction fine-tuning data generated in step 2, and use LoRA (Low-Rank Adaptation) low-rank adaptation technology to perform efficient instruction fine-tuning on the pre-trained model. The instruction fine-tuning script parameters are as follows: Figure 6 As shown in the figure, the LoRA fine-tuning settings fully consider the computing resource constraints and model performance requirements: the rank (r) of LoRA is set to 16, which is lower than that in the pre-training stage to further reduce the computing requirements and speed up the fine-tuning speed; the scaling factor (α) is set to 8 to further control the amplitude of parameter updates and prevent overfitting; the Dropout is set to 0.1, which is higher than that in the pre-training stage to adapt to more complex user question-and-answer scenarios and improve the generalization ability of the model; the pre-trained model is loaded and initialized based on the LoRA weights saved in step 3, the main model weights are frozen, and only the inserted LoRA layer is optimized to ensure the efficiency and stability of fine-tuning; 2. Targeted fine-tuning of patient-friendly science popularization capabilities: During the instruction fine-tuning stage, focus on the targeted improvement of the model's patient-friendly science popularization capabilities; the loss function uses cross-entropy loss (Cross-Entropy Loss), the learning rate is set to 1e-4, and the Cosine Annealing scheduling strategy is adopted. The learning rate is dynamically adjusted during the training process to improve the stability and convergence speed of model training; the training round is set to 5 epochs to ensure that the model can fully learn the knowledge and instruction paradigm in the instruction data; during the training process, the model's performance indicators on the validation set, such as question-answering accuracy, answer fluency, patient-friendliness, etc., are continuously monitored, and manual evaluation is performed, and the fine-tuning strategy and parameters are adjusted in a timely manner to ensure that the model's final science popularization question-answering capabilities can meet user needs; after the instruction fine-tuning is completed, the final fine-tuned LoRA weights are saved.

[0036] Step 6: User Questioning and System Interaction Services. This step aims to build a user-friendly human-computer interaction interface and integrate a trained large language model for ultrasound to provide users with a convenient and efficient Q&A service for ultrasound examination results. The specific implementation details are as follows: 1. Model loading and deployment based on the llm-factory framework: The llm-factory framework (an efficient and easy-to-use large language model deployment and inference framework) is used to load the large language model in the ultrasound field that has been pre-trained and fine-tuned by instructions; the llm-factory framework provides a high-performance model inference engine, a flexible API interface, and complete model management and monitoring functions to ensure the stability, efficiency and scalability of the system operation; the model is deployed on a cloud server or a local server, and an API interface is provided to the outside world for the user to call the interactive interface; users can enter questions or upload ultrasound report content in the text box, and the system will automatically parse the key information and give corresponding answers. The user's questions and the model's answers are as follows: Figure 7 As shown; for example, a user asks: "My breast ultrasound test shows 'hypoechoic', what does this mean?" The model responds: "Hello, in breast ultrasound testing, 'hypoechoic' refers to an area on the ultrasound image that shows a lower (darker) echo than the surrounding tissue. This is an imaging finding used to describe certain characteristics of breast tissue, but it cannot directly indicate whether it is benign or malignant. A comprehensive judgment needs to be made in combination with other examination results and medical history. There is no need to be overly nervous about the term 'hypoechoic'. It is only descriptive language, and the specific situation requires further judgment by the doctor." This answer is based on professional ultrasound knowledge and provides scientific and reliable information. At the same time, the language expression is close to user needs, avoiding the accumulation of professional terms, making it easier for ordinary patients to understand and effectively alleviating patients' anxiety.

[0037] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A method for popularizing ultrasound examination results through question-answering based on a large language model, characterized in that: The following steps are involved: S1: Perform data augmentation on the initial training data to obtain a high-quality dataset in the ultrasound field; S2: performing data cleaning on the high-quality ultrasound dataset to obtain a training dataset; S3: Based on the general fine-tuning instruction set, use the GPT-4 API tool to obtain the ultrasound field instruction dataset; S4: Pre-training the large language model using a low-rank adaptation method based on the training data set to obtain a pre-trained large language model; S5: fine-tuning the pre-trained large language model to obtain a large model for ultrasound examination result question answering; S6: Use the ultrasound examination result question-answering model to answer user questions.

2. The ultrasound examination result question-answering popular science method based on a large language model according to claim 1, characterized in that: The initial training data includes ultrasound examination report samples, which include image annotations and text descriptions; the data enhancement of the initial training data includes converting medical synonyms and professional terminology expansion of case descriptions, and converting colloquial descriptions into professional medical terminology expressions.

3. The ultrasound examination result question-answering popular science method based on a large language model according to claim 1, characterized in that: The data cleaning of the high-quality ultrasound data set includes privacy filtering, dirty data processing and low-quality data screening.

4. The ultrasound examination result question-answering popular science method based on a large language model according to claim 1, characterized in that: Step S3 specifically includes: adding refined instructions to the Prompt template based on the general fine-tuning instruction set, and using the GPT-4 API tool to generate an ultrasound field instruction dataset in a self-guided instruction manner.

5. The ultrasound examination result question-answering popular science method based on a large language model according to claim 1, characterized in that: Step S4 specifically includes: inserting a LoRA adapter into the key layer of the large language model, setting the rank value of the LoRA adapter to the first rank value, setting the scaling factor of the LoRA adapter to the first scaling factor, setting the random inactivation ratio of the LoRA adapter to the first random inactivation ratio, optimizing only low-rank parameters, freezing other model weights, so that the model maintains its original performance and adapts to new tasks; according to the training data set, pre-training the large language model in an autoregressive form, saving the pre-trained LoRA weights, and obtaining a pre-trained large language model.

6. The ultrasound examination result question-answering popular science method based on a large language model according to claim 5, characterized in that: The large language model is ChatGLM3, the first rank value is 32, the first scaling factor is 16, and the first random deactivation ratio is 0.

05.

7. The ultrasound examination result question-answering popular science method based on a large language model according to claim 1, characterized in that: Step S5 specifically includes: setting the rank value of the LoRA adapter to the second rank value, setting the scaling factor of the LoRA adapter to the second scaling factor, setting the random deactivation ratio of the LoRA adapter to the second random deactivation ratio, optimizing only the inserted LoRA layer, and keeping the basic model weight unchanged; loading the pre-trained model, initializing based on the saved LoRA weights, and freezing the master weights.

8. The ultrasound examination result question-answering popular science method based on a large language model according to claim 7, characterized in that: The second rank value is 16, the second scaling factor is 8, the second random dropout ratio is 0.1, the preset learning rate is 1e-4, and the preset round number is 5.

9. A Q&A system for popularizing ultrasound examination results based on a large language model, characterized by: The system includes the following modules: The data enhancement module is configured to: perform data enhancement on the initial training data to obtain a high-quality dataset in the ultrasound field; A data cleaning module is configured to: perform data cleaning on the high-quality ultrasound data set to obtain a training data set; The ultrasound field instruction data generation module is configured to: obtain the ultrasound field instruction dataset using the GPT-4 API tool based on the general fine-tuning instruction set; A pre-training module is configured to: pre-train the large language model using a low-rank adaptation method according to the training data set to obtain a pre-trained large language model; An instruction fine-tuning module is configured to: perform instruction fine-tuning on the pre-trained large language model to obtain a large model for ultrasound examination result question answering; The user question module is configured to answer user questions using the ultrasound examination result question-answering model.

Images