Medical treatment guide model training method and system, terminal and medium

Abstract

The invention relates to the field of medical model training, and particularly discloses a medical treatment guide model training method and system, a terminal and a medium, and the method comprises the steps: carrying out the incremental pre-training of a selected large language model through priori knowledge, and obtaining a basic large language model; acquiring a public real doctor-patient dialogue data set; constructing a doctor-patient dialogue template, and generating and constructing a doctor-patient dialogue data set by using a third-party large language model based on the doctor-patient dialogue template; combining the real doctor-patient dialogue data set and the constructed doctor-patient dialogue data set into a data set, and recording the data set as an initial doctor-patient dialogue data set; performing duplicate removal preprocessing on the initial doctor-patient dialogue data set to obtain a doctor-patient dialogue training data set; and using the doctor-patient dialogue training data set to carry out supervision fine tuning training on the basic large language model to obtain a doctor-seeing and hospital-guiding model. According to the invention, the joint training data set is formed based on the public data set and the automatic construction data set, so that the accuracy of the medical treatment guide model is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical model training, and in particular to a medical consultation model training method, system, terminal and medium. Background Art

[0002] Traditional medical guidance mainly relies on manual preliminary triage and department recommendation. Patients usually need to describe their symptoms through face-to-face consultation, and doctors or nurses recommend the corresponding department and treatment based on their experience. This method has the following problems and disadvantages: low efficiency, face-to-face consultation takes a long time, especially during peak hours, patients need to wait a long time to get a preliminary diagnosis and department recommendation. Accuracy depends on individual experience. Different medical staff have different levels of experience and knowledge, which may lead to inconsistent recommendation results and affect the accuracy of diagnosis and treatment. Uneven resource utilization. Due to the lack of accurate guidance, some departments may be overcrowded, while other departments' resources are not fully utilized, affecting the overall allocation efficiency of medical resources.

[0003] With the rapid development of artificial intelligence technology, natural language processing (NLP) technology is increasingly being used in the field of medical health. As an important tool to improve the efficiency of medical services and patient experience, the medical guidance system can improve the efficiency of medical guidance for patients. However, traditional medical guidance systems often rely on manually designed rules and templates, which are difficult to handle complex and changeable doctor-patient dialogue scenarios, affecting the accuracy of medical guidance. Summary of the invention

[0004] To solve the above problems, the present invention provides a medical guidance model training method, system, terminal and medium, which constitute a joint training data set based on a public data set and an automatically constructed data set to train the medical guidance model and improve the accuracy of the medical guidance model.

[0005] In a first aspect, the technical solution of the present invention provides a medical guidance model training method, comprising the following steps: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Obtain a public dataset of real doctor-patient conversations; Build a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; The real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset are merged into one dataset, which is recorded as the initial doctor-patient dialogue dataset; Perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; The doctor-patient dialogue training dataset is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

[0006] In an optional implementation, obtaining a public dataset of real doctor-patient conversations specifically includes: Use web crawler technology to collect doctor-patient conversation data from the target website. The crawled data should include department information, conversation content, timestamp, and user ID; Preprocess the crawled data, including data cleaning by removing HTML tags, special characters, and repeated information, converting the text into the target format, and performing dialogue segmentation on the standardized text; Establish a department name mapping table and use fuzzy matching algorithm to match department information in crawled data to standard department names; The crawled data is stored in a standard format, which includes conversation content, timestamp, user ID, and standard department name.

[0007] In an optional implementation, a doctor-patient dialogue dataset is generated and constructed based on the doctor-patient dialogue template using a third-party large language model, specifically including: Design corresponding dialogue scenarios and prompts for the target departments of the target hospitals based on the doctor-patient dialogue template; Input the designed dialogue scenarios and prompts into the selected third-party large language model to generate the doctor-patient dialogue content; Preprocess the generated doctor-patient dialogue content, including post-processing, cleaning and data enhancement; The preprocessed doctor-patient conversation content is generated to construct a doctor-patient conversation dataset.

[0008] In an optional implementation, the preprocessed doctor-patient conversation content is generated to construct a doctor-patient conversation dataset, specifically including: Input each dialogue text in the doctor-patient dialogue content generated by the third-party large language model into another third-party large language model to generate new doctor-patient dialogue content; Compare the content similarity and semantic consistency of the two doctor-patient conversations; The doctor-patient dialogue contents with content similarity greater than the similarity threshold and semantic consistency score greater than the consistency threshold are retained; All doctor-patient conversation contents are retained to construct a doctor-patient conversation dataset.

[0009] In an optional implementation, the initial doctor-patient dialogue data set is preprocessed to remove duplicates to obtain a doctor-patient dialogue training data set, specifically including: Convert each conversation text in the initial doctor-patient conversation dataset into a MinHash signature; Calculate the Jaccard similarity between each MinHash signature; If the Jaccard similarity of two dialogue texts is greater than the Jaccard similarity threshold, one of the dialogue texts is retained; The deduplicated conversation texts are used to form a doctor-patient conversation training dataset.

[0010] In an optional implementation, the basic large language model is supervised and fine-tuned using the doctor-patient dialogue training dataset to obtain a medical guidance model, specifically including: Use stratified approach to divide the doctor-patient dialogue training dataset into training set, validation set and test set; Based on the training set, validation set, and test set, the QLoRA strategy is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

[0011] In a second aspect, the technical solution of the present invention provides a medical guidance model training system, comprising: Basic large language model training module: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Real dialogue acquisition module: obtains public real doctor-patient dialogue dataset; Construct a dialogue acquisition module: construct a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; Dialogue dataset construction module: merge the real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset into one dataset, recorded as the initial doctor-patient dialogue dataset; Training data acquisition module: perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; Model training module: Use the doctor-patient dialogue training dataset to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

[0012] In a third aspect, the technical solution of the present invention provides a terminal, including: A memory, used for storing a medical guidance model training program; A processor is used to implement the steps of the medical guidance model training method as described in any of the above items when executing the medical guidance model training program.

[0013] In a fourth aspect, the technical solution of the present invention provides a computer-readable storage medium, on which a medical guidance model training program is stored. When the medical guidance model training program is executed by a processor, the steps of the medical guidance model training method as described in any of the above items are implemented.

[0014] The present invention provides a medical guidance model training method, system, terminal and medium, which have the following beneficial effects compared with the prior art: the intelligent guidance model is trained by automatically constructing doctor-patient dialogue data through the model, which not only greatly enriches the diversity and authenticity of the training data, effectively reduces the labor cost, but also significantly enhances the model's ability to understand and cope with complex scenarios, while protecting patient privacy, promoting rapid iteration and optimization of the model, and thus improving the accuracy of the model's medical guidance. The guidance model is trained based on a large number of real data sets and constructed data sets, and the accuracy exceeds the experience judgment of ordinary guides, further improving the accuracy of the model's medical guidance. The model is allowed to continue to absorb new medical field data for training on the basis of existing large-scale pre-training, so that the model can continuously learn the latest medical knowledge, case characteristics, etc., thereby deepening the understanding and generalization ability of the medical field, helping the model to adapt to various complex clinical scenarios, improve the ability to identify diseases, and thus improve the accuracy of the model's medical guidance. On the basis of incremental pre-training, the model is fine-tuned using labeled doctor-patient dialogue data for medical guidance tasks, which can improve the guidance ability of the model. In addition, supervised fine-tuning ensures that the model can not only generalize to a wide range of medical knowledge, but also make accurate judgments in different scenarios and output results in accordance with specific format requirements, thereby achieving the transformation from general knowledge to professional applications, thereby improving the accuracy of the model's medical guidance. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a flow chart of a medical consultation guidance model training method provided by an embodiment of the present invention.

[0016] Figure 2 The present invention provides a schematic block diagram of a medical consultation model training system structure.

[0017] Figure 3 It is a schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0020] The collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of the case data involved in the technical solution disclosed in this application comply with the provisions of relevant laws and regulations, are used for legal purposes and do not violate public order and good morals, and necessary measures are taken to prevent the illegal use of case data to maintain case data security, network security and national security.

[0021] Figure 1 This is a flow chart of a medical guidance model training method provided by an embodiment of the present invention. Figure 1 The execution subject may be a medical guidance model training system. The medical guidance model training method provided in the embodiment of the present invention is executed by a computer device, and accordingly, the medical guidance model training system runs in the computer device. According to different requirements, the order of the steps in the flowchart can be changed, and some can be omitted.

[0022] The medical guidance model training method provided in this embodiment is based on joint training of public data sets and automatically constructed data sets, and at the same time, the guidance process is automated and intelligent through incremental pre-training and supervised fine-tuning two-stage training models. First, the system pre-configures various medical data, including doctor-patient dialogue data, department description data, etc. The model automatically constructs training data, which greatly enriches the diversity and authenticity of training data, reduces labor costs, and enhances the model's ability to understand and cope with complex scenarios. At the same time, the system ensures the protection of patient privacy. In the incremental pre-training stage, the open source large language model is trained using data unique to the medical field to better adapt to the language and knowledge structure of the medical field. Incremental pre-training helps the model to continuously learn the latest medical knowledge and case characteristics, and improves the model's adaptability to complex clinical scenarios. In the supervised fine-tuning stage, the model is fine-tuned using automatically generated doctor-patient dialogue data. By designing a task fine-tuning strategy, the system can optimize the model's guidance performance according to different training data volumes and hardware resources. The model can automatically recommend appropriate departments based on the input symptom descriptions and medical records, and give preliminary diagnostic suggestions. Finally, this embodiment realizes the automation, standardization and intelligence of medical guidance through the combination of incremental pre-training and supervised fine-tuning, improves the efficiency and accuracy of medical guidance, and enhances the patient's medical experience.

[0023] like Figure 1 As shown, the method includes the following steps.

[0024] S1, use prior knowledge to perform incremental pre-training on the selected large language model to obtain the basic large language model.

[0025] Exemplarily, there are two types of prior knowledge. The first is the descriptive data of different departments and specialties collated by medical experts, and the second is the real-world electronic medical record text obtained from cooperative medical institutions. The prior knowledge about hospital departments is mainly collected by medical experts using these two aspects of data to enhance the model. It should be noted that the collected instructions can be processed into a standard format by technical personnel, such as {"Department":"Endocrinology Department", "Description":"The Endocrinology Department is mainly responsible for the diagnosis and treatment of endocrine system diseases such as diabetes, thyroid diseases, and adrenal diseases. The department provides insulin treatment, thyroid function testing, hormone replacement therapy and other services.", "Medical Text":"Patient Li, female, 63 years old, diagnosed with hypothyroidism, recommended thyroid hormone replacement therapy."}.

[0026] Select a suitable large language model. When selecting a model, mainly refer to the model's Chinese processing capabilities, the medical knowledge it contains, and the model's demand for hardware resources. Then perform incremental pre-training on the selected large language model, that is, use data unique to the medical field (the acquired prior knowledge) to perform incremental pre-training on the selected large language model to make it better adapt to the language and knowledge structure of the medical field. After incremental pre-training the selected large language model using prior knowledge, a basic large language model with the language and knowledge structure of the medical field is obtained.

[0027] S2, obtain a public dataset of real doctor-patient conversations.

[0028] The purpose of this step is to obtain accurate real-world public doctor-patient conversation data. Doctor-patient conversation data can be collected through public medical websites, and the department names in the public data set are mapped to the standard department names that need to be recommended.

[0029] S2.1, use web crawler technology to collect doctor-patient conversation data from the target website. The crawled data should include department information, conversation content, timestamp, and user ID.

[0030] First, determine the data source, including mainstream public Chinese medical websites, ensure that the website allows data crawling, and comply with the website's terms and conditions, including crawling policies and privacy regulations. Then use web crawler technology to collect doctor-patient conversation data from the target website. The crawled data should include public anonymized department information, conversation content, timestamps, user IDs and other information. This information has been anonymized and does not contain personal sensitive information.

[0031] S2.2, preprocess the crawled data.

[0032] Data cleaning: Remove noise data such as HTML tags, special characters, and repeated information.

[0033] Text standardization: convert text to lowercase, unify word spellings, handle abbreviations and dialects, etc.

[0034] Dialogue segmentation: Segment the dialogue text into independent sentences or dialogue turns.

[0035] S2.3, establish a department name mapping table and use the fuzzy matching algorithm to match the department information in the crawled data to the standard department name.

[0036] Collect a list of standard department names and match them with department names in the public dataset to build a department name mapping table. Since department names may differ, use a fuzzy matching algorithm (such as Levenshtein distance) for matching. After the automatic matching is completed, manual proofreading is performed: Manual proofreading of the matching results to ensure accuracy.

[0037] S2.4, store the crawled data in a standard format, which includes conversation content, timestamp, user ID, and standard department name.

[0038] Store the preprocessed data in a standard format, such as JSON or CSV. The data format should contain information such as conversation content, timestamp, user ID, and standard department name.

[0039] It should be noted that the medical guidance model outputs the department according to the content of the conversation. When training the medical guidance model, it is necessary to configure labels for the samples, that is, the department to which each conversation text should be output. This step establishes a department name mapping table to facilitate the creation of labels for public doctor-patient conversation texts.

[0040] S3, build a doctor-patient dialogue template, and use a third-party large language model to generate and build a doctor-patient dialogue dataset based on the doctor-patient dialogue template.

[0041] First, we design a prompt template that is automatically constructed for the doctor-patient dialogue dataset: based on the big model’s understanding of natural language, we clarify the goal of this big language model question and answer, the scenario of the question and answer, multiple detailed requirements for the question and answer content, and the output expression format requirements and other information items.

[0042] For example, the prompt template is "You are a senior physician in a tertiary-level {hospital_type} hospital, and you are now required to construct a batch of dialogue data for training the patient guidance model. \n When constructing data, please follow the following case to construct {sample_count} training samples.

[0043] \n Note: 1. Please ensure that the constructed samples have large differences; \n 2. The range of the role of the dialogue is: [patient, patient's parents, patient's children, doctor, patient's family, patient's children, patient's daughter, patient's son]; \n 3. The number of rounds of dialogue can be single or multiple rounds, or only the patient's description; \n 4. Please output the constructed samples in json format; \n 5. The guidance department can only be one or more of the following department list; \n 6. When selecting multiple departments, please use the symbol '|' to splice them; \n 7. Each department in the department list is selected at least once\n Department list: {deparments} \n Case: {ref_samples}".

[0044] Then determine the target hospital and department types: Determine the types of hospitals (such as public hospitals, private hospitals, specialized hospitals, etc.) and department types (such as internal medicine, surgery, pediatrics, gynecology, etc.) that need to be covered. Consider the distribution of hospital and department types to ensure the representativeness of the data set.

[0045] Then choose a suitable third-party large language model: Choose a third-party large language model that supports dialogue generation, such as Tongyi Qianwen, Zhipu Qingyan's ChatGLM, kimi large model, etc. Main considerations: model performance, accessibility, cost and other factors.

[0046] Then design the dialogue scenarios and prompts: Design corresponding dialogue scenarios and prompts for each department type to guide the model to generate dialogue content that conforms to the actual situation. The prompts contain information such as the patient's symptoms, questions, needs, and the doctor's role and expertise.

[0047] Further use the API to automatically generate conversation data: Use the selected third-party large language model API, input the designed conversation scenarios and prompts, and automatically generate doctor-patient conversation data. It is necessary to set parameters such as the length and number of generated conversations to control the scale and quality of the data.

[0048] Finally, the data is post-processed, cleaned, and enhanced.

[0049] Specifically, this step includes the following process.

[0050] S3.1, design corresponding dialogue scenarios and prompts for the target departments of the target hospitals based on the doctor-patient dialogue template.

[0051] It should be noted that the medical guidance model outputs the department according to the content of the conversation. When training the medical guidance model, it is necessary to configure labels for the samples, that is, the department to which each conversation text should be output. This step sets conversation scenes and prompts for different departments of different hospitals, which can facilitate the creation of labels for public doctor-patient conversation texts.

[0052] S3.2, input the designed dialogue scenario and prompts into the selected third-party large language model to generate the doctor-patient dialogue content.

[0053] S3.3, pre-process the generated doctor-patient dialogue content, including post-processing, cleaning and data enhancement.

[0054] Data post-processing and cleaning: Check the generated conversation data and remove duplicate, erroneous or illogical conversations. Format and standardize the conversation content, such as removing special characters and unifying word spellings.

[0055] Data enhancement: Use data enhancement techniques, such as synonym replacement, sentence rearrangement, etc., to increase the size and diversity of the dataset. Noise data can be added as needed to simulate the uncertainty and complexity of real conversations.

[0056] S3.4, construct a doctor-patient dialogue dataset by generating the preprocessed doctor-patient dialogue content.

[0057] To ensure the quality of the self-built data set, this embodiment uses multiple large model APIs to cross-validate the doctor-patient dialogues generated by the third-party large language model.

[0058] S3.4.1, input each dialogue text in the doctor-patient dialogue content generated by the third-party large language model into another third-party large language model to generate new doctor-patient dialogue content.

[0059] S3.4.2, compare the content similarity and semantic consistency of the two doctor-patient conversations.

[0060] S3.4.3, retain the doctor-patient dialogue content with content similarity greater than the similarity threshold and semantic consistency score greater than the consistency threshold.

[0061] S3.4.4, all retained doctor-patient conversation contents are generated to construct a doctor-patient conversation dataset.

[0062] Exemplarily, the conversation data generated by manufacturer A will be verified using the API of manufacturer B, and only data with consistent verification results will be retained. Generate doctor-patient conversation data using the large language model API of manufacturer A, ensuring that the generated data contains sufficient sample size and diversity, covering the target hospital and department types. Use the large language model API of manufacturer B to verify the conversation data generated by manufacturer A, including inputting each conversation text generated by manufacturer A into the API of manufacturer B and obtaining the corresponding output results. Compare the output results of manufacturer A and manufacturer B, such as the similarity and semantic consistency of the conversation content. Set a consistency threshold for the verification results. Only retain data with consistent output results from manufacturer A and manufacturer B, and remove inconsistent data. The consistency threshold can be adjusted according to actual conditions, for example, a similarity threshold, a semantic consistency score, etc. can be set. Merge the verified and screened data with the public corpus dataset.

[0063] S4, merge the real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset into one dataset, recorded as the initial doctor-patient dialogue dataset.

[0064] The real doctor-patient dialogue dataset obtained in step 2 and the constructed doctor-patient dialogue dataset generated by a third-party large language model in step 3 are merged to obtain a preliminary joint dataset.

[0065] S5, performing deduplication preprocessing on the initial doctor-patient dialogue data set to obtain a doctor-patient dialogue training data set.

[0066] Since the constructed data is automatically generated by the model, there may be great similarity between the data. Therefore, this embodiment uses the MinHash algorithm to merge and remove duplicates of the automatically constructed data set and the public corpus data set obtained from the Internet, and finally obtain the training data of the model.

[0067] It should be noted that the automatically constructed dataset and the public corpus dataset are merged into a large dataset. If necessary, the merged dataset is checked to ensure that the data format is consistent and necessary preprocessing is performed.

[0068] S5.1, convert each conversation text in the initial doctor-patient conversation dataset into a MinHash signature.

[0069] To implement the MinHash algorithm using a MinHash library (such as datasketch), you first need to convert each conversation text in the merged dataset into a MinHash signature.

[0070] S5.2, calculate the Jaccard similarity between each MinHash signature.

[0071] S5.3, if the Jaccard similarity of two dialogue texts is greater than the Jaccard similarity threshold, one of the dialogue texts is retained.

[0072] S5.4, the deduplicated conversation texts are used to form a doctor-patient conversation training dataset.

[0073] Set the Jaccard similarity threshold, and treat conversation texts with Jaccard similarity exceeding the threshold as duplicates, and only keep one of them. During the process, you need to set appropriate MinHash parameters, such as the number of hash functions, signature length, etc., to control the accuracy and efficiency of deduplication.

[0074] S6, use the doctor-patient dialogue training dataset to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

[0075] S6.1, use stratified approach to divide the doctor-patient dialogue training dataset into training set, validation set and test set.

[0076] Use stratified sampling to divide the preprocessed doctor-patient conversation data into training, validation, and test sets to ensure the representativeness of the data distribution. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the model performance. Different split ratios can be used, such as 80% / 10% / 10% or 70% / 15% / 15%.

[0077] S6.2, based on the training set, validation set and test set, the basic large language model is supervised and fine-tuned using the QLoRA strategy to obtain the medical guidance model.

[0078] Format the dataset into a format suitable for QLoRA fine-tuning, such as converting the conversation text into input-output pairs, where the input is the patient consultation content and the output is the department or specialist recommended by the doctor.

[0079] Based on the basic language model trained in step 1, use the QLoRA library (such as qlo raft) to implement the QLoRA fine-tuning strategy. First, use the qlo raft library to load the pre-trained model. Second, use the qlo raft.Linear class to replace the linear layer that needs to be fine-tuned in the model. Then, set the hyperparameters of QLoRA, such as rank size, learning rate, number of training steps, etc. Next, use PyTorch for training, input the training data into the model, and calculate the loss function. Then, use an optimizer (such as Adam) to update the model parameters. During the training process, save the checkpoints of the model regularly for subsequent evaluation and deployment.

[0080] During the training process, the training set is used to train the model, and the validation set is used to adjust the model parameters. The test set is used to evaluate the model performance, and the corresponding indicators, such as accuracy, precision, recall, F1 value, etc., are recorded, and the results are evaluated and compared based on the F1 value.

[0081] Finally, according to the needs of the medical guidance, the prompts input to the medical guidance model and the corresponding model outputs can be designed. The medical guidance model is fine-tuned using the doctor-patient dialogue data to optimize the model's performance on the medical guidance task and output the results in the specified format. The optimal model configuration is found by adjusting hyperparameters such as the learning rate, batch size, and learning rate scheduling strategy.

[0082] Example of prompt template for medical consultation model: Based on the following conversation information between the patient and the doctor, please recommend a suitable department in the following department dictionary for consultation. \nDepartment dictionary: preventive health care department, general medical department, internal medicine, surgery, obstetrics and gynecology, women's health department, pediatrics, pediatric surgery, children's health department, ophthalmology, otolaryngology, stomatology, dermatology, medical cosmetology, psychiatry, infectious disease department, tuberculosis department, endemic disease department, oncology department, emergency medicine department, rehabilitation medicine department, sports medicine department, occupational disease department, hospice department, special medicine and military medicine department, anesthesia department, pain department, critical care medicine department, medical laboratory department, pathology department, medical imaging department, traditional Chinese medicine department, integrated traditional Chinese and Western medicine department, other business departments. \nConversation information: Patient's parents: My daughter has been coughing recently, especially at night, and she has a little fever. (Female, 5 years old) Doctor: Has the child been exposed to smoke or allergens? Patient's parents: The house has been renovated recently, and there may be some paint smell. Please recommend an appropriate department name based on the patient's symptoms and possible causes.

[0083] An embodiment of a medical guidance model training method is described in detail above. Based on the medical guidance model training method described in the above embodiment, an embodiment of the present invention also provides a medical guidance model training system corresponding to the method.

[0084] Figure 2 is a schematic block diagram of the structure of a medical guidance model training system provided by an embodiment of the present invention. In this embodiment, the medical guidance model training system 200 can be divided into multiple functional modules according to the functions it performs, such as Figure 2 The functional modules may include: a basic large language model training module 210, a real dialogue acquisition module 220, a constructed dialogue acquisition module 230, a dialogue data set construction module 240, a training data acquisition module 250 and a model training module 260. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory.

[0085] Basic large language model training module 210: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model.

[0086] The real dialogue acquisition module 220 is used to acquire a public real doctor-patient dialogue data set.

[0087] Constructing a dialogue acquisition module 230: constructing a doctor-patient dialogue template, and using a third-party large language model to generate and construct a doctor-patient dialogue dataset based on the doctor-patient dialogue template.

[0088] The dialogue data set construction module 240: combines the real doctor-patient dialogue data set and the constructed doctor-patient dialogue data set into one data set, which is recorded as the initial doctor-patient dialogue data set.

[0089] Training data acquisition module 250: performing deduplication preprocessing on the initial doctor-patient dialogue data set to obtain a doctor-patient dialogue training data set.

[0090] Model training module 260: Use the doctor-patient dialogue training data set to perform supervised fine-tuning training on the basic large language model to obtain a medical guidance model.

[0091] The medical guidance model training system of this embodiment is used to implement the aforementioned medical guidance model training method. Therefore, the specific implementation method of the system can be seen in the embodiment part of the medical guidance model training method in the previous text. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be introduced in detail here.

[0092] In addition, since the medical guidance model training system of this embodiment is used to implement the aforementioned medical guidance model training method, its function corresponds to that of the aforementioned method and will not be repeated here.

[0093] Figure 3A schematic diagram of the structure of a terminal 300 provided in an embodiment of the present invention includes: a processor 310, a memory 320 and a communication unit 330. The processor 310 is used to implement the following steps when implementing the medical guidance model training program stored in the memory 320: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Obtain a public dataset of real doctor-patient conversations; Build a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; The real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset are merged into one dataset, which is recorded as the initial doctor-patient dialogue dataset; Perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; The doctor-patient dialogue training dataset is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model. The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0094] The computer storage medium stores a medical consultation guidance model training program, and when the medical consultation guidance model training program is executed by the processor, the following steps are implemented: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Obtain a public dataset of real doctor-patient conversations; Build a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; The real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset are merged into one dataset, which is recorded as the initial doctor-patient dialogue dataset; Perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; The doctor-patient dialogue training dataset is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

[0095] The above disclosure is only a preferred embodiment of the present invention, but the present invention is not limited thereto. Any non-creative changes that can be thought of by a person skilled in the art, as well as several improvements and modifications made without departing from the principle of the present invention, should fall within the protection scope of the present invention.

Claims

1. A medical guidance model training method, characterized in that: The following steps are involved: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Obtain a public dataset of real doctor-patient conversations; Build a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; The real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset are merged into one dataset, which is recorded as the initial doctor-patient dialogue dataset; Perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; The doctor-patient dialogue training dataset is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

2. The medical guidance model training method according to claim 1, characterized in that: Obtain a public dataset of real doctor-patient conversations, including: Use web crawler technology to collect doctor-patient conversation data from the target website. The crawled data should include department information, conversation content, timestamp, and user ID; Preprocess the crawled data, including data cleaning by removing HTML tags, special characters, and repeated information, converting the text into the target format, and performing dialogue segmentation on the standardized text; Establish a department name mapping table and use fuzzy matching algorithm to match department information in crawled data to standard department names; The crawled data is stored in a standard format, which includes conversation content, timestamp, user ID, and standard department name.

3. The medical guidance model training method according to claim 1, characterized in that: Based on the doctor-patient dialogue template, a third-party large language model is used to generate and construct a doctor-patient dialogue dataset, including: Design corresponding dialogue scenarios and prompts for the target departments of the target hospitals based on the doctor-patient dialogue template; Input the designed dialogue scenarios and prompts into the selected third-party large language model to generate the doctor-patient dialogue content; Preprocess the generated doctor-patient dialogue content, including post-processing, cleaning and data enhancement; The preprocessed doctor-patient conversation content is generated to construct a doctor-patient conversation dataset.

4. The medical guidance model training method according to claim 3, characterized in that: The preprocessed doctor-patient dialogue content is used to construct a doctor-patient dialogue dataset, which includes: Input each dialogue text in the doctor-patient dialogue content generated by the third-party large language model into another third-party large language model to generate new doctor-patient dialogue content; Compare the content similarity and semantic consistency of the two doctor-patient conversations; The doctor-patient dialogue contents with content similarity greater than the similarity threshold and semantic consistency score greater than the consistency threshold are retained; All doctor-patient conversation contents are retained to construct a doctor-patient conversation dataset.

5. The medical guidance model training method according to any one of claims 1 to 4, characterized in that: The initial doctor-patient dialogue data set is preprocessed to remove duplicates to obtain the doctor-patient dialogue training data set, which specifically includes: Convert each conversation text in the initial doctor-patient conversation dataset into a MinHash signature; Calculate the Jaccard similarity between each MinHash signature; If the Jaccard similarity of two dialogue texts is greater than the Jaccard similarity threshold, one of the dialogue texts is retained; The deduplicated conversation texts are used to form a doctor-patient conversation training dataset.

6. The medical guidance model training method according to claim 5, characterized in that: The doctor-patient dialogue training dataset is used to supervise and fine-tune the basic large language model to obtain the medical guidance model, including: Use stratified approach to divide the doctor-patient dialogue training dataset into training set, validation set and test set; Based on the training set, validation set, and test set, the QLoRA strategy is used to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

7. A medical guidance model training system, characterized in that: include, Basic large language model training module: Use prior knowledge to perform incremental pre-training on the selected large language model to obtain a basic large language model; Real dialogue acquisition module: obtains public real doctor-patient dialogue dataset; Construct a dialogue acquisition module: construct a doctor-patient dialogue template, and use a third-party large language model to generate a doctor-patient dialogue dataset based on the doctor-patient dialogue template; Dialogue dataset construction module: merge the real doctor-patient dialogue dataset and the constructed doctor-patient dialogue dataset into one dataset, recorded as the initial doctor-patient dialogue dataset; Training data acquisition module: perform deduplication preprocessing on the initial doctor-patient dialogue data set to obtain the doctor-patient dialogue training data set; Model training module: Use the doctor-patient dialogue training dataset to perform supervised fine-tuning training on the basic large language model to obtain the medical guidance model.

8. A terminal, characterized in that: include: A memory, used for storing a medical guidance model training program; A processor is used to implement the steps of the medical guidance model training method as described in any one of claims 1 to 6 when executing the medical guidance model training program.

9. A computer-readable storage medium, characterized in that: The readable storage medium stores a medical guidance model training program, and when the medical guidance model training program is executed by the processor, the steps of the medical guidance model training method according to any one of claims 1 to 6 are implemented.

Images