Medical language model construction method and device, storage medium and product

Abstract

The invention discloses a medical language model construction method and device, a storage medium and a product, and relates to the field of computer language models. The method comprises the steps of obtaining data in a general pre-training data set, and performing field classification and quality filtering to obtain a high-quality medical data set; constructing a first sample data set according to the high-quality medical data set and the universal pre-training data set, and training the first language model by using the first sample data set to obtain a first training model; constructing a second sample data set according to the high-quality medical data set and the open-source medical SFT data set, and training the first training model by using the second sample data set to obtain a second training model; training the second training model by using the third sample data set to obtain a third training model; and training the third training model by using the preference sample data set to obtain the medical language model. The problem that a traditional method ignores the importance of data construction in the CPT stage is solved.

Description

Technical Field

[0001] The present application relates to the field of computer language models, and in particular to a method, device, storage medium and product for constructing a medical language model. Background Art

[0002] In recent years, the emergence of generative large language models such as ChatGPT and LLaMA has completely changed human-computer interaction. These models have performed well in basic text understanding and complex problem-solving tasks, demonstrating capabilities close to human understanding and reasoning. However, in industrial applications, the professionalism and cost-effectiveness of large language models (LLMs) are more concerned. Although a series of closed-source models such as GPT-4 still perform well in professional fields, it is not convenient to use such APIs (Application Programming Interfaces) to handle domain-specific problems considering data privacy risks. In the open source community, the lack of domain-specific knowledge often limits the performance of open source models in professional fields such as medicine. The complexity and depth of medical knowledge pose major challenges to the development of accurate and secure medical LLMs. Despite this, the industry generally believes that medical large language models have great potential and can make significant contributions to diagnostic assistance, consultation, drug recommendations, etc. Therefore, it is crucial to develop a fully open source large language model tailored for the medical field.

[0003] Currently, there are several medical models available in this field, such as the Hua Tuo model (the first knowledge-based open source model of biomedical LLM in China). However, most of these models rely only on supervised fine-tuning (SFT). As we all know, pre-training is a key stage in learning domain-specific knowledge, and relying entirely on SFT will result in the model only being able to produce answers in a fixed format. Another approach attempts to combine pre-training with supervised fine-tuning, which is to convert the pre-training data in a specific domain into a unified format similar to SFT data, such as (instruction, output) pairs generated using GPT-3.5. This method of synthesizing large amounts of data may result in a large amount of erroneous knowledge that is consistent with GPT-4 but different from human expertise, and the cost of data synthesis is high. In addition, some researchers introduced Zhongjing (a commonly used LLaMA-based large language model for traditional Chinese medicine) into traditional Chinese medicine to achieve pipeline training from pre-training, SFT to RLHF (Reinforcement Learning from Human Feedback). However, this approach involves two conversion stages of the base model, which may lead to problems such as catastrophic forgetting or model degradation. In addition, previous studies have mainly focused on data construction in the SFT stage, while ignoring the importance of data construction in the Continuous Pretraining (CPT) stage. However, a well-designed CPT data strategy is also crucial for incorporating medical expertise into the model. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present application provides a method, device, storage medium and product for constructing a medical language model.

[0005] In a first aspect, the present application provides a method for constructing a medical language model, comprising: Get a general pre-training dataset; Performing domain classification and quality filtering on the data in the general pre-training dataset to obtain a high-quality medical dataset; Combining the high-quality medical data set with part of the data in the general pre-training data set to obtain a first sample data set; training a first language model using the first sample data set to obtain a first training model; Get the open source medical SFT dataset; Combining the high-quality medical data set with the open source medical SFT data set to obtain a second sample data set; using the second sample data set to train the first training model to obtain a second training model; Constructing a third sample data set according to the open source medical SFT data set; training the second training model using the third sample data set to obtain a third training model; A preference sample data set is constructed according to the open source medical SFT data set, and the third training model is trained using the preference sample data set to obtain the medical language model.

[0006] Furthermore, the data in the general pre-training dataset is subjected to domain classification and quality filtering to obtain a high-quality medical dataset, including: Using a domain classifier to perform domain classification on the data in the general pre-training data set to obtain medical data; Performing a quality filter on the medical data using a rule-based data quality filtering method; Extracting part of the data from the medical data after the primary quality filtering to form a fourth sample; Using the second language model to perform quality scoring on the fourth samples, and determining labels of each fourth sample according to the quality scoring results; wherein the labels of the fourth samples include high quality and low quality; The BGE-M3 model is trained using the fourth sample and its label to obtain a quality assessment model; The quality assessment model is used to perform secondary quality filtering on the medical data after the primary quality filtering to obtain high-quality medical data, and then obtain a high-quality medical data set.

[0007] Furthermore, two rounds of quality scoring are performed on the fourth samples using the second language model, samples whose difference between the two rounds of quality scoring is greater than or equal to the first threshold are deleted, and labels of the retained fourth samples are determined according to the quality scoring results.

[0008] Furthermore, the construction process of the domain classifier includes: Extracting part of the data from the general pre-training data set to form a fifth sample; Using a second language model to perform domain labeling on the fifth sample to obtain a label of the fifth sample; The BGE-M3 model is trained using the fifth sample and its label to obtain the domain classifier.

[0009] Furthermore, the second language model is used to perform two rounds of domain labeling on the fifth sample; when the labels of the two rounds of domain labeling are inconsistent, the corresponding sample is deleted.

[0010] Furthermore, a third sample data set is constructed based on the open source medical SFT data set, including: Perform multi-dimensional scoring on each dialogue in the open source medical SFT dataset; Calculate the comprehensive score of each conversation based on the multi-dimensional scores of the conversation; Selecting a first valid dialogue according to a comprehensive score of each dialogue and a second threshold; Based on the open source medical SFT dataset, the conditional information score and the direct information score are calculated; wherein the calculation formula of the conditional information score is: ; in, Indicates that each conversation in the open source medical SFT dataset is The conditional information score of the round, Indicates that each conversation in the open source medical SFT dataset is Content information of each round, Represents the historical content information of each conversation in the open source medical SFT dataset, ; Indicates The number of tokens in each round, Indicates Round No. Tokens, Represents a given historical content information and Before the round -1 token is generated The probability of a token, Represents parameters; The calculation formula of the direct information score is: ; in, Indicates that each conversation in the open source medical SFT dataset is Direct information score of rounds; The influence factor is calculated according to the conditional information score and the direct information score; wherein the calculation formula of the influence factor is: ; in, Indicates that each conversation in the open source medical SFT dataset is Impact factor of each round; Selecting a second effective dialogue according to the influence factor of each dialogue and a third threshold; A third sample data set is constructed according to the first valid dialogue and the second valid dialogue.

[0011] Furthermore, a preference sample dataset is constructed based on the open source medical SFT dataset, including: Using the second language model to respond to queries that do not have a correct response in the open source medical SFT dataset, obtaining a first response to each query that does not have a correct response; Using the third training model to respond to queries in the open source medical SFT dataset that do not have a correct response, to obtain a second response to each query that does not have a correct response; Using the second language model, scoring the first response and the second response of each query without a correct response respectively to obtain a first score and a second score; determining a preference pair for each query that does not have a correct response based on a first score for the first response and a second score for the second response for each query that does not have a correct response; For queries with correct responses in the open source medical SFT dataset, the correct response is selected as a selected answer for the query with the correct response, and one is randomly selected from the remaining responses of the query with the correct response as a rejected answer, thereby constructing a preference pair for the query with the correct response; The preference sample data set is constructed according to the preference pairs of all queries.

[0012] In a second aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the steps in the medical language model construction method as described in the first aspect.

[0013] In a third aspect, the present application further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps in the medical language model construction method described in the first aspect.

[0014] In a fourth aspect, the present application further provides a computer program product, comprising a computer program / instructions, which, when executed by a processor, implement the steps in the medical language model construction method as described in the first aspect of the present application.

[0015] The beneficial effects of this application are: This application uses a first sample data set to perform stable CPT on a first language model, thereby solving the distribution difference between general data and data in a specific field; uses a second sample data set to perform enhanced CPT on a first training model, thereby narrowing the gap between pre-trained data and fine-tuning data, and achieving a smooth transition of the training model from general data to data in a specific field, and then to fine-tuning data, thereby gradually enhancing the medical field knowledge of the training model and solving the problem that traditional methods ignore the importance of data construction in the CPT stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of the method for constructing a medical language model in an embodiment of the present application; Figure 2 The scores of the second training model CareBot in the embodiment of the present application on seven benchmark tests; Figure 3 The comparison results of the third training model CareBot-Chat in the embodiment of the present application and various baselines on the single-round dialogue benchmark Huatuo26M-test; Figure 4 The comparison results of the medical language model CareBot-Chat (RL) in the embodiment of the present application with various baselines on the single-round dialogue benchmark Huatuo26M-test; Figure 5 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following is a clear and complete description of the technical solutions in this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Embodiment 1 Figure 1 The flowchart of the method for constructing a medical language model provided by the present application is shown in FIG. Figure 1 As shown, at step S1, a general pre-training data set is obtained.

[0021] In order to optimize the use of existing general data resources and minimize the cost of acquiring new medical-related data, the present embodiment extracts medical field-specific pre-training data from the general corpus with extensive coverage of 15T. The general corpus includes web content, encyclopedias, books, and academic papers, such as C4, Pile, Wudao, and PubMed.

[0022] Next, go to step S2. In step S2, the data in the general pre-training dataset is subjected to domain classification and quality filtering to obtain a high-quality medical dataset.

[0023] In order to ensure the quality of data in a specific field, the collected general pre-training data set is subjected to field classification and quality assessment. Since the general pre-training data set comes from a diverse data set and lacks clear field labels, the general pre-training data set must first be field classified to extract high-quality medical data. In a specific embodiment of the present application, a field classifier is used to field classify all data in the general pre-training data set to obtain medical data. The construction process of the field classifier includes: A part of the data (for example, 40k) is extracted from the general pre-training data set to form the fifth sample; the second language model is used to perform domain annotation on the fifth sample to obtain the label of the fifth sample; the general vector model BGE-M3 is trained using the fifth sample and its label to obtain a domain classifier.

[0024] In this embodiment, the second language model uses the GPT-4 model. In order to improve the accuracy of domain labeling, the second language model is used to perform two rounds of domain labeling on the fifth sample; when the labels of the two rounds of domain labeling are inconsistent, the corresponding sample is deleted, and the retained 36k data and its labels are used to train the general vector model BGE-M3 to obtain a domain classifier.

[0025] In order to ensure the quality of medical data obtained through field classification, this application uses a rule-based data quality filtering method and an LLM-based data quality filtering method to perform quality filtering on medical data. The rules in the rule-based data quality filtering method include deleting data with insufficient marks, too many special characters, harmful content and privacy information, and eliminating noise data.

[0026] After a quality filter of the medical data using a rule-based data quality filtering method, the following problems were found: (1) The data includes advertisements and marketing content, which may seriously distort the output preference of the model; (2) The data contains grammatical errors, semantic inconsistencies, splicing, irrelevant content, and image and video clips. Such data is harmful to model training because it provides the least valuable information for autoregressive learning.

[0027] In order to solve these problems, this application adopts a data quality filtering method based on LLM to perform secondary quality filtering on the medical data after the primary quality filtering, specifically including: A part of the data (for example, 20k) is extracted from the medical data after the first quality filtering to form a fourth sample; the second language model is used to perform quality scoring on the fourth sample, and the label of each fourth sample is determined according to the quality scoring result; wherein the label of the fourth sample includes high quality (represented by 1) and low quality (represented by 0); the general vector model BGE-M3 is trained by using the fourth sample and its label to obtain the quality assessment model DataRater; the quality assessment model DataRater is used to perform secondary quality filtering on the medical data after the first quality filtering to obtain high-quality medical data, and then obtain a high-quality medical data set.

[0028] In order to further improve the accuracy of quality assessment, this application uses the second language model to perform two rounds of quality scoring on the fourth sample, deletes the samples whose difference between the two rounds of quality scoring is greater than or equal to the first threshold, and determines the labels of each retained fourth sample based on the quality scoring result. Exemplarily, the quality score range is 0 to 5, and the first threshold is set to 2. When two rounds of quality scoring are performed on a fourth sample, and the difference between the two rounds of quality scoring (for example, if the quality score of one round is 5 and the quality score of the other round is 3, then the difference between the two rounds is 3) is greater than 2 points, the fourth sample is deleted, and finally 15k high-quality medical data is obtained.

[0029] This application designs a quality assessment model DataRater, which uses comprehensive quality assessment standards to evaluate the grammatical accuracy, information density, semantic consistency, and domain-specific attributes of medical data after a quality filter. The quality assessment model DataRater effectively mitigates data bias and ensures the performance and generalization ability of the second training model CareBot in the medical field.

[0030] Next, go to step S3. In step S3, the high-quality medical data set obtained in step S2 is combined with part of the data in the general pre-training data set obtained in step S1 to obtain a first sample data set.

[0031] In this embodiment, the high-quality medical data set is combined with part of the data in the general pre-training data set in a ratio of 19:1, and the token-level distribution of Chinese:English is 1:9.

[0032] Next, go to step S4. At step S4, the first language model is trained using the first sample data set (i.e., stabilized CPT) to obtain a first training model. In order to balance medical domain knowledge and general knowledge, stabilizing CPT ensures that the model maintains and enhances its general language comprehension ability while focusing on medical knowledge. In this embodiment, the first language model uses the LLaMA3-8B model.

[0033] Next, go to step S5. At step S5, obtain an open source medical SFT dataset. Part of the data of the open source medical SFT dataset comes from real-world medical diagnosis dialogues, and part comes from the content generated by the GPT-3.5 model. This combination ensures that the model provides informative, clear, and logically consistent answers, while also providing professional and personalized consultation similar to that of medical experts.

[0034] Next, go to step S6. At step S6, the high-quality medical dataset is combined with the open source medical SFT dataset to obtain a second sample dataset.

[0035] Next, go to step S7. In step S7, the first training model is trained using the second sample data set (ie, enhanced CPT) to obtain a second training model CareBot.

[0036] In order to integrate medical knowledge in the model pre-training stage and smoothly transition to specific domain tasks, this application designs an enhanced CPT stage. In the enhanced CPT stage, the high-quality medical dataset is combined with the open source medical SFT dataset in a 1:1 ratio, and the token-level distribution of Chinese: English is 4:6. It is worth noting that in the stable CPT and enhanced CPT, this application gradually increases the proportion of Chinese data. The enhanced CPT narrows the gap between pre-training data and fine-tuning data; the stable CPT and enhanced CPT facilitate the smooth transition of the model from general data to domain-specific data and finally to fine-tuning data, thereby gradually enhancing the domain knowledge of the model.

[0037] Next, go to step S8. At step S8, a third sample data set is constructed according to the open source medical SFT data set.

[0038] The open source medical SFT dataset contains various question types, such as multiple-choice questions from medical examinations, single-round disease diagnosis, and multi-round health consultations. It integrates data from seven public sources: Chinese medical dialogue data Huatuo26M, MedDialog, ChatMed Consult, Chat-Doctor, CMB, and MedQA. Some real doctor-patient dialogues are retained, and the remaining doctor-patient dialogues are input into the GPT-4 model for rewriting to obtain more detailed responses. This ensures the diversity of the open source medical SFT dataset, which can help the second training model CareBot better adapt to different types of medical problems and patient conditions, thereby improving its performance in various scenarios.

[0039] A relatively small high-quality dataset is sufficient to fine-tune a large language model, and the focus is on effectively filtering out "good data" from massive data to achieve competitive performance with the least amount of data. Like the standard data cleaning process, this application first removes duplicates from the open source medical SFT dataset and eliminates data related to safety issues such as violence and bias.

[0040] In the specific implementation of the present application, single-round medical dialogue and multi-round medical dialogue are used to select valid dialogues from the open source medical SFT data set, and then the selected valid dialogues are combined to obtain a third sample data set. For single-round medical dialogue, the present application believes that "good data" should have complex instructions and high-quality responses. Therefore, the present application adopts the Deita method, which specifically includes: Each conversation in the open source medical SFT dataset is scored from the two dimensions of complexity and response quality; the comprehensive score of each conversation is calculated based on each score of the conversation; the first valid conversation is selected based on the comprehensive score of each conversation and the second threshold.

[0041] In this embodiment, the complexity model is used to score the complexity of each dialogue to obtain the complexity score of each dialogue. ; Use the quality model to score the response quality of each dialogue and obtain the quality score of each dialogue , then the comprehensive score of each dialogue When the overall score of each conversation When it is greater than or equal to the second threshold, the dialogue is the first valid dialogue.

[0042] Use the Deita method to calculate the overall score for each conversation , and then the average of the comprehensive scores is calculated to obtain the final score of the entire conversation. However, compared with single-round medical conversations, this paper finds that multi-round medical conversations have two specific challenges: (1) low correlation between different rounds will have a negative impact on the early information correlation of subsequent rounds; (2) excessive correlation between rounds will lead to serious context duplication and redundant information. To solve these problems, this application proposes a ConFilter method, which uses an influence factor based on cross-entropy loss to evaluate the impact of historical information on each conversation. In a specific embodiment of this application, the ConFilter method includes: Based on the open source medical SFT dataset, the conditional information score and the direct information score are calculated; the impact factor is calculated based on the conditional information score and the direct information score; the second effective dialogue is selected based on the impact factor of each dialogue and the third threshold.

[0043] During the instruction adjustment process, a dialogue The loss is obtained by Continuously predict the current round The next token in is calculated: (1) in, Indicates that each conversation in the open source medical SFT dataset is Content information of each round, Represents the historical content information of each conversation in the open source medical SFT dataset, ; Indicates The number of tokens in each round, Indicates Round No. Tokens, Represents a given historical content information and Before the round -1 token is generated The probability of a token, Indicates a parameter. This application will It is defined as the conditional information score, which measures the ability to generate the current round under the guidance of the corresponding historical information.

[0044] To measure the ability of a large language model to generate this transition alone, this application also defines a direct information score: (2); in, Indicates that each conversation in the open source medical SFT dataset is The direct information score of each turn. This application considers that a higher direct information score may indicate a more challenging or complex turn. Finally, by calculating and The impact factor is estimated by the ratio between : (3); in, Indicates that each conversation in the open source medical SFT dataset is The impact factor of each round. , indicating that historical information has a negative impact on the current round, that is, the correlation between contexts is very low; , indicating that historical information has a positive impact on the current round, that is, the correlation between contexts is high. However If it is too small, it means that the context is highly repetitive and the information is highly redundant. Therefore, a third threshold is set to filter the data. <1, that is, the second valid conversation. In this embodiment, the third threshold is set to 0.7.

[0045] The ConFilter method proposed in this application aims to measure the indicators of correlation between multiple rounds, which helps to filter multi-round medical conversations. The addition of high-quality multi-round medical conversations not only improves the model's ability to understand user intentions and generate relevant responses, but also ensures a natural and smooth conversation experience, thereby improving user comfort and satisfaction.

[0046] The first valid dialogue selected from the single-round medical dialogue and the second valid dialogue selected from the multi-round medical dialogue are combined to obtain the third sample dataset. The third sample dataset is a highly diverse medical SFT dataset, including single-round and multi-round medical dialogues, as well as multiple-choice questions on medical subject knowledge, covering more than 15 departments and more than 100 disease specialties. Please note that the third sample dataset is the largest existing open source bilingual medical SFT dataset, which supports a variety of medical applications, clinical tasks, and online consultations, and significantly improves the performance of the second training model CareBot in multiple dimensions.

[0047] Next, go to step S9. In step S9, the second training model CareBot is trained using the third sample data set (ie, SFT stage) to obtain the third training model CareBot-Chat.

[0048] Next, go to step S10. At step S10, a preference sample dataset is constructed according to the open source medical SFT dataset.

[0049] In order to make the final medical language model output consistent with human preferences while retaining the basic capabilities obtained in the CPT and SFT stages, this application uses samples with the same distribution as the open source medical SFT dataset to construct subjective preference data and objective preference data, and then combines the subjective preference data and the objective preference data to obtain a preference sample dataset.

[0050] In order to construct a DPO (Direct Preference Optimization) pair in which the selected response is closely related to human preferences, subjective preference data is constructed. Specifically, for queries that do not have a correct response in the open source medical SFT dataset, the second language model is used to respond to it as a professional and helpful doctor to obtain the first response to the query; the third training model CareBot-Chat is used to respond to each query that does not have a correct response to obtain the second response to the query; the first response and the second response of each query that does not have a correct response are scored respectively using the second language model to obtain the first score and the second score; the preference pair of each query that does not have a correct response is determined according to the first score of the first response and the second score of the second response of each query that does not have a correct response.

[0051] The second language model scores the first and second responses of each query without a correct response in terms of fluency, relevance, completeness, and medical proficiency, and selects the superior response (i.e., the response corresponding to the higher of the first and second scores) as the selected response in the DPO pair, and selects the inferior response as the rejected response in the DPO pair.

[0052] Although RLHF can guide large language models to be consistent with human expectations, a large number of studies have shown that this approach may cause large language models to forget the abilities acquired in the DPT and SFT stages. In order to alleviate this problem, this application also constructs objective preference data. Specifically, for queries with correct responses in the open source medical SFT dataset, the correct response is selected as the selected answer for the query, and one is randomly selected from the remaining responses to the query as a rejected answer, thereby constructing a preference pair for the query. For example, in a multiple-choice question, if the correct response is option A, then one is randomly selected from options B, C, and D as a rejected answer.

[0053] Next, go to step S11. In step S11, the third training model is trained using the preference sample data set to obtain a medical language model CareBot-Chat (RL).

[0054] In the RLHF stage, the second language model is used to create positive and negative medical data pairs, and the Direct Preference Optimization (DPO) algorithm is used to match the output of the medical language model with the human expression style to provide personalized answers and suggestions and enhance the overall user experience and satisfaction.

[0055] In order to verify the effectiveness of the method of the present invention, the results of each stage of the present invention are verified. Figure 2The second training model CareBot is evaluated on seven common medical benchmarks. Considering that this application is to train a medical language model that performs well in both Chinese and English, while striving to improve the Chinese medical ability, it is ensured that the English medical ability of the model is slightly reduced. Figure 2 It can be seen that for the English benchmarks (MMLU-Med, PubMedQA, MedQA, MedM-CQA), the performance of CareBot (stable CPT) (i.e., the first training model) and Care-Bot (stable CPT and enhanced CPT) (i.e., the second training model) has slightly decreased. This is expected because the basic model LLaMA-8B of the present application already has strong English capabilities. However, for the Chinese benchmarks (C-Eval-Med, CMMLU-Med, CMB), the CareBot (stable CPT) (i.e., the first training model) and Care-Bot (stable CPT and enhanced CPT) (i.e., the second training model) of the present application showed significant improvements, especially the second training model achieved significant gains. This confirms that the stable CPT and enhanced CPT of the present invention effectively integrate medical field knowledge into the model, thereby greatly enhancing its Chinese medical capabilities.

[0056] Table 1 shows the results of five widely used benchmarks. For the English benchmark MedQA, CareBot-Chat, the third training model of this application, is 11.47 points higher than ChatGPT and 13.11% higher than the most powerful open source medical LLM BioMistral. For MMLU-Med, CareBot-Chat, the third training model of this application, is 1.57% higher than ChatGPT and 8.81 points higher than MedAlpaca. For the Chinese benchmarks CMB and CMMLU-Med, HuatuoGPT II became the best performing model, and the model of this application did not have an advantage over it. For C-Eval-Med, the model of this application only achieved comparable results with HuatuoGPT II. This is because CareBot is built on LLaMA3-8B, and the large language model has an innate advantage in English. Therefore, its performance in Chinese is as expected. However, in terms of average scores, the third training model CareBot-Chat is 6.69% higher than the leading open source medical model HuatuoGPT II and 8.65% higher than ChatGPT, which fully demonstrates the outstanding performance of CareBot in the field of medical applications and has made important contributions to the field of medical AI.

[0057] Table 1 Results of various models on five widely used benchmarks In Table 1, all scores are the average of three random tests (p < 0.05 under t-test).

[0058] Figure 3 The comparison results of the third training model CareBot-Chat of this application with various baselines on the single-round dialogue benchmark Huatuo26M-test are shown. Figure 3 The numbers in are the percentages of wins, draws, and losses for CareBot-Chat, the third training model of the present application. It is obvious that the third training model CareBot-Chat achieves performance comparable to ChatGPT and HuatuoGPT, indicating that the baseline HuatuoGPT performs similarly to ChatGPT under this evaluation framework. In addition, the model of the present application is significantly better than Zhongjing, MedAlpaca, and BioMistral. The poor performance of MedAlpaca and BioMistral can be attributed to their limited Chinese capabilities and insufficient medical SFT data. It is worth noting that Zhongjing, which combines pre-training, SFT, and RLHF stages, performs poorly. This may be due to catastrophic forgetting or model degradation caused by the two training stages of the base model. In addition, its instruction fine-tuning data is limited and of poor quality, which explains why models with larger parameters than CareBot of the present application perform worse. However, it is observed that the model of the present application is only competitive with HuatuoGPTll-7B in the Huatuo-Eval framework. In Figure 4 In the paper, the medical language model CareBot-Chat (RL) of this application was compared with other baselines and basically achieved the same performance as CareBot-Chat. Figure 4 The numbers in are the percentages of wins, draws, and losses of the medical language model CareBot-Chat (RL) of this application.

[0059] Tables 2 and 3 show the results of the multi-turn dialogue benchmarks CMtMedQA and CMB-clin, respectively. Overall, the performance of the two baselines is basically the same, with HuatuoGPT and HuatuoGPT ll performing significantly better. Among the four dimensions of fluency, relevance, completeness, and medical proficiency, the fluency and proficiency scores are particularly high, indicating that the medical law masters' answers are coherent and have a solid grasp of medical terminology. However, the relevance and completeness scores are low, indicating that there is room for improvement in providing highly relevant and comprehensive answers to specific questions. Our model CareBot-Chat achieved strong performance in all dimensions, with average scores of 4.35 and 4.22, respectively, with significant advantages in relevance and completeness. This highlights the effectiveness of our high-quality SFT dataset and multi-turn medical dialogue selection method, which significantly enhances the model's contextual understanding and ensures the coherence and consistency of the dialogue.

[0060] Table 2 Scores of different models on CMtMedQA Model Name Fluency Relevance Completeness Medical proficiency Average score ChatGPT 4.94 4.21 3.98 4.04 4.29 HuatuoGPT-7B 4.94 3.28 3.22 4.19 3.91 Zhongjing-13B 2.26 1.68 1.62 2.63 2.05 MedAlpaca-7B 4.35 2.18 1.97 3.29 2.95 Biomistral-7B 4.48 2.45 2.17 3.55 3.16 HuatuoGPT II-7B 4.96 3.41 3.47 4.27 4.03 CareBot-Chat 4.99 4.67 4.20 4.26 4.53 CareBot-Chat (RL) 4.96 4.70 4.31 4.34 4.58 Table 3 Scores of different models on CMB-Clin Model Name Fluency Relevance Completeness Medical proficiency Average score ChatGPT 4.78 3.75. 3.77 4.01 4.08 HuatuoGPT-7B 4.69 3.25 3.08 4.00 3.76 Zhongjing-13B 3.14 2.14 1.89 3.11 2.57 MedAlpaca-7B 3.31 1.69 1.49 2.58 2.27 Biomistral-7B 3.79 2.10 1.82 3.09 2.70 HuatuoGPT II-7B 4.75 3.43 3.43 4.22 3.96 CareBot-Chat 4.82 4.20 3.68 4.16 4.22 CareBot-Chat (RL) 4.74 4.28 4.01 4.21 4.31

[0061] Embodiment 2 The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, the electronic device includes: a memory, a processor, and a computer program / instruction stored in the memory, and the processor executes the computer program / instruction to implement the medical language model construction method in the first embodiment of the present application.

[0062] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0063] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0064] Although not shown, the embodiment of the present invention further provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the medical language model construction method in the first embodiment of the present application is implemented.

[0065] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0066] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the medical language model construction method in the first embodiment of the present application.

[0067] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application. Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for constructing a medical language model, characterized in that: The construction method comprises: Get a general pre-training dataset; Performing domain classification and quality filtering on the data in the general pre-training dataset to obtain a high-quality medical dataset; Combining the high-quality medical data set with part of the data in the general pre-training data set to obtain a first sample data set; training a first language model using the first sample data set to obtain a first training model; Get the open source medical SFT dataset; Combining the high-quality medical data set with the open source medical SFT data set to obtain a second sample data set; using the second sample data set to train the first training model to obtain a second training model; Constructing a third sample data set according to the open source medical SFT data set; training the second training model using the third sample data set to obtain a third training model; A preference sample data set is constructed according to the open source medical SFT data set, and the third training model is trained using the preference sample data set to obtain the medical language model.

2. The method for constructing a medical language model according to claim 1, characterized in that: The data in the general pre-training dataset is subjected to domain classification and quality filtering to obtain a high-quality medical dataset, including: Using a domain classifier to perform domain classification on the data in the general pre-training data set to obtain medical data; Performing a quality filter on the medical data using a rule-based data quality filtering method; Extracting part of the data from the medical data after the primary quality filtering to form a fourth sample; Using the second language model to perform quality scoring on the fourth samples, and determining labels of each fourth sample according to the quality scoring results; wherein the labels of the fourth samples include high quality and low quality; The BGE-M3 model is trained using the fourth sample and its label to obtain a quality assessment model; The quality assessment model is used to perform secondary quality filtering on the medical data after the primary quality filtering to obtain high-quality medical data, and then obtain a high-quality medical data set.

3. The method for constructing a medical language model according to claim 2, characterized in that: The second language model is used to perform two rounds of quality scoring on the fourth samples, samples whose difference between the two rounds of quality scoring is greater than or equal to the first threshold are deleted, and labels of the retained fourth samples are determined according to the quality scoring results.

4. The method for constructing a medical language model according to claim 2, characterized in that: The construction process of the domain classifier includes: Extracting part of the data from the general pre-training data set to form a fifth sample; Using a second language model to perform domain labeling on the fifth sample to obtain a label of the fifth sample; The BGE-M3 model is trained using the fifth sample and its label to obtain the domain classifier.

5. The method for constructing a medical language model according to claim 4, characterized in that: The second language model is used to perform two rounds of domain labeling on the fifth sample; when the labels of the two rounds of domain labeling are inconsistent, the corresponding sample is deleted.

6. The method for constructing a medical language model according to claim 1, characterized in that: A third sample dataset is constructed based on the open source medical SFT dataset, including: Perform multi-dimensional scoring on each dialogue in the open source medical SFT dataset; Calculate the comprehensive score of each conversation based on the multi-dimensional scores of the conversation; Selecting a first valid dialogue according to a comprehensive score of each dialogue and a second threshold; Based on the open source medical SFT dataset, the conditional information score and the direct information score are calculated; wherein the calculation formula of the conditional information score is: ； in, Indicates that each conversation in the open source medical SFT dataset is The conditional information score of the round, Indicates that each conversation in the open source medical SFT dataset is The content information of each round, Represents the historical content information of each conversation in the open source medical SFT dataset, ; Indicates The number of tokens in each round, Indicates The first round Tokens, Represents a given historical content information and Before the round -1 token is generated The probability of a token, Represents parameters; The calculation formula of the direct information score is: ； in, Indicates that each conversation in the open source medical SFT dataset is Direct information score of rounds; The impact factor is calculated according to the conditional information score and the direct information score; wherein the calculation formula of the impact factor is: ； in, Indicates that each conversation in the open source medical SFT dataset is Impact factor of each round; Selecting a second effective dialogue according to the influence factor of each dialogue and a third threshold; A third sample data set is constructed according to the first valid dialogue and the second valid dialogue.

7. The method for constructing a medical language model according to claim 1, characterized in that: A preference sample dataset is constructed based on the open source medical SFT dataset, including: Using the second language model to respond to queries that do not have a correct response in the open source medical SFT dataset, obtaining a first response to each query that does not have a correct response; Using the third training model to respond to queries in the open source medical SFT dataset that do not have a correct response, to obtain a second response to each query that does not have a correct response; Using the second language model, scoring the first response and the second response of each query without a correct response respectively to obtain a first score and a second score; determining a preference pair for each query that does not have a correct response based on a first score for the first response and a second score for the second response for each query that does not have a correct response; For queries with correct responses in the open source medical SFT dataset, the correct response is selected as a selected answer for the query with the correct response, and one is randomly selected from the remaining responses of the query with the correct response as a rejected answer, thereby constructing a preference pair for the query with the correct response; The preference sample data set is constructed according to the preference pairs of all queries.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the steps in the medical language model construction method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps in the medical language model construction method as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the medical language model construction method as described in any one of claims 1 to 7 are implemented.

Images