Medical LLM model fine tuning method and related equipment

Abstract

The invention provides a medical LLM model fine tuning method and related equipment. The method comprises the following steps: acquiring a data set related to a target application scene; the data set comprises a single-round question and answer sample, and the single-round question and answer sample comprises a question text and an answer text corresponding to the question text; obtaining a multi-round dialogue rule corresponding to a multi-round dialogue task constructed based on a task demand of the multi-round dialogue task related to the target application scene, and converting the single-round question and answer sample into a multi-round dialogue sample conforming to the multi-round dialogue rule; and further performing fine tuning training on a pre-trained LLM basic model based on the multi-round dialogue sample to obtain an LLM service model for performing the multi-round dialogue task based on the multi-round dialogue rule.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of medical technology, and in particular, to a medical LLM model fine-tuning method and related equipment. Background Art

[0002] Large Language Models (LLM), or pre-trained models, are language models built with deep neural networks containing more than tens of billions of parameters. They use self-supervised learning methods to master many language phenomena through pre-training on large and diverse public datasets.

[0003] Furthermore, by using a data set related to a specific application scenario to perform fine-tuning training on the pre-trained LLM basic model, an LLM service model for performing multi-round dialogue tasks related to the specific application scenario can be obtained.

[0004] However, the fine-tuned LLM service model often has logical confusion problems when performing multi-round dialogue tasks, such as repeatedly asking the same questions to users, or directly outputting the final inference without obtaining sufficient information. This not only brings a very bad user experience to users, but also fails to meet their actual usage needs. Even wrong inferences may have serious negative impacts on users. Summary of the invention

[0005] In view of this, one or more embodiments of the present specification provide a medical LLM model fine-tuning method and related equipment.

[0006] In a first aspect, this specification provides an LLM model fine-tuning method, the method comprising:

[0007] Acquire a data set related to the target application scenario; the data set includes a plurality of single-round question-answering samples, each of which includes a question text and an answer text corresponding to the question text;

[0008] Acquire a task requirement of a multi-round dialogue task related to the target application scenario, construct a multi-round dialogue rule corresponding to the multi-round dialogue task, and convert the single-round question and answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule;

[0009] Based on the multi-round dialogue samples, the pre-trained LLM basic model is further fine-tuned to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

[0010] In a second aspect, this specification provides an LLM model fine-tuning device, the device comprising:

[0011] A question-and-answer data acquisition unit, configured to acquire a data set related to a target application scenario; the data set includes a plurality of single-round question-and-answer samples, each of which includes a question text and an answer text corresponding to the question text;

[0012] A multi-round dialogue sample generating unit, configured to obtain a task requirement of a multi-round dialogue task related to the target application scenario, construct a multi-round dialogue rule corresponding to the multi-round dialogue task, and convert the single-round question-and-answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule;

[0013] A fine-tuning unit is used to further perform fine-tuning training on the pre-trained LLM basic model based on the multi-round dialogue samples to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

[0014] Accordingly, this specification also provides a computing device, comprising: a memory and a processor; the memory stores a computer program / instruction that can be executed by the processor; when the processor executes the computer program / instruction, it executes the LLM model fine-tuning method described in the first aspect above.

[0015] Accordingly, the present specification also provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the LLM model fine-tuning method as described in the first aspect above is executed.

[0016] Accordingly, the present specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the LLM model fine-tuning method described in the first aspect above is executed.

[0017] In summary, when the present application fine-tunes the pre-trained LLM basic model so that it can ideally perform multi-round dialogue tasks related to the target application scenario, it is considered that the single-round question and answer samples contained in the data set directly obtained and related to the target application scenario are often composed of only a question text and a corresponding answer text. If the LLM basic model is fine-tuned directly based on the single-round question and answer sample, the fine-tuned model is prone to logical confusion when performing multi-round dialogue tasks, such as repeatedly asking the same question to the user, or directly outputting the final inference without obtaining sufficient information, and the effect is extremely poor. Based on this, the present application can first convert the above-mentioned single-round question and answer samples into multi-round dialogue samples that conform to the preset multi-round dialogue rules, and then use these multi-round dialogue samples to fine-tune the LLM basic model so that the model can learn the multi-round dialogue rules. Correspondingly, the fine-tuned model will also perform multi-round dialogue tasks according to the multi-round dialogue rules, so as to respond logically, improve model performance, and improve user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of an LLM model fine-tuning method provided by an exemplary embodiment;

[0019] Figure 2 This is a schematic diagram of a multi-round dialogue that complies with medical consultation rules provided by an exemplary embodiment;

[0020] Figure 3 is a flowchart of another LLM model fine-tuning method provided by an exemplary embodiment;

[0021] Figure 4 It is a structural schematic diagram of an LLM model fine-tuning device provided by an exemplary embodiment;

[0022] Figure 5 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0026] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0027] This specification provides a technical solution. After obtaining a data set related to LLM model fine-tuning, the data set is first converted into multi-round dialogue samples that conform to preset multi-round dialogue rules, and then these multi-round dialogue samples are used to fine-tune the LLM model, so that the fine-tuned model can perform multi-round dialogue tasks according to the multi-round dialogue rules, thereby improving the logic and performance of the model's responses.

[0028] During implementation, the present application may first obtain a data set related to the target application scenario, wherein the data set may include multiple single-round question and answer samples, and each single-round question and answer sample may include a question text and an answer text corresponding to the question text. Then, the present application may obtain the task requirements of a multi-round dialogue task based on the target application scenario, construct a multi-round dialogue rule corresponding to the multi-round dialogue task, and convert the above single-round question and answer samples into multi-round dialogue samples that comply with the multi-round dialogue rules. Then, the present application may further perform fine-tuning training on the pre-trained LLM basic model based on the converted multi-round dialogue samples, thereby obtaining an LLM service model that performs multi-round dialogue tasks based on the above multi-round dialogue rules.

[0029] In the above technical scheme, when the present application fine-tunes the pre-trained LLM basic model so that it can ideally perform multi-round dialogue tasks related to the target application scenario, considering that the single-round question and answer samples contained in the data set directly obtained and related to the target application scenario are often composed of only a question text and a corresponding answer text, if the LLM basic model is fine-tuned directly based on the single-round question and answer samples, the fine-tuned model is prone to logical confusion when performing multi-round dialogue tasks, such as repeatedly asking the user the same question, or directly outputting the final inference without obtaining sufficient information, and the effect is extremely poor. Based on this, the present application can first convert the above-mentioned single-round question and answer samples into multi-round dialogue samples that conform to the preset multi-round dialogue rules, and then use these multi-round dialogue samples to fine-tune the LLM basic model so that the model can learn the multi-round dialogue rules. Correspondingly, the fine-tuned model will also perform multi-round dialogue tasks according to the multi-round dialogue rules, so as to respond in an organized and logical manner, improve the model performance, and improve the user experience.

[0030] See also Figure 1 , Figure 1 This is a flowchart of an LLM model fine-tuning method provided by an exemplary embodiment. The method can be applied to a computing device, such as a smart wearable device, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., which is not specifically limited in this specification. Figure 1 As shown, the method may specifically include the following steps S101 to S103.

[0031] Step S101, obtaining a data set related to a target application scenario; the data set includes a plurality of single-round question-and-answer samples, each of which includes a question text and an answer text corresponding to the question text.

[0032] In one illustrated embodiment, a pre-trained LLM basic model may be first obtained, wherein the LLM basic model has mastered many language phenomena through pre-training and can handle reasoning tasks based on natural language understanding, such as multi-round dialogue tasks with users.

[0033] Furthermore, in order to apply the LLM basic model to multi-round dialogue tasks related to a specific target application scenario, it is necessary to further fine-tune the LLM basic model through a data set related to the target application scenario, so as to obtain an LLM service model that can smoothly perform multi-round dialogue tasks related to the target application scenario. In an illustrated embodiment, the above-mentioned target application scenario can be a medical consultation scenario, a life assistant scenario, or a sentiment analysis scenario, etc., and this specification does not specifically limit this.

[0034] For the purpose of fine-tuning the above target application scenario, after obtaining the pre-trained LLM basic model, it is necessary to further obtain a data set related to the target application scenario.

[0035] It should be noted that this specification does not specifically limit the specific content contained in the data set. In an illustrated embodiment, the data set may include multiple single-round question and answer samples related to the target application scenario, each single-round question and answer sample may be a question and answer pair, and a question and answer pair may consist of a question text and an answer text corresponding to the question text. In an illustrated embodiment, the data set may also include various journals, articles, textbooks and other materials related to the target application scenario, and this specification does not specifically limit this.

[0036] The following is an explanation of the datasets in this specification with reference to specific application scenario examples.

[0037] Example 1: Medical consultation scenario

[0038] In one illustrated embodiment, taking the target application scenario as a medical consultation scenario as an example, the multiple single-round question and answer samples contained in the above data set can be single-round question and answer samples corresponding to different disease categories generated based on the acquired clinical data of virtual patients corresponding to different disease categories.

[0039] In an illustrated embodiment, the clinical data of these virtual patients corresponding to different disease categories may be data compiled by relevant models or professional medical personnel based on the medical records of real patients, thereby effectively avoiding leakage of patient privacy.

[0040] For example, taking the target application scenario as a medical consultation scenario, the question text and answer text contained in a single round of question and answer samples may be as follows:

[0041] Question text: Hello, doctor. I am a 62-year-old female patient who recently discovered that I have had right ureteral stenosis for more than 2 years. I recently had a urinary tract CT scan, which showed thickening of the wall of the lower segment of the right ureter and obvious water accumulation in the upstream urinary tract. The doctor recommended a combination of CTU examination and endoscopy. In addition, I also noticed that the left adrenal gland was slightly thickened, which may be related to post-hysterectomy. I would like to ask you to give me some advice on how to treat this problem based on my medical history.

[0042] Answer text: Dear patient, based on the information and symptoms you provided, combined with the results of the CT scan, your condition seems to be caused by stenosis of the lower right ureter. This condition may have caused thickening of the tube wall and hydroureteral accumulation in the upstream urinary tract. The slight thickening of the left adrenal gland is most likely related to your previous hysterectomy. At present, my diagnosis is hydroureteral accumulation in the upstream urinary tract caused by ureteral stenosis. We need to combine the results of CTU examination and endoscopy to confirm the diagnosis and formulate a treatment plan.

[0043] Example 2: Life Assistant Scenario

[0044] In one illustrated embodiment, taking the target application scenario as a life assistant scenario as an example, the single-round question and answer samples contained in the above data set can be single-round question and answer samples corresponding to different user intentions generated based on various types of software operation data corresponding to different user intentions of virtual users.

[0045] In an illustrated embodiment, the various types of software operation data corresponding to the different user intentions of virtual users may be data written by relevant models or staff based on the software operation data of real users, thereby effectively avoiding leakage of user privacy.

[0046] For example, taking the target application scenario as a life assistant scenario, the question text and answer text contained in a single round of question and answer sample may be specifically as follows:

[0047] Question text: Hello, life assistant, please buy me a movie ticket. I want to watch the recently released movie A. I plan to watch this movie after lunch. I will choose a movie theater closest to my current location. It is best to have a middle seat. If there is no middle seat, I will not consider it.

[0048] Answer text: Hello, according to your requirements, we have selected Cinema B, which is the closest to your current location and has movie A showing. After noon, movie A will be shown twice in Cinema B, at 2pm and 3pm respectively. There are also middle seats available for the 3pm show, including 6th row 15 seats, 6th row 16 seats, 6th row 17 seats, 7th row 16 seats, 8th row 15 seats and 8th row 16 seats. Please confirm your seat and I will help you buy the movie ticket directly after confirmation.

[0049] Step S102, obtaining task requirements of a multi-round dialogue task related to the target application scenario, constructing a multi-round dialogue rule corresponding to the multi-round dialogue task, and converting the single-round question and answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule.

[0050] Furthermore, after obtaining the data set related to the target application scenario, given that the data contained in the data set is often scattered and lacks complete dialogue logic, it is impossible to guide the model to perform multi-round dialogue tasks well. Therefore, in order to improve the fine-tuning training effect of the LLM basic model so that the fine-tuned LLM service model can perform multi-round dialogue tasks in an organized and logical manner, it is necessary to first convert the data contained in the data set (such as single-round question and answer samples) into multi-round dialogue samples that conform to the preset multi-round dialogue rules.

[0051] In an illustrated embodiment, the multi-round dialogue rule may be a multi-round dialogue rule corresponding to the multi-round dialogue task constructed based on the task requirements of the multi-round dialogue task related to the target application scenario. Exemplarily, the multi-round dialogue rule may be a rule manually written by a staff member based on the task requirements, or a rule automatically extracted by a relevant model from historical experience data or materials corresponding to the task requirements, etc., which is not specifically limited in this specification.

[0052] In one illustrated embodiment, taking the target application scenario as a medical consultation scenario, it should be understood that the diagnosis of a disease and the formulation of a treatment plan often require sufficient patient evidence support, but when describing their own condition, the patient will not tell all the information at once. At the same time, the patient is usually unable to judge which information is helpful for the diagnosis of the disease. Therefore, during the consultation process, it is often necessary to collect this information step by step through the doctor's guided questions.

[0053] Based on this, multi-round dialogue tasks related to medical consultation scenarios often require doctors and patients to communicate through multiple conversations, so as to gradually understand the patient's feelings and needs, collect information such as the patient's symptoms, test results, and medical history, and ultimately diagnose the disease and effectively formulate or adjust relevant treatment plans.

[0054] Correspondingly, the task requirements of multi-round dialogue tasks related to medical consultation scenarios are generally to obtain medical diagnosis results (for example, including determining possible diseases and providing corresponding examination and treatment suggestions, etc.), and the multi-round dialogue rules in medical consultation scenarios are medical consultation rules constructed to communicate with patients in an organized and logical manner about information such as symptoms, examination results and needs, in order to obtain accurate medical diagnosis results.

[0055] In an illustrated embodiment, the medical consultation rules may include a medical consultation sequence (or medical consultation logic) that should be followed during the medical consultation process to ensure that the consultation process is effective, safe, and organized. Based on this, in an illustrated embodiment, the medical consultation sequence may include symptom inquiries, examination result inquiries, medical history inquiries, and output of medical diagnosis results, etc., which are not specifically limited in this specification. Alternatively, in an illustrated embodiment, the medical consultation sequence may also include symptom inquiries, examination result inquiries, work and rest and eating habits inquiries, and output of medical diagnosis results, etc., which are not specifically limited in this specification.

[0056] As mentioned above, when describing their condition, patients do not provide all the information at once. They usually describe the pain or discomfort in a part of the body first. Therefore, referring to the above medical consultation sequence can more effectively collect more comprehensive patient information, so that the final diagnosis result will be more reasonable and accurate.

[0057] Furthermore, in an illustrated embodiment, the medical consultation rule further includes at least one inquiry rule shown below:

[0058] Symptom inquiry rules: When asking about symptoms, ask about symptoms multiple times until all the symptoms obtained include several symptoms that can be used to diagnose any target disease. It should be understood that when patients describe their symptoms in detail, they may not be able to determine which symptoms are helpful for diagnosing the disease. Therefore, when asking about symptoms, the model needs to ask about details multiple times to obtain as much symptom information as possible, and then obtain several key symptoms that can be used to diagnose a target disease from these symptoms.

[0059] Examination result inquiry rules: When inquiring about examination results, inquire about the examination results of several examination items used to diagnose the target disease. It should be understood that objective examination results are crucial for disease diagnosis and can ensure the reliability and accuracy of disease diagnosis. However, in the medical field, there are many kinds of examination items, and not every examination is necessary for the current user's situation. Therefore, in order to obtain the necessary examination results more quickly and fully, and to use them as a basis for diagnosis to improve the efficiency of medical consultation, the model can mainly ask the user for the examination results of several examination items that are helpful in diagnosing the target disease corresponding to the acquired symptoms. In an illustrated embodiment, the examination result inquiry rule can also include the inquiry order of several examination items that need to be inquired, and the inquiry order can be a corresponding order given by professional medical personnel based on diagnosis and treatment knowledge and actual clinical experience, etc., and this specification does not make specific restrictions on this.

[0060] Medical history inquiry rules: When inquiring about medical history, inquire about the medical history related to the target disease. It should be understood that the user's medical history generally includes medication history, surgical history, past medical history (also known as past medical history) and marital history, etc. When inquiring about medical history, it is necessary to selectively ask questions based on the patient's condition and the possible target diseases determined above, and do not inquire too much about the patient's privacy, so as to improve the patient's consultation experience.

[0061] It should be noted that the specific contents of the above medical consultation rules are only exemplary. In some possible implementations, there may be any other possible consultation order and inquiry rules, etc., and this specification does not make specific limitations on this.

[0062] In summary, after constructing the multi-round dialogue rules corresponding to the multi-round dialogue tasks of the target application scenario, the single-round question and answer samples contained in the above data set can be converted into multi-round dialogue samples that conform to the multi-round dialogue rules. For example, still taking the medical consultation scenario as an example, after constructing the medical consultation rules corresponding to the multi-round dialogue tasks of the medical consultation scenario, the single-round question and answer samples contained in the above data set can be converted into multi-round dialogue samples that conform to the medical consultation rules.

[0063] It should be noted that this specification does not specifically limit the specific implementation method of converting a single-round question and answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rules. In an illustrated implementation, a single-round question and answer sample can be manually rewritten by a staff member into a multi-round dialogue sample that conforms to the multi-round dialogue rules. In an illustrated implementation, a single-round question and answer sample can also be input into a pre-trained text generation model, and the text generation model automatically generates a multi-round dialogue sample that conforms to the multi-round dialogue rules based on the input single-round question and answer sample, and so on. This specification does not specifically limit this.

[0064] It is understandable that in a medical consultation scenario, since there are many departments and disease categories, and the differences between different categories of diseases are often large, corresponding medical consultation rules can be constructed for each disease category. Accordingly, for the single-round question and answer samples corresponding to different disease categories contained in the data set, they can be converted according to the medical consultation rules corresponding to different disease categories, thereby obtaining multi-round dialogue samples corresponding to different disease categories, and so on. This specification does not make specific limitations on this.

[0065] In addition, in the medical consultation scenario, the disease conditions contained in the single-round Q&A sample written based on the clinical data of the virtual patient may be more complex and detailed. Based on this, when converting the single-round Q&A sample into a text that conforms to a multi-round dialogue, it can include: first mapping the disease conditions contained in the single-round Q&A sample (such as "right hydronephrosis") to the corresponding disease category (such as "hydronephrosis"), and then converting the single-round Q&A sample after the disease condition mapping into a multi-round dialogue text that conforms to the above medical consultation rules.

[0066] It should be understood that although information such as the site of disease occurrence (for example, left or right side), the stage of disease development and severity is crucial in some clinical studies, this part of the content has little impact on obtaining key evidence for diagnosing the disease during the consultation process, and the introduction of more complex disease conditions will make model training more complicated. Therefore, by mapping more complex and detailed disease conditions into disease categories, this application can not only realize data collation of a large number of single-round question and answer samples based on clinical data to ensure that the collection of disease data is comprehensive and accurate, but also accurately match the medical consultation rules for different disease categories constructed above, so as to efficiently and accurately convert single-round question and answer samples corresponding to different disease categories into multi-round dialogue samples that conform to the medical consultation rules corresponding to different disease categories.

[0067] For example, based on the single-round question-and-answer sample of the medical consultation scenario shown in Example 1 above, the converted multi-round dialogue sample that conforms to the medical consultation rules may be as follows:

[0068] Patient: Hello, doctor. I am a 62-year-old female patient who recently discovered that I have had right ureteral stenosis for more than two years.

[0069] Doctor: Hello, have you had any symptoms in the past two years, such as back pain or difficulty urinating?

[0070] Patient: Yes, I sometimes feel discomfort in my lower back and occasionally have difficulty urinating.

[0071] Doctor: I see. What related tests have you done? For example, urine routine or urinary system B-ultrasound?

[0072] Patient: I did a urinalysis and the results showed a slight erythrocytosis.

[0073] Doctor: I see. In addition to urinalysis, have you ever done B-ultrasound or CT scan of the urinary system?

[0074] Patient: Yes, the urinary system B-ultrasound showed signs of stenosis in the right ureter, and then I performed a plain scan CT scan.

[0075] Doctor: What specific information did the CT scan show?

[0076] Patient: CT results showed thickening of the wall of the lower segment of the right ureter and obvious hydrops in the upstream urinary tract.

[0077] Doctor: Have you had a magnetic resonance urography (MRU) or CT urography (CTU)?

[0078] Patient: Not yet, I just received these suggestions.

[0079] Doctor: I understand. Have you ever had a ureteroscopy?

[0080] Patient: No, the only tests currently available are urine routine, B-ultrasound and CT scan.

[0081] Doctor: OK, have you ever had any related surgery?

[0082] Patient: Yes, I had a hysterectomy.

[0083] Doctor: What was the reason for your hysterectomy?

[0084] Patient: Because I had uterine fibroids before.

[0085] Doctor: I see. Based on the information and symptoms you provided, combined with the results of the CT scan, my preliminary diagnosis at this stage is that there may be stenosis in the lower segment of the right ureter, resulting in thickening of the tube wall and hydroureteral accumulation in the upstream urinary tract. The slight thickening of the left adrenal gland may be related to your previous hysterectomy. To confirm the diagnosis, I recommend that you undergo MRU or CTU examination, as well as ureteroscopy.

[0086] For example, based on the single-round question-and-answer sample of the life assistant scenario shown in Example 2 above, the converted multi-round dialogue sample that conforms to the user's ticket purchase rules can be shown as follows:

[0087] User: Hello life assistant, please buy me a movie ticket.

[0088] Life Assistant: OK, which movie would you like to watch?

[0089] User: Movie A.

[0090] Life Assistant: OK, what time do you plan to go see this movie?

[0091] User: I'll go watch it after I finish my meal, any time in the afternoon is fine.

[0092] Life Assistant: Okay, then do you have any requirements for the cinema and seats?

[0093] User: I want a cinema that is closest to me, preferably one with middle seats. If there are no middle seats, I won’t consider it.

[0094] Life Assistant: OK, according to your request, I have selected Cinema B, which is showing Movie A and is closest to your current location. After noon, Movie A will be shown twice in Cinema B, at 2pm and 3pm. There are also middle seats available for the 3pm show, including 6th row, 15 seats, 6th row, 16 seats, 6th row, 17 seats, 7th row, 16 seats, 8th row, 15 seats, and 8th row, 16 seats. Please confirm your seat, and I will help you buy the movie ticket directly after confirmation.

[0095] Step S103, further fine-tuning the pre-trained LLM basic model based on the multi-round dialogue samples to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

[0096] Furthermore, after converting the single-round question and answer samples contained in the above data set into multi-round dialogue samples that conform to the multi-round dialogue rules, the pre-trained LLM basic model can be further fine-tuned based on the converted multi-round dialogue samples, thereby obtaining an LLM service model that can perform the multi-round dialogue task based on the multi-round dialogue rules.

[0097] For example, still taking the medical consultation scenario as an example, after the single-round question and answer samples contained in the above data set are converted into multi-round dialogue samples that conform to the medical consultation rules, the pre-trained LLM basic model can be further fine-tuned based on the converted multi-round dialogue samples to obtain an LLM service model that can perform multi-round dialogue tasks based on the medical consultation rules, such as a medical LLM model.

[0098] In one illustrated embodiment, see Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a multi-round dialogue that complies with medical consultation rules provided by an exemplary embodiment. Figure 2 As shown, the computing device is equipped with a fine-tuned medical LLM model, and the user can input his or her own consultation needs through the input device provided by the computing device (such as a keyboard or a touch screen) to ask questions to the medical LLM model.

[0099] Furthermore, if Figure 2 As shown in the figure, after obtaining the user's question, the medical LLM model can analyze the user's current situation and needs, and first ask the user what symptoms they currently have according to the medical consultation rules. Among them, when asking about symptoms, the medical LLM model can ask multiple questions based on the user's answer, and has obtained sufficient symptom information.

[0100] Furthermore, if Figure 2As shown, after obtaining sufficient symptom information from the user, the medical LLM model can ask the user what examinations have been done and the results of the examinations according to the medical consultation rules. Correspondingly, the user can answer the questions of the medical LLM model. In an embodiment shown, after obtaining the examination results answered by the user, if the analysis shows that the current examination is not sufficient, the user can be advised to do some more critical examination items in the future.

[0101] Furthermore, if Figure 2 As shown, the medical LLM model can also ask the user about the medical history related to the currently possible diagnosed disease according to the medical consultation rules to comprehensively analyze the user's condition. Correspondingly, the user can answer the inquiries of the medical LLM model.

[0102] Finally, if Figure 2 As shown, the medical LLM model can output corresponding medical diagnosis results based on various types of information (including symptoms, examination results, and medical history, etc.) obtained through the aforementioned multiple rounds of dialogue. For example, the medical diagnosis results may include, for example, a diagnosed disease, and a treatment plan or further examination suggestions given in combination with the needs of the user, which is not specifically limited in this specification.

[0103] As mentioned above, after fine-tuning the LLM basic model based on multi-round dialogue samples that conform to the medical consultation rules, the fine-tuned medical LLM model can communicate with patients in a more organized, logical, and effective manner according to the medical consultation rules, thereby improving the quality of medical services.

[0104] In addition, it should be noted that this specification does not specifically limit the specific implementation method of fine-tuning training. In one embodiment shown, the LLM basic model can be fine-tuned by using supervised fine-tuning (SFT); or, the LLM basic model can be fine-tuned by using preference fine-tuning (or preference learning), such as direct preference optimization (DPO), or reinforcement learning from human feedback (RLHF), etc. This specification does not specifically limit this.

[0105] In an illustrated embodiment, in order for the LLM model to better learn the implicit multi-round dialogue rules in the dialogue and align with the rules, preference pairs can be selected for preference learning. In addition, considering that DPO can directly optimize the degree to which the model satisfies human preferences, there is no need to explicitly define a reward function, nor is there a need to train a separate reward model to evaluate the model output, while RLHF usually requires training a reward model first to predict human preferences for model output, etc. Based on this, DPO can be preferably used to fine-tune the LLM basic model.

[0106] In one illustrated embodiment, see Figure 3 , Figure 3 FIG. 1 is a flow chart of another LLM model fine-tuning method provided by an exemplary embodiment. Figure 3 As shown, the DPO-based fine-tuning method can generally include two stages: a supervised fine-tuning stage and a preference optimization stage.

[0107] like Figure 3 As shown, in the supervised fine-tuning stage, the pre-trained LLM basic model can be supervised fine-tuned based on the multi-round dialogue samples converted from the single-round question-answering samples, and the supervised fine-tuned LLM basic model can be used as the policy model in the preference fine-tuning. Among them, this specification does not specifically limit the specific implementation process of supervised fine-tuning, which will not be elaborated here.

[0108] like Figure 3 As shown, in the preference optimization stage, the LLM basic model can be optimized based on the above-mentioned strategy model and multi-round dialogue samples. In an embodiment shown, when the LLM basic model is optimized based on the above-mentioned strategy model and multi-round dialogue samples, it can specifically include: taking the above-mentioned multi-round dialogue samples as positive preference samples, specifically, the dialogue text corresponding to the model contained in the multi-round dialogue samples can be taken as the positive preference sample; and, the dialogue text corresponding to the user contained in the above-mentioned multi-round dialogue samples is input into the strategy model, and the dialogue text output by the strategy model performing the multi-round dialogue task based on the dialogue text corresponding to the user is taken as the negative preference sample; further, the LLM basic model can be optimized based on the preference pair composed of the above-mentioned positive preference sample and the negative preference sample.

[0109] It should be noted that when preference pairs are used for preference optimization, the higher the quality of the preference pairs, the better the effect of preference optimization. Specifically, when the similarity between the dialogue text output by the policy model and the corresponding positive preference sample is greater (that is, the difference between the positive and negative samples is smaller), the calculated loss is closer to 0, which is not conducive to the model's understanding and learning of the preferences of positive and negative samples; conversely, when the similarity between the dialogue text output by the policy model and the corresponding positive preference sample is smaller (that is, the difference between the positive and negative samples is larger), the calculated loss is larger, which can prompt the model to better understand and learn the preferences of positive and negative samples and achieve better preference optimization effects.

[0110] In general, if only supervised fine-tuning is performed on the LLM basic model, the dialogue text output by the obtained policy model usually does not reach the ideal state (i.e., it cannot align with human preferences), and the difference between it and the positive preference samples is often large. However, in some possible cases, the dialogue text output by the policy model obtained after supervised fine-tuning of the LLM basic model may be relatively ideal, and the difference between it and the positive preference samples may be small. In this case, if the dialogue text output by the policy model is directly used as a negative preference sample, and it is used as a preference pair with the positive preference sample to optimize the preference of the LLM basic model, the final preference optimization effect is often very poor.

[0111] Based on this, this specification further provides a method for processing negative preference samples. First, multiple dialogue texts output by the policy model performing multiple rounds of dialogue tasks can be obtained, and then these dialogue texts can be screened and processed to obtain more appropriate negative preference samples.

[0112] In an illustrated embodiment, if the similarity between the dialogue text output by the policy model when executing a multi-round dialogue task and the corresponding positive preference sample is lower than a preset threshold, the dialogue text can be used as a negative preference sample.

[0113] For example, taking the multi-round dialogue samples in the medical consultation scenario described in the above step S102 as an example, based on the patient's first round of dialogue "Hello, doctor. I am a 62-year-old female patient. I recently discovered that I have had right ureteral stenosis for more than 2 years", the positive preference sample is "Hello, have you had any symptoms in the past two years, such as low back pain or difficulty urinating", and the dialogue text output by the strategy model may be "Hello, how can I help you". Obviously, the dialogue text output by the strategy model in this example is quite different from the corresponding positive preference sample. For example, if the similarity is less than a preset threshold, the dialogue text output by the strategy model "Hello, how can I help you" can be used as a negative preference sample.

[0114] For example, still taking the patient's first round of dialogue "Hello, doctor. I am a 62-year-old female patient. I recently found that I have had right ureteral stenosis for more than two years" as an example, the dialogue text output by the strategy model may be "Hello, have you had any symptoms in the past two years, such as low back pain, etc." Obviously, the difference between the dialogue text output by the strategy model in this example and the corresponding positive preference sample is small. For example, if the similarity is greater than the preset threshold, the dialogue text cannot be directly used as a negative preference sample.

[0115] In an illustrated implementation, the dialogue text output by the policy model when executing multiple rounds of dialogue tasks may also be repeated, and the repeated dialogue text may be used as a negative preference sample.

[0116] For example, taking the patient's second round of dialogue "Yes, I sometimes feel discomfort in my waist, and occasionally feel a little difficult to urinate" as an example, the positive preference sample is "I understand. What related examinations have you done? For example, urine routine or urinary system B-ultrasound", and the dialogue text output by the strategy model may be "Have you done urine routine or urinary system B-ultrasound?" Obviously, the difference between the dialogue text output by the strategy model in this example and the corresponding positive preference sample is small, and it cannot be directly used as a negative preference sample. At this time, the dialogue text output by the strategy model in the first round of dialogue (for example, "Hello, have you had any symptoms, such as low back pain, etc. in the past two years") can be repeated, and the repeated dialogue text can be used as a negative preference sample in the second round of dialogue. In other words, "Hello, have you had any symptoms, such as low back pain, etc. in the past two years" can be used as a negative preference sample in the second round of dialogue, which is different from the above positive preference sample "I understand. What related examinations have you done? For example, urine routine or urinary system B-ultrasound" and can constitute a high-quality preference pair.

[0117] In an illustrated implementation, the order of the dialogue texts output by the policy model when executing multiple rounds of dialogue tasks may also be disrupted, and the dialogue texts after the disrupted order are used as negative preference samples.

[0118] For example, still taking the patient's second round of dialogue "Yes, I sometimes feel discomfort in my waist, and occasionally feel a little difficult to urinate" as an example, the positive preference sample is "I understand. What related examinations have you done? Such as routine urine tests or urinary system B-ultrasound", the dialogue text output by the strategy model may be "Have you done urine routine tests or urinary system B-ultrasound tests?" Obviously, this dialogue text cannot be directly used as a negative preference sample. At this time, the order of the dialogue text output by the strategy model can be disrupted. For example, the dialogue text output by the strategy model in the subsequent eighth round of dialogue (such as "Why did you have a hysterectomy") is used as a negative preference sample in the second round of dialogue. It is quite different from the above positive preference sample "I understand. What related examinations have you done? Such as routine urine tests or urinary system B-ultrasound", and can constitute a high-quality preference pair.

[0119] It should be noted that the above method of obtaining negative preference samples is only an example. In addition, negative preference samples in preference pairs can be obtained by any other possible method, and this specification does not make any specific limitations on this.

[0120] In summary, when the present application fine-tunes the pre-trained LLM basic model so that it can ideally perform multi-round dialogue tasks related to the target application scenario, it is considered that the single-round question and answer samples contained in the data set directly obtained and related to the target application scenario are often composed of only a question text and a corresponding answer text. If the LLM basic model is fine-tuned directly based on the single-round question and answer sample, the fine-tuned model is prone to logical confusion when performing multi-round dialogue tasks, such as repeatedly asking the same question to the user, or directly outputting the final inference without obtaining sufficient information, and the effect is extremely poor. Based on this, the present application can first convert the above-mentioned single-round question and answer samples into multi-round dialogue samples that conform to the preset multi-round dialogue rules, and then use these multi-round dialogue samples to fine-tune the LLM basic model so that the model can learn the multi-round dialogue rules. Correspondingly, the fine-tuned model will also perform multi-round dialogue tasks according to the multi-round dialogue rules, so as to respond logically, improve model performance, and improve user experience.

[0121] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides an LLM model fine-tuning device. The device can be applied to a computing device, such as a smart wearable device, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., which is not specifically limited in this specification. Please refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a structure of an LLM model fine-tuning device provided by an exemplary embodiment. Figure 4 As shown, the device 40 includes:

[0122] A question-and-answer data acquisition unit 401 is used to acquire a data set related to a target application scenario; the data set includes a plurality of single-round question-and-answer samples, each of which includes a question text and an answer text corresponding to the question text;

[0123] A multi-round dialogue sample generating unit 402 is used to obtain a multi-round dialogue rule corresponding to the multi-round dialogue task based on the task requirements of the multi-round dialogue task related to the target application scenario, and convert the single-round question and answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule;

[0124] The fine-tuning unit 403 is used to further perform fine-tuning training on the pre-trained LLM basic model based on the multi-round dialogue samples to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

[0125] In one illustrated embodiment, the target application scenario is a medical consultation scenario, the multi-round dialogue rules include medical consultation rules, and the task requirements include obtaining a medical diagnosis result.

[0126] In an illustrated embodiment, the medical consultation rule includes a medical consultation sequence; wherein the medical consultation sequence includes sequentially performing symptom inquiry, examination result inquiry, medical history inquiry, and outputting a medical diagnosis result.

[0127] In an illustrated embodiment, the medical consultation rules further include at least one inquiry rule shown below:

[0128] When conducting symptom inquiry, ask about symptoms multiple times until all symptoms obtained include several symptoms that can be used to diagnose any target disease;

[0129] When inquiring about the test results, inquiring about the test results of several test items used to confirm the target disease;

[0130] When taking a medical history, ask about the medical history related to the target disease.

[0131] In an illustrated embodiment, the question and answer data acquisition unit 401 is specifically used to:

[0132] Based on the acquired clinical data of virtual patients corresponding to different disease categories, single-round question-and-answer samples corresponding to different disease categories are generated.

[0133] In an illustrated embodiment, the multi-round dialogue sample generating unit 402 is specifically configured to:

[0134] The disease names contained in the single-round question and answer samples are mapped to corresponding disease categories, and the single-round question and answer samples after the disease name mapping are converted into multi-round dialogue samples that comply with the multi-round dialogue rules.

[0135] In an illustrated embodiment, the fine-tuning unit 403 is specifically configured to:

[0136] Based on the multi-round dialogue samples, preference fine-tuning is performed on the pre-trained LLM basic model.

[0137] In an illustrated embodiment, the fine-tuning unit 403 is specifically configured to:

[0138] Based on the multi-round dialogue samples, supervised fine-tuning is performed on the pre-trained LLM basic model, and the supervised fine-tuned LLM basic model is used as the strategy model in preference fine-tuning;

[0139] Based on the strategy model and the multi-round dialogue samples, the LLM basic model is subjected to preference optimization.

[0140] In an illustrated embodiment, the fine-tuning unit 403 is specifically configured to:

[0141] Taking the dialogue texts corresponding to the model contained in the multi-round dialogue samples as positive preference samples;

[0142] Inputting the dialogue text corresponding to the user contained in the multi-round dialogue sample into the strategy model, and using the dialogue text output by the strategy model when executing the multi-round dialogue task as a negative preference sample;

[0143] Based on the preference pairs formed by the positive preference samples and the negative preference samples, preference optimization is performed on the LLM basic model.

[0144] In an illustrated embodiment, the fine-tuning unit 403 is specifically configured to:

[0145] If the similarity between the dialogue text output by the strategy model when executing the multi-round dialogue task and the corresponding positive preference sample is lower than a preset threshold, the dialogue text is used as a negative preference sample; and / or,

[0146] Repeating the dialogue text output by the strategy model when executing the multi-round dialogue task, and using the repeated dialogue text as a negative preference sample; and / or,

[0147] The order of the dialogue texts output by the policy model when executing the multi-round dialogue task is disrupted, and the dialogue texts after the disrupted order are used as negative preference samples.

[0148] The implementation process of the functions and effects of each unit in the above-mentioned device 40 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 40 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0149] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0150] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0151] Corresponding to the above method embodiment, the embodiment of this specification also provides a computing device. Figure 5 , Figure 5 FIG. 1 is a schematic diagram of a computing device provided by an exemplary embodiment. Figure 5 As shown, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Figure 5As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be executed by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the LLM model fine-tuning method in the embodiment of this specification, including: obtaining a data set related to the target application scenario; the data set includes multiple single-round question and answer samples, each single-round question and answer sample contains a question text and an answer text corresponding to the question text; obtaining a multi-round dialogue rule corresponding to the multi-round dialogue task constructed based on the task requirements of the multi-round dialogue task related to the target application scenario, and converting the single-round question and answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule; further performing fine-tuning training on the pre-trained LLM basic model based on the multi-round dialogue sample to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rule, and so on.

[0152] For a detailed description of each step of the above-mentioned LLM model fine-tuning method, please refer to the previous content and will not be repeated here.

[0153] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored. When these computer programs are executed by a processor, they execute the various steps of the LLM model fine-tuning method in the embodiment of this specification. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0154] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0155] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0156] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0157] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0158] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0159] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0160] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A LLM model fine-tuning method, characterized in that: The method comprises: Obtain a data set related to the target application scenario; the data set includes multiple single-round question and answer samples, each single-round question and answer sample includes a question text and an answer text corresponding to the question text; Obtaining a multi-round dialogue rule corresponding to a multi-round dialogue task based on a task requirement related to the target application scenario, and converting the single-round question-and-answer sample into a multi-round dialogue sample that complies with the multi-round dialogue rule; Based on the multi-round dialogue samples, the pre-trained LLM basic model is further fine-tuned to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

2. The method according to claim 1, characterized in that The target application scenario is a medical consultation scenario, the multi-round dialogue rules include medical consultation rules, and the task requirements include obtaining medical diagnosis results.

3. The method according to claim 2, characterized in that The medical consultation rules include a medical consultation sequence; wherein the medical consultation sequence includes symptom inquiry, examination result inquiry, medical history inquiry and output of medical diagnosis results in sequence.

4. The method according to claim 3, characterized in that The medical inquiry rules further include at least one inquiry rule shown below: When conducting symptom inquiries, ask about symptoms multiple times until all the symptoms obtained include several symptoms that can be used to diagnose any target disease; When inquiring about the test results, inquiring about the test results of several test items used to confirm the target disease; When taking a medical history, inquire about the medical history related to the target disease.

5. The method according to claim 2, characterized in that The step of obtaining a data set related to the target application scenario includes: Based on the obtained clinical data of virtual patients corresponding to different disease categories, single-round question and answer samples corresponding to different disease categories are generated.

6. The method according to claim 5, characterized in that Converting the single-round question-and-answer sample into a multi-round dialogue sample that complies with the multi-round dialogue rule includes: The disease names contained in the single-round question and answer samples are mapped to corresponding disease categories, and the single-round question and answer samples after the disease name mapping are converted into multi-round dialogue samples that comply with the multi-round dialogue rules.

7. The method according to claim 1, characterized in that The further performing fine-tuning training on the pre-trained LLM basic model based on the multi-round dialogue samples includes: Based on the multi-round dialogue samples, preference fine-tuning is performed on the pre-trained LLM basic model.

8. The method according to claim 7, characterized in that The preference fine-tuning of the pre-trained LLM basic model based on the multi-round dialogue samples includes: Performing supervised fine-tuning on the pre-trained LLM basic model based on the multi-round dialogue samples, and using the supervised fine-tuned LLM basic model as the policy model in preference fine-tuning; Based on the strategy model and the multi-round dialogue samples, preference optimization is performed on the LLM basic model.

9. The method according to claim 8, characterized in that The performing preference optimization on the LLM basic model based on the strategy model and the multi-round dialogue samples includes: Taking the dialogue texts corresponding to the model contained in the multi-round dialogue samples as positive preference samples; Inputting the conversation text corresponding to the user contained in the multi-round conversation sample into the strategy model, and using the conversation text output by the strategy model when executing the multi-round conversation task as a negative preference sample; Based on the preference pairs formed by the positive preference samples and the negative preference samples, preference optimization is performed on the LLM basic model.

10. The method according to claim 9, characterized in that The step of using the dialogue text output by the policy model when executing the multi-round dialogue task as a negative preference sample includes: If the similarity between the dialogue text output by the strategy model when executing the multi-round dialogue task and the corresponding positive preference sample is lower than a preset threshold, the dialogue text is used as a negative preference sample; and / or, Repeating the dialogue text output by the policy model when executing the multi-round dialogue task, and using the repeated dialogue text as a negative preference sample; and / or, The order of the dialogue texts output by the policy model when executing the multi-round dialogue task is disrupted, and the dialogue texts after the disrupted order are used as negative preference samples.

11. An LLM model fine-tuning device, characterized in that: The device comprises: A question-and-answer data acquisition unit, configured to acquire a data set related to a target application scenario; the data set includes a plurality of single-round question-and-answer samples, each of which includes a question text and an answer text corresponding to the question text; a multi-round dialogue sample generation unit, configured to obtain a multi-round dialogue rule corresponding to a multi-round dialogue task based on the task requirements of the multi-round dialogue task related to the target application scenario, and convert the single-round question-and-answer sample into a multi-round dialogue sample that conforms to the multi-round dialogue rule; A fine-tuning unit is used to further perform fine-tuning training on the pre-trained LLM basic model based on the multi-round dialogue samples to obtain an LLM service model that performs the multi-round dialogue task based on the multi-round dialogue rules.

12. A computing device, characterized in that include: memory and processor; The memory stores computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, it performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images