Guidance model training method and device, storage medium and program product

Abstract

The method comprises the following steps: extracting question data from pre-diagnosis dialogue data; obtaining a reference standard answer based on diagnosis data; inputting the question data into a collaborative model and a basic model respectively to obtain different replies, and constructing preference data in combination with the question data and the reference standard answer; generating a virtual QRA data set based on the diagnosis data, fine-tuning the basic model to obtain a reference model; introducing a safety constraint rule, training a guide diagnosis model to obtain an expected model in combination with the preference data and the reference model. The method can introduce a safety constraint rule in the guide diagnosis model reinforcement learning stage, clearly define the "safety red line", and ensure that the guide diagnosis model output is safe and controllable. Compared with the traditional DPO training method of constructing preference data by only using the guide diagnosis model and the basic model, the method introduces a collaborative model, makes the data more diverse, and enables the guide diagnosis model to learn more extensive human preference patterns in the training.

Description

Technical Field

[0001] This disclosure relates to the field of model training technology, and in particular to a training method, apparatus, storage medium, and program product for a triage model. Background Technology

[0002] In the medical field, intelligent triage systems use large models to obtain users' chief complaint information through natural language interaction and provide medical advice accordingly. The aim is to help users quickly find the right department and doctor, improve medical efficiency, reduce the workload of hospital triage staff, and optimize the user's medical experience.

[0003] However, current triage models face numerous challenges in practical applications. On the one hand, users often lack medical knowledge, leading to inaccurate descriptions and biases in the chief complaint information obtained by the triage model, thus affecting the accuracy of its output consultation suggestions. On the other hand, while structured medical data generated during the consultation phase, such as electronic medical records and examination reports, contains rich medical information, this data has not been fully utilized to guide the optimization training of pre-diagnosis models. This makes it difficult for triage models to learn and improve from actual treatment outcomes, limiting their performance enhancement. Summary of the Invention

[0004] In view of this, the present disclosure provides a training method, apparatus, storage medium, and program product for a triage model. It utilizes safety constraint rules, which are explicit "safety red lines" introduced during the reinforcement learning phase of triage model training, to ensure the safety, reliability, controllability, and interpretability of the model output. Compared to the traditional DPO training method that only utilizes the triage model and the basic model to construct preference data, this solution introduces an additional collaborative model to construct preference data. This makes the preference data richer and more diverse, allowing the triage model to access more diverse information during training and thus learn a wider range of human preference patterns.

[0005] Firstly, this disclosure provides a method for training a triage model, employing the following technical solution:

[0006] Acquire pre-diagnosis dialogue data and in-diagnosis data, and extract question data from the pre-diagnosis dialogue data;

[0007] Based on the diagnostic data, obtain the reference standard answer for the question data;

[0008] The question data is input into the collaborative model and the basic model respectively, and the first response output by the collaborative model and the second response output by the basic model are obtained.

[0009] Based on the question data, the content of the first response, the content of the second response, and the reference standard answer, preference data is constructed;

[0010] Based on the diagnostic data, a virtual QRA dataset is obtained;

[0011] The base model is supervised and fine-tuned based on the virtual QRA dataset to obtain a reference model;

[0012] Obtain safety constraint rules, and train the triage model based on the preference data, the safety constraint rules, and the reference model to obtain the desired model.

[0013] Optionally, obtaining the reference standard answer for the question data based on the diagnostic data includes:

[0014] The diagnostic data is preprocessed to obtain key data;

[0015] Based on the user's identifier and the timeline of medical visits, the problem data, the key data, and the in-diagnosis data are aligned.

[0016] The aligned question data, key data, and diagnostic data are input into the teacher's large language model to generate a reference standard answer for each question data.

[0017] Optionally, constructing preference data based on the question data, the content of the first response, the content of the second response, and the reference standard answer includes:

[0018] Based on the question data and the corresponding reference standard answers, the content of the first response and the content of the second response are compared, and based on the comparison results, high-quality response content and low-quality response content are determined.

[0019] The question data, the high-quality response content, and the low-quality response content are combined into preference data.

[0020] Optionally, obtaining the virtual QRA dataset based on the diagnostic data includes:

[0021] QRA samples were constructed according to disease types;

[0022] The diagnostic data is split to obtain diagnostic and treatment data sets for different disease types;

[0023] According to the disease type, the QRA sample and the set of diagnostic and treatment data are filled into a preset template to form multiple prompt messages;

[0024] Each prompt message is input into the teacher model, and the virtual QRA data output by the teacher model constitutes a virtual QRA dataset.

[0025] Optionally, the step of obtaining security constraint rules, based on the preference data, the security constraint rules, and the reference model, trains the triage model to obtain a desired model, including:

[0026] Construct an initial empty preference dataset;

[0027] The current preference data is determined through a traversal operation, and high-quality and low-quality response content is extracted from the current preference data.

[0028] Determine whether the content of the high-quality response complies with the security constraint rules;

[0029] If the content of the high-quality response conforms to the security constraint rules, then the current preference data is stored in the preference dataset;

[0030] If the content of the high-quality response does not comply with the security constraint rules, then it is determined whether the content of the non-high-quality response complies with the security constraint rules.

[0031] If the non-high-quality response content meets the security constraint rules, then the high-quality response content and the non-high-quality response content are reversed to form new preference data, and the new preference data is stored in the preference dataset.

[0032] If the non-quality response content does not comply with the security constraint rules, the current preference data will be deleted.

[0033] After all preference data has been traversed, the triage model is trained based on the preference dataset and the reference model to obtain the desired model.

[0034] Optionally, the step of obtaining security constraint rules, based on the preference data, the security constraint rules, and the reference model, trains the triage model to obtain a desired model, including:

[0035] Based on the question data, high-quality and low-quality answer content contained in the preference data, and the safety constraint rules contained in the rule list, the SC-DPO loss function is constructed.

[0036] The triage model is trained based on the reference model and the SC-DPO loss function to obtain the desired model.

[0037] Optionally, the expression for the SC-DPO loss function is:

[0038]

[0039] In the formula, Represents the SC-DPO loss function; This represents the distribution of the preference dataset; This represents all preference data samples obtained from distribution D. ; This represents the problem data; This indicates a high-quality response. This indicates a non-high-quality response. This represents the sigmoid function; Indicates hyperparameters; This represents the patient guidance model; These represent the parameters of the patient guidance model; Represents the reference model; Indicates the parameters of the reference model; Expressing expectations; This indicates that the triage model, given input Generate a response under the following conditions The probability of; This indicates that the reference model is in response to a given input. Generate a response under the following conditions The probability of; This indicates that the triage model, given input Generate a response under the following conditions The probability of; This indicates that the reference model is in response to a given input. Generate a response under the following conditions The probability of; This indicates the hyperparameter for safety penalty weights; Represents a list of rules; This represents the safety penalty function, which is applied to the triage model given the input. The response generated under the condition If the security constraint rules in the rule list are not met, then If the triage model is given input The response generated under the condition If the security constraints in the rule list are met, then .

[0040] Secondly, this disclosure also provides a training system for a triage model, which adopts the following technical solution:

[0041] A data acquisition module is prepared to acquire pre-diagnosis dialogue data and in-diagnosis data, and to extract question data from the pre-diagnosis dialogue data.

[0042] The standard answer acquisition module is used to acquire reference standard answers to the question data based on the diagnostic data.

[0043] The response content acquisition module is used to input the question data into the collaborative model and the basic model respectively, and acquire the first response content output by the collaborative model and the second response content output by the basic model;

[0044] A preference data construction module is used to construct preference data based on the question data, the content of the first response, the content of the second response, and the reference standard answer;

[0045] The virtual data acquisition module is used to acquire a virtual QRA dataset based on the diagnostic data.

[0046] The model supervised fine-tuning module is used to supervise and fine-tune the base model based on the virtual QRA dataset to obtain a reference model;

[0047] The triage model training module is used to obtain safety constraint rules, and to train the triage model based on the preference data, the safety constraint rules, and the reference model to obtain the desired model.

[0048] Thirdly, this disclosure also provides a computer device, which adopts the following technical solution:

[0049] The computer device includes:

[0050] At least one processor; and,

[0051] A memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the training method of any of the above-described triage models.

[0053] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing computer instructions for causing a computer to execute the training method of any of the above-described triage models.

[0054] Fifthly, embodiments of this disclosure also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0055] The training method for the triage model provided in this disclosure comprehensively collects patient information at different stages of the medical process by acquiring pre-diagnosis dialogue data and in-diagnosis data. Question data is extracted from the pre-diagnosis dialogue data, accurately identifying patients' questions and needs, providing clear input for the model, and ensuring that the model learns and optimizes based on actual patient questions. Reference standard answers to the question data are obtained based on the in-diagnosis data, providing a reliable benchmark for the triage model training. These standard answers are derived from the actual diagnostic process and have high accuracy and authority, giving the triage model a clear goal during learning and enabling it to optimize in the right direction, improving the accuracy of triage results. The question data is processed using a collaborative model and a basic model to obtain different responses. These two models have different characteristics and advantages, and their outputs are complementary, providing diverse choices for subsequent construction of preference data, increasing the richness and comprehensiveness of the data. By integrating question data, responses from different models, and reference standard answers to construct preference data, the differences and strengths of various responses compared to the standard answer can be fully reflected. This preference data provides rich learning materials for the triage model, enabling it to learn a wider range of human preference patterns during training and improving its adaptability and generalization ability to different situations. A virtual QRA dataset, obtained based on in-diagnosis data, further enriches the type and scale of training data. The virtual QRA dataset can simulate various possible pre-diagnosis and in-diagnosis scenarios, providing more learning samples for the basic model. This helps the basic model better understand and handle complex medical problems, improving its performance and accuracy. Supervised fine-tuning of the basic model using the virtual QRA dataset allows it to better adapt to the needs of medical triage scenarios. Through fine-tuning, the reference model can learn the patterns and rules in the virtual dataset, providing more accurate references and guidance for subsequent triage model training. Obtaining safety constraint rules and introducing them into the triage model training sets clear boundaries and specifications for the model's output, ensuring the safety and reliability of the triage results. By combining preference data and reference models to train the triage model, the triage model can learn human preferences while following safety rules, thereby obtaining an expected model that is both in line with user preferences and safe and reliable, improving the quality and efficiency of triage and better meeting user needs.

[0056] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A flowchart illustrating the training method for the triage model provided in this embodiment of the disclosure;

[0059] Figure 2 A flowchart illustrating the reference standard answer generation method provided in this embodiment of the disclosure;

[0060] Figure 3 A flowchart illustrating the preference data construction method provided in this embodiment of the disclosure;

[0061] Figure 4 A flowchart illustrating the method for obtaining a virtual QRA dataset provided in this embodiment of the disclosure;

[0062] Figure 5 A flowchart illustrating the method for obtaining security constraint rules provided in this embodiment of the disclosure;

[0063] Figure 6 A flowchart illustrating the triage model training method provided in this embodiment of the disclosure;

[0064] Figure 7 Another schematic diagram of the process for training the triage model provided in this embodiment of the present disclosure;

[0065] Figure 8 A schematic diagram of the training system for the triage model provided in this embodiment of the disclosure;

[0066] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. Detailed Implementation

[0067] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0068] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0069] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0070] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0071] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0072] Reference Figure 1 This disclosure provides a training method for a triage model, including the following steps:

[0073] S1: Obtain pre-diagnosis dialogue data and in-diagnosis data, and extract question data from the pre-diagnosis dialogue data;

[0074] S2: Based on the diagnostic data, obtain the reference standard answers for the problem data;

[0075] S3: Input the problem data into the collaborative model and the basic model respectively, and obtain the first response output by the collaborative model and the second response output by the basic model;

[0076] S4: Construct preference data based on question data, the content of the first response, the content of the second response, and the reference standard answer;

[0077] S5: Obtain a virtual QRA dataset based on in-diagnosis data;

[0078] S6: Perform supervised fine-tuning of the base model based on the virtual QRA dataset to obtain a reference model;

[0079] S7: Obtain safety constraint rules. Based on preference data, safety constraint rules, and reference models, train the triage model to obtain the desired model.

[0080] The training method for the triage model disclosed herein comprehensively collects patient information at different stages of the medical process by acquiring pre-diagnosis dialogue data and in-diagnosis data. Question data is extracted from the pre-diagnosis dialogue data, accurately identifying patients' questions and needs, providing clear input for the model, and ensuring that the model learns and optimizes based on actual patient questions. Reference standard answers to the question data are obtained based on the in-diagnosis data, providing a reliable benchmark for the triage model training. These standard answers are derived from the actual diagnostic process and have high accuracy and authority, giving the triage model a clear goal during learning and enabling it to optimize in the right direction, improving the accuracy of triage results. The question data is processed using a collaborative model and a basic model to obtain different responses. These two models have different characteristics and advantages, and their outputs are complementary, providing diverse choices for subsequent construction of preference data, increasing the richness and comprehensiveness of the data. By combining question data, responses from different models, and reference standard answers to construct preference data, the differences and strengths of different responses compared to the standard answer can be fully reflected. This preference data provides rich learning materials for the triage model, enabling it to learn a wider range of human preference patterns during training and improve the model's adaptability and generalization ability to different situations.

[0081] By acquiring a virtual QRA dataset based on in-diagnosis data, the types and scale of training data are further enriched. The virtual QRA dataset can simulate various possible pre-diagnosis and in-diagnosis scenarios, providing more learning samples for the base model. This helps the base model better understand and handle complex medical problems, improving its performance and accuracy. Supervised fine-tuning of the base model using the virtual QRA dataset allows it to better adapt to the needs of medical triage scenarios. Through fine-tuning, the reference model can learn the patterns and rules in the virtual dataset, providing more accurate reference and guidance for the subsequent training of the triage model.

[0082] By acquiring safety constraint rules and incorporating them into the training of the triage model, clear boundaries and specifications are set for the model's output, ensuring the safety and reliability of the triage results. Training the triage model using preference data and a reference model enables it to learn human preferences while adhering to safety rules, resulting in a desired model that is both user-friendly and safe, thus improving the quality and efficiency of triage and better meeting user needs.

[0083] In S1, pre-diagnosis dialogue data is extracted from the natural language interaction data between users and the pre-diagnosis big data model. This data includes natural language question-and-answer pairs formed by different users' proactive descriptions of their chief complaints, symptom questions and answers, past medical history, and personal information. Because the language expression in pre-diagnosis dialogue data is relatively free, the information is relatively preliminary, and it is from the user's perspective, it is necessary to use natural language processing techniques, such as text parsing and semantic understanding algorithms, to preprocess and analyze these question-and-answer pairs in order to extract the question data, such as the symptom descriptions and medical history-related questions raised by the user.

[0084] Data obtained from hospital information systems during diagnosis is presented in structured or semi-structured formats, including electronic medical records, doctor-written progress notes, laboratory reports, and diagnostic conclusions. This data is characterized by accuracy, complete structure, and an expert perspective. By integrating pre-diagnosis dialogue data and in-diagnosis data, and comparing the problematic data with diagnostic conclusions and medical orders within the in-diagnosis data, the accuracy and completeness of both data are further verified and supplemented.

[0085] To ensure data security, user identity information (such as name) in pre-diagnosis dialogue data and in-diagnosis data is replaced with a unique identifier automatically generated by the system. This identifier is only used for internal system statistics and data association and does not contain any personally identifiable information, thereby effectively avoiding security risks.

[0086] Reference Figure 2 The flowchart illustrating the method for generating reference standard answers, "Obtaining reference standard answers for question data based on diagnostic data," includes the following steps:

[0087] S21: Preprocess the diagnostic data to obtain key data;

[0088] S22: Align problem data, key data, and in-diagnosis data based on user identification and timeline of medical visits;

[0089] S23: Input the aligned question data, key data, and diagnostic data into the teacher's large language model to generate a reference standard answer for each question data.

[0090] In S21, preprocessing includes initial data acquisition, data cleaning and standardization. Initial data includes chief complaint, present illness, past medical history, examination results and medical orders. Data cleaning and standardization refers to removing redundant information, standardizing medical terminology and handling missing values.

[0091] In the diagnostic data, specific keywords, such as "chief complaint," are found through text matching. Following these keywords, a continuous segment of text is extracted as the chief complaint. For example, if the text contains "chief complaint: cough for 3 days," then "cough for 3 days" is extracted as the chief complaint. For complex text structures, regular expressions can be used for precise matching to ensure accurate extraction of the chief complaint information.

[0092] Similarly, the method of matching identifiers is used to find the "present illness" identifier. The extracted text should then describe in detail the development of the symptoms, including the time of onset, changes, and factors that relieve or aggravate the symptoms. For example, "Present illness: The user developed a cough 3 days ago without any obvious cause. It was initially a single cough and was not taken seriously. In the past 2 days, the cough has worsened and is accompanied by sputum." This entire passage should be extracted as the present illness information.

[0093] Past medical history includes information on allergies, surgeries, and more. The "past medical history" marker is located in the medical record text, and then different types of past medical history information are further identified. For allergy history, if the text mentions "past medical history: history of penicillin allergy," then "penicillin allergy history" is extracted as allergy history information. For surgical history, such as "previous appendectomy," then "appendectomy" is extracted as surgical history information.

[0094] The examination results include imaging reports, laboratory test results, etc. Search for keywords such as "examination results" and "examination" to extract relevant information. Imaging reports may contain information such as "chest X-ray shows increased lung markings," and laboratory test results may contain information such as "blood routine test indicates elevated white blood cell count." Accurately extract this information.

[0095] Medical orders typically include medication regimens and recommended examinations. Extract relevant information by searching for the "medical order" label. A medication regimen might be "oral amoxicillin capsules, 0.5g three times daily," and a recommended examination might be "further chest CT scan recommended." Extract and organize these details separately.

[0096] Removing redundant information involves examining the extracted key information and removing redundant content such as duplicate records and process logs. For example, identical symptom descriptions or operation records appearing repeatedly in different locations are retained only once. At the same time, irrelevant procedural text, such as the doctor's thought process or notes during recording, is deleted to ensure data simplicity.

[0097] Standardizing medical terminology involves creating a standardized dictionary of medical terms to unify disseminated medical terminology into standard terms. For example, "hypertension" is standardized to "primary hypertension," and "diabetes" to "type 2 diabetes." By extracting key information, non-standard terms are identified and replaced according to the dictionary, ensuring consistency and accuracy in medical terminology.

[0098] Handling missing values ​​involves uniformly marking any missing information in key data. For example, if allergy history information is missing, it is marked as "not recorded"; if part of the test results is missing, it is also marked accordingly, such as "no relevant test result record". This facilitates subsequent data statistics and analysis and avoids errors caused by missing values.

[0099] Store pre-processed and standardized key data in appropriate storage media, such as databases or new CSV files. The stored data structure should be clear to facilitate subsequent querying, analysis, and use. Different types of key information can be stored in different fields or tables to improve data organization and manageability.

[0100] In S22, problem data, key data, and diagnosis data all contain unique user identification information. By comparing user identifiers, problem data, key data, and diagnosis data belonging to the same user are linked together to form a preliminary dataset centered on the user.

[0101] After establishing associations based on unique identifiers, the key element of a consultation timeline is introduced. The specific time a user asks a question is recorded in the question data, while information such as the start time of the consultation, examination time, and diagnosis time is retrieved from the key data and consultation data. The question data, key data, and consultation data are then sorted chronologically, and preliminary matching is performed between the question data and the key and consultation data within their respective time periods. For example, if a user asks a question about a symptom at a certain moment before their consultation, relevant diagnoses and examinations before and after that moment are searched in the key and consultation data, allowing the data to initially correspond in the time dimension.

[0102] Based on initial time matching, entity extraction is performed on question data, key data, and diagnostic data. Key entity information such as symptoms, diseases, and body parts is identified from the question data. For example, for the question "I've had a terrible headache these past few days, what's wrong?", the extracted entity is "headache". The same entity extraction operation is performed on key data and diagnostic data. For instance, if the diagnostic report mentions "migraine causing headache", entities such as "migraine" and "headache" are extracted. Through entity extraction, core information is extracted from the data, preparing for subsequent semantic analysis.

[0103] Semantic similarity analysis is used to calculate the similarity between entities extracted from the problem data and entities extracted from the key data and diagnostic data. For example, "headache" and "headache" have high semantic similarity and can be considered strongly related; while "headache" and "migraine," although not exactly the same, are closely related in the medical context. Based on the semantic similarity results, the matching relationship between the problem data, key data, and diagnostic data is further adjusted and determined. For data with highly similar entities, they are more precisely bound together, thereby enhancing the accuracy of data alignment and ensuring a high degree of semantic consistency among the problem data, key data, and diagnostic data.

[0104] Although a correspondence between problem data, key data, and diagnostic data was established through unique identifiers, timeline matching, entity extraction, and semantic similarity analysis, some inaccuracies or ambiguities may still exist. Therefore, manual review is conducted, with professional medical personnel or data auditors checking the results of the automated matching to verify the reasonableness of the association between problem data and key / diagnostic data, and to identify any incorrect matches or omissions. Any issues discovered are promptly corrected to ensure accurate data alignment and provide a reliable data foundation for generating subsequent reference standard answers.

[0105] In S23, the aligned question data, key data, and diagnostic data are organized according to the input format required by the teacher's large language model. This means that question data is input as the question portion, key data as important prompts, and diagnostic data as background knowledge, all simultaneously input into the model. Upon receiving the input data, the teacher's large language model uses its pre-trained knowledge and algorithms for reasoning, comprehensively considering the important information provided by the question data and key data, as well as the background knowledge from the diagnostic data, to generate a reference standard answer for each question. The LLM language model is then used to summarize and convert the reference standard answers output by the teacher's large language model into clear and standardized medical expressions. By removing colloquial and vague expressions, the reference standard answers become more professional and accurate. For example, common symptom descriptions are converted into standard medical terminology, and unclear medical orders are transformed into standardized medication regimens and dosage instructions, ensuring that the final reference standard answers conform to the norms and requirements of the medical field.

[0106] The obtained reference answers are reviewed by reviewers who possess certain medical knowledge and are able to judge the reasonableness and accuracy of the answers, checking whether they conform to medical logic and are consistent with clinical data and key data. Answers that do not meet the requirements are corrected to improve the quality of the generated reference answers.

[0107] In S3, for each question's data, input is shared into both the collaborative model and the basic model. Both models analyze and output corresponding responses. The response output by the collaborative model is defined as the first response, and the response output by the basic model is defined as the second response. The collaborative model can be Deepseek-R1, and the basic model can be Qwen3 or Qwen / Qwen1.5 - 7B - Chat.

[0108] In S4, refer to Figure 3 The flowchart illustrating the preference data construction method shows that "constructing preference data based on question data, first response content, second response content, and reference standard answer" includes the following steps:

[0109] S41: Based on the question data and the corresponding reference standard answers, compare the content of the first response with the content of the second response, and determine the high-quality response and the low-quality response based on the comparison results;

[0110] S42: Combine question data, high-quality response content, and low-quality response content into preference data.

[0111] In the above steps, we collect pre-diagnosis user question data, such as: "Which clinic should I go to if I have a headache, fever and sore throat?"; obtain the reference standard answer corresponding to the question data, such as: "ENT"; and obtain the response content generated by the collaborative model and the basic model for the question data, such as: the first response content output by the collaborative model: "ENT", and the second response content output by the basic model: "Internal Medicine".

[0112] By comparing the responses generated by the collaborative model and the base model with the reference standard answer, and evaluating the accuracy, completeness, and logical coherence of each response, it is determined which response is closer to the reference standard answer. For example, the first response is completely consistent with the reference standard answer, while the second response, although relevant, is not accurate enough. Therefore, the first response output by the collaborative model is superior to the second response output by the base model.

[0113] Based on the comparative analysis results, a comparative response sample was constructed, in the following format:

[0114] completion_a: The result generated by model b, e.g., "Otolaryngology".

[0115] completion_b: The result generated by model b, e.g., "Internal Medicine".

[0116] preference: a(chosen)>b(rejected)

[0117] Where a refers to the collaborative model and b refers to the basic model.

[0118] The comparative response sample has precisely specified the superiority and inferiority attributes of the first and second response content. Therefore, it is possible to extract high-quality and low-quality response content from the comparative response sample and combine the question data, high-quality response content and low-quality response content into preference data.

[0119] In the above method, by combining question data, reference standard answers, and the output results of collaborative and basic models to construct preference data, it is possible to accurately capture users' preference patterns in scenarios such as medical consultation. This preference data enables the model to provide answers that are more in line with user needs in subsequent interactions, greatly improving the user experience and the accuracy and effectiveness of model services.

[0120] In S5, refer to Figure 4 The flowchart illustrating the method for obtaining a virtual QRA dataset, "Obtaining a virtual QRA dataset based on in-clinic data," includes the following steps:

[0121] S51: Construct QRA samples according to disease type;

[0122] S52: Segment the diagnostic data to obtain a set of diagnostic and treatment data for different disease types;

[0123] S53: According to the disease type, the QRA sample and diagnosis and treatment data set are filled into the preset template to form multiple prompt messages;

[0124] S54: Input each prompt message into the teacher model, and construct a virtual QRA dataset from the virtual QRA data output by the teacher model.

[0125] Here, Q stands for Question, which describes the user's symptoms and intentions; R stands for Reasoning, which involves detailed analysis and reasoning, including but not limited to: initial assessment of symptoms, a list of possible causes or diseases, the logic for excluding or confirming certain diseases, consideration of differential diagnoses, and the basis for drawing a final conclusion; and A stands for Action, which is the response content that provides a final conclusion based on the reasoning process, such as suggested departments, possible diagnostic directions, or preliminary recommendations.

[0126] Based on established disease classification standards, common problem patterns, analytical derivation process patterns, and conclusion patterns associated with various diseases are collected and integrated into QRA samples for each disease. Based on the diagnostic conclusion field in the diagnostic data, the data is categorized and split according to disease type. For example, diagnostic data for all users diagnosed with "diabetes" is grouped into one category, diagnostic data for users diagnosed with "hypertension" is grouped into another, and so on, forming diagnostic data sets for different disease types.

[0127] QRA samples and clinical data sets belonging to the same disease category are populated into a preset template to generate prompts for different disease categories. The following is an example of the prompts for respiratory medicine:

[0128] You are currently an experienced chief physician in respiratory medicine and a skilled medical mentor. Your task is to construct an SFT sample for training an AI triage model based on the structured medical record information provided below.

[0129] This sample needs to include three parts: a "question" simulating the user's tone, a detailed "analysis process", and a clear "final conclusion".

[0130] Please strictly adhere to the following requirements:

[0131] 1. Simulate user questions: Rewrite the "chief complaint" and "present illness" in the medical record into natural, conversational user questions.

[0132] 2. Analyzing the reasoning process:

[0133] It must be presented in a structured, point-by-point format.

[0134] The first step is "symptom breakdown and preliminary assessment," summarizing key positive and negative signs.

[0135] The second step is “differential diagnosis,” which involves listing at least 2-3 possible diseases and explaining the reasons for supporting or excluding them.

[0136] The third step is "information synthesis and decision-making," which, based on the above analysis, determines the most likely diagnostic direction and the reasons for choosing the appropriate department.

[0137] The entire process must be logically rigorous and reflect clinical reasoning.

[0138] 3. Final Conclusion (Answer): Based on the analysis process, provide a clear and easy-to-understand medical advice (such as recommending a department).

[0139] Please return your output strictly according to the JSON format below, without any extra text.

[0140] {

[0141] "question": "(The question you generated from the simulated user)",

[0142] "reasoning": "(The analysis process you generated)",

[0143] "answer": "(Your final conclusion)"

[0144] }

[0145] [Input medical data set]

[0146] json

[0147] {

[0148] "chief_complaint": "Coughing for 3 days",

[0149] "history_of_present_illness": "The user developed a dry cough 3 days ago without any obvious cause, without fever, chest tightness, or shortness of breath..."

[0150] "examinations": "Body temperature 36.8℃, clear breath sounds on lung auscultation, no dry or wet rales heard."

[0151] "diagnosis": "upper respiratory tract infection"

[0152] "doctor_advice": "It is recommended to take xx medication orally, drink plenty of water, and get plenty of rest. If symptoms worsen, please consult a doctor."

[0153] }

[0154] Each prompt is sequentially input into a trained teacher model, which then generates corresponding virtual QRA data based on the prompt content. These virtual QRA data are collected to form a virtual QRA dataset for subsequent model training or validation tasks. This method significantly improves the teacher model's logical reasoning ability and interpretability by training it to understand the reasoning path from symptoms to diagnosis or advice, rather than relying solely on pattern matching. Furthermore, by utilizing existing real-world medical record data, it ensures that the "analysis and reasoning process" is detailed and logically clear, explicitly demonstrating the teacher model's thought process from input to output, while also improving the accuracy of the final generated virtual conclusions.

[0155] In S6, the optimization objective is set as minimizing the token-level loss function to measure the difference between the base model's predictions and the true labels, thereby improving the model's accuracy. AdamW is chosen as the optimizer, which can adaptively adjust the learning rate of each parameter, helping the base model converge more stably and quickly. During training, a Warm-up + Cosine Decay parameter tuning strategy is adopted. The learning rate is gradually increased in the early stages of training to stabilize learning, and gradually decreased in the later stages, allowing the base model to finely adjust its parameters. The virtual QRA dataset is used as training data, and a high-quality sft_qra_data.jsonl file is generated and loaded. This file contains cleaned and processed training data for subsequent supervised fine-tuning. The base model and its corresponding tokenizer are loaded from the pre-trained model using relevant tools. A dedicated dataset loading tool is used to load the sft_qra_data.jsonl file as a JSON dataset for use in training the base model. A series of training parameters are defined, including the output directory, training batch size for each device, gradient accumulation steps, learning rate, number of training epochs, number of logging steps, and number of save steps. If the hardware supports it, mixed precision training can also be enabled to improve training efficiency. A formatting function is defined to convert JSON objects (referring to the JSON dataset) into a dialogue format string that the base model can understand. This function processes each training data point during SFT (Supervised Fine-Tuning) training to ensure that the training data meets the input requirements of the base model. Finally, the SFTTrainer is initialized by passing in the base model, training parameters, training dataset, tokenizer, and formatting function, while setting the maximum sequence length, and then the training process is started. During training, the base model will use the AdamW optimizer and the Warm-up + Cosine Decay parameter tuning strategy to progressively optimize its parameters by minimizing the token-level loss function, ultimately obtaining a reference model. Through the above steps, supervised fine-tuning of the base model can be effectively performed, improving its performance and accuracy in specific tasks.

[0156] In S7, refer to Figure 5 The flowchart illustrating the method for obtaining security constraint rules shows that "obtaining security constraint rules" includes the following steps:

[0157] S71: Obtain initial rules input by medical experts from different fields, and determine the triggers, constraint types, and matching methods that match the initial rules;

[0158] S72: Based on triggers, constraint types, and matching methods, convert initial rules into initial security constraint rules;

[0159] S73: Review and verify the initial security constraint rules to obtain the security constraint rules.

[0160] In S71, medical experts from different fields, such as cardiology, neurology, and pediatrics, are organized to hold seminars or collect initial rules based on their professional knowledge and clinical experience through online questionnaires. These rules usually revolve around common high-risk medical scenarios, misdiagnosis risks, and safe medication use, such as recognizing symptoms of acute myocardial infarction, stroke, and febrile seizures in children, as well as avoiding giving incorrect treatment advice.

[0161] If the initial rule involves complex semantic understanding, such as determining whether user input belongs to a specific high-risk disease scenario, for example, the example rule "SR-001" below which detects a combination of symptoms highly related to acute myocardial infarction, a semantic classifier trigger should be selected. This type of trigger can handle semantic information, allowing for more accurate risk identification through a trained model. The example rule "SR-001" is as follows:

[0162] rule_id: "SR-001"

[0163] Name: "Suspected symptoms of myocardial infarction necessitate referral to the emergency room"

[0164] Description: "Detect combinations of symptoms highly correlated with acute myocardial infarction in user input and ensure that the model response includes mandatory emergency medical instructions."

[0165] trigger:

[0166] type: "semantic_classifier" # Trigger type

[0167] model_path: "models / classifiers / cardiac_emergency_detector.pt" # Trigger model path

[0168] threshold: 0.95 # Trigger activation threshold

[0169] constraint:

[0170] type: "MUST_INCLUDE" # Constraint type

[0171] elements: # Constraint elements

[0172] - Call emergency services immediately

[0173] - "Go to the emergency room immediately"

[0174] - "Do not go there on your own"

[0175] match_method: "semantic" # Element matching method

[0176] For cases where the initial rule trigger condition is clearly a simple keyword, such as the example rule "SR-002" which detects high-risk symptoms like "coma" and "convulsions," a keyword_list trigger is suitable. It is simple, direct, and allows for fast matching. The example rule "SR-002" is as follows:

[0177] rule_id: "SR-002"

[0178] Name: "Home observation for high-risk symptoms is prohibited"

[0179] Description: "When any symptoms defined as 'high-risk' are detected, the model is prohibited from recommending non-active interventions such as 'staying home' or 'drinking more water.'"

[0180] trigger:

[0181] type: "keyword_list"

[0182] Keywords: ["coma", "convulsions", "blindness", "respiratory arrest"]

[0183] constraint:

[0184] type: "MUST_NOT_CONTAIN"

[0185] elements:

[0186] - "Home Observation"

[0187] - "Get more rest"

[0188] - "Handle it yourself"

[0189] - "Let's see in a few days"

[0190] match_method: "fuzzy" # Fuzzy matching

[0191] If the initial rule's triggering condition is unsuitable for simple keyword matching and doesn't require complex semantic classification, but rather needs to identify semantically similar expressions, an embedding similarity trigger can be used. This trigger, as a technical solution between keyword matching and semantic classifiers, has unique advantages. For example, using a pre-trained word vector model (such as Word2Vec, BERT, etc.), the initial rule triggering condition (e.g., "chest squeezing pain") can be transformed into a vector embedding. After receiving user input, the same model is used to convert it into a vector, and the cosine similarity between this vector and the triggering condition vector is calculated. A similarity threshold is set; if the calculated result exceeds the threshold, the rule is triggered. This trigger can balance semantic understanding and matching efficiency.

[0192] When the purpose of the initial rule is to ensure that the content generated by the model covers specific key information, the MUST_INCLUDE constraint type should be selected. In this case, the key information required by the initial rule must be clearly defined and listed as an elements list. The model's completion (generated content) must contain at least one element from this list. For example, in the example rule "SR-001", the initial rule "Suspected myocardial infarction symptoms must be directed to the emergency room" aims to have the model respond with instructions to seek emergency medical care. Therefore, the constraint type is determined to be MUST_INCLUDE, and "Call emergency services immediately", "Go to the emergency room immediately", and "Do not go on your own" are listed as elements. Thus, the content generated by the model must contain at least one element from the elements list.

[0193] If the initial rule prohibits the model from generating certain specific content, the MUST_NOT_CONTAIN constraint type should be selected. The content prohibited by the initial rule should be listed as an elements list, ensuring that the model's completion does not contain any elements from this list. For example, in the example rule "SR-002," the initial rule "prohibits recommendations for home observation of high-risk symptoms." Its purpose is to avoid the model recommending suggestions without active intervention. Therefore, the constraint type is determined to be MUST_NOT_CONTAIN, and "home observation," "rest more," "self-treatment," and "wait and see" are listed as elements, ensuring that the model's generated content does not contain any elements from this list.

[0194] If the initial rule has high semantic requirements, meaning it needs to accurately match semantics (e.g., rule "SR-001" requires the model's response to contain semantically related content to emergency medical instructions), then semantic similarity matching should be chosen to ensure the model's output is semantically correct. If the initial rule allows some flexibility and doesn't require exact matching (e.g., rule "SR-002" prohibits recommendations without active intervention, such as "home observation"), then fuzzy matching can handle differences in wording. If the initial rule requires high matching accuracy, then exact matching should be chosen.

[0195] In S72, assign a unique `rule_id` to the initial rule, and name and describe it, clearly stating its purpose and function, such as "SR-001" or "Suspected myocardial infarction symptoms must be directed to the emergency room" in the example rule. Configure the trigger parameters. For the `semantic_classifier` trigger, specify the model path and activation threshold, as in the configuration of the example rule "SR-001"; for the `keyword_list` trigger, list the keywords, as in the configuration of the example rule "SR-002". Based on the determined constraint type, fill in the constraint elements and matching method. For the `MUST_INCLUDE` constraint type, list the elements that must be included; for the `MUST_NOT_CONTAIN` constraint type, list the elements that cannot be included, and specify whether the matching method is "semantic", "fuzzy", or "exact". Based on the above, the rule number, name, and description assigned to the initial rule, along with the configured trigger parameters (model path, activation threshold, or keyword list, etc., set according to different trigger types), and the constraint elements and matching methods filled in according to the constraint type, are integrated to form a security constraint rule. This rule can effectively standardize the output content of the model and ensure that it meets specific security and business requirements.

[0196] In S73, medical experts and technicians review the initial safety constraint rules. Medical experts examine the logical rationality of the rules from a professional perspective, ensuring that the triggering and constraint conditions align with medical common sense and clinical practice. Technicians check the correctness of the rule configuration and the reasonableness of the trigger and constraint parameter settings. The rules are validated using a test dataset containing various possible user inputs and model-generated content to simulate real-world scenarios. For each initial safety constraint rule, the system checks whether the triggers fire correctly and whether the constraints effectively constrain the model output. For example, for rule "SR-001," the system verifies whether the trigger activates when the input contains symptoms related to acute myocardial infarction, and whether the model response includes the prescribed emergency medical instructions. For rule "SR-002," the system verifies whether the model output does not contain prohibited content when the input contains high-risk symptoms. Based on the review and validation results, the initial safety constraint rules are adjusted and optimized. If triggers are found to misjudge or miss, the trigger parameters, such as thresholds or keyword lists, are adjusted. If constraints fail to effectively constrain the model output, the constraint elements or matching methods are adjusted. After multiple adjustments and verifications until the rules achieve satisfactory results, the security constraint rules are finally obtained and stored in the rule list.

[0197] After obtaining the security constraint rules, DPO training is performed based on the security constraint rules. This disclosure provides two possible implementation schemes. In one possible implementation scheme, refer to... Figure 6 The flowchart illustrating the patient guidance model training method shows the following steps: "Training the patient guidance model based on preference data, safety constraint rules, and a reference model to obtain the desired model":

[0198] S74: Construct an initial empty preference dataset;

[0199] S75: Determine the current preference data through traversal operations, and extract high-quality and low-quality response content from the current preference data;

[0200] S76: Determine whether the content of the high-quality response complies with the security constraint rules; if yes, proceed to S77; if no, proceed to S78.

[0201] S77: Store the current preference data in the preference dataset;

[0202] S78: Determine whether the content of the non-high-quality response complies with the security constraint rules; if yes, proceed to S79; if no, proceed to S710.

[0203] S79: Reverse the preferences of high-quality and low-quality responses to form new preference data, and store the new preference data in the preference dataset;

[0204] S710: Delete the current preference data;

[0205] S711: Determine if all preference data has been traversed; if yes, execute S712; if no, return to S75 to determine the new current preference data.

[0206] S712: Based on the preference dataset and reference model, train the triage model to obtain the desired model.

[0207] In the above process, all preference data is traversed, and the preference data being traversed is the current preference data. High-quality and low-quality responses are extracted. By matching high-quality and low-quality responses with safety constraint rules, the processing method for the current preference data can be determined. Specifically, if high-quality responses conform to the safety constraint rules, the current preference data has training value and is retained. If high-quality responses do not conform to the safety constraint rules, but low-quality responses do, it indicates an error in defining the superiority / inferiority attributes of high-quality and low-quality responses. The original high-quality responses are corrected to low-quality responses, and the original low-quality responses are corrected to high-quality responses. This preference reversal creates new preference data, which is then saved. If neither high-quality nor low-quality responses conform to the safety constraint rules, the current preference data has no training value and is deleted.

[0208] The method for matching both high-quality and low-quality responses with security constraint rules is as follows: A rule matching function `check_safety` is constructed to check whether the response content (including both high-quality and low-quality responses) conforms to the security constraint rules. An empty list `violated_rules` is created to store the rule numbers of the security constraint rules that the response content does not conform to. Then, each security constraint rule in the rule list is traversed. Based on the triggers in the security constraint rules, it is determined whether the response content meets the trigger conditions. If the trigger conditions are met, it is further determined whether the response content meets the constraint conditions. If the constraint conditions are met, the response content is determined to conform to the security constraint rules, and the traversal continues to the next security constraint rule. If the constraint conditions are not met, the response content is determined to not conform to the security constraint rules, the rule number of the security constraint rule is stored in `violated_rules`, and the traversal continues. If the trigger conditions are not met, the response content is determined not to have triggered the security constraint rule, indicating that the response content is safe and conforms to the security constraint rules, and the traversal continues to the next security constraint rule. After the traversal is completed, the `violated_rules` list is returned. Based on the `violated_rules` list, the security constraint rules that the response content does not conform to can be clearly identified.

[0209] The initial model is the triage model, and a reference model is loaded to provide a benchmark and comparison for training. A loss function, defined according to the DPO method, is used to measure the consistency between the triage model's output and the preference data. During training, the triage model generates a third response based on the question data in the preference data. This third response is compared with high-quality and low-quality responses in the preference data, and the DPO loss function is calculated. The optimizer updates the parameters of the triage model based on the DPO loss function. This training and update process is repeated until the triage model converges or reaches a preset number of rounds. The performance of the trained triage model is evaluated using a validation set, and the desired model is obtained upon completion. The formula for calculating the DPO loss function is as follows:

[0210]

[0211] In the formula, Represents the DPO loss function; This represents the patient guidance model; These represent the parameters of the patient guidance model; Represents the reference model; Indicates the parameters of the reference model; Expressing expectations; This represents the distribution of the preference dataset; This represents all preference data samples obtained from distribution D. ; This represents the problem data; This indicates a high-quality response. This indicates a non-high-quality response. This represents the sigmoid function. , Indicates intermediate quantity. , represents the natural constant, which is the base of the natural logarithm; This represents a hyperparameter, specifically a hyperparameter that controls the amplification factor of the logit scaling term, used to adjust the slope of the sigmoid function; This indicates that the triage model, given input Generate a response under the following conditions The probability of; This indicates that the reference model is in response to a given input. Generate a response under the following conditions The probability of; This indicates that the triage model, given input Generate a response under the following conditions The probability of; This indicates that the reference model is in response to a given input. Generate a response under the following conditions The probability of.

[0212] This method corrects the preference data through safety constraint rules, and then uses the corrected preference data to train the triage model using DPO. This not only improves the training effect, but also makes the static processing method simple to implement and does not intrude on the core code of the DPO training framework (Transformers, etc.).

[0213] In another possible implementation, refer to Figure 7 Another flowchart illustrating the patient guidance model training method shows that "training the patient guidance model based on preference data, safety constraint rules, and a reference model to obtain the desired model" includes the following steps:

[0214] S713: Construct the SC-DPO loss function based on the question data, high-quality and low-quality response content contained in the preference data, and the safety constraint rules contained in the rule list;

[0215] S714: Train the triage model based on the reference model and the SC-DPO loss function to obtain the desired model.

[0216] In the steps described above, preference data is loaded, and the triage model (i.e., the policy model) and the reference model perform forward propagation on these responses to obtain their logits (raw scores of the model output). Using these logits, the DPO loss function is calculated. This function aims to optimize the triage model, enabling it to more effectively distinguish between selected and rejected responses. When calculating the DPO loss function, a penalty term for the safety constraint rules is introduced. This modification combines the standard DPO loss with the safety penalty, forming the SC-DPO loss function, resulting in a more accurate total loss. Based on this total loss, backpropagation is performed to calculate the gradient, and then the optimizer updates the model's weights. This process ensures that the triage model considers not only response preferences but also strictly adheres to safety constraints during optimization.

[0217] The formula for calculating the SC-DPO loss function is as follows:

[0218]

[0219] In the formula, Represents the SC-DPO loss function; This represents the safety penalty weight hyperparameter, used to control the severity of safety penalties; Represents a list of rules; This represents the safety penalty function, which is applied to the triage model given the input. The response generated under the condition If the security constraint rules in the rule list are not met, then If the triage model is given input The response generated under the condition If the security constraints in the rule list are met, then .

[0220] This method significantly improves the traditional DPO loss function by introducing a penalty term for safety constraint rules, constructing a more comprehensive SC-DPO loss function. This improvement not only aims to help the triage model understand the importance of safety through learning data, but also equips the triage model with a clear and enforced "safety brake" through a manually adjustable external constraint system. Guided by the SC-DPO loss function, the optimization direction of the triage model shifts from simply "becoming more like human preferences" to "pursuing optimal human preferences while ensuring safety." This shift ensures that the triage model strictly adheres to safety red lines when generating answers, while encouraging it to pursue better human preferences while following key safety rules. In this way, the safety and reliability of the triage model in practical applications are significantly improved, achieving direct and precise control over the bottom line of AI behavior, thereby optimizing the user experience while ensuring safety.

[0221] Existing technologies typically involve only DPO training of a reference model and a triage model. This solution introduces a collaborative model, which replaces the triage model and works with the reference model before supervised fine-tuning to construct preference data. By utilizing this preference data during reinforcement learning, the triage model can learn a wider range of human preference patterns during training. This application overcomes the limitations of existing technologies. In reinforcement learning, preference data serves as crucial guidance information. When the triage model makes a decision, it compares it with the preference data. If the decision aligns with the human preferences reflected in the preference data, a positive reward is given; otherwise, negative feedback is provided. Through this reward and punishment mechanism, the triage model can gradually adjust its strategy, continuously learning and adapting to a wider range of human preference patterns. Furthermore, by leveraging the preference data constructed from the reference model and collaborative model before supervised fine-tuning, the triage model is exposed to more diverse preference information during training, thereby improving its generalization ability and adaptability. It can better address the triage needs of various users, providing more accurate and user-preference-aligned triage suggestions in practical applications, thus improving the quality and efficiency of triage.

[0222] Reference Figure 8 This disclosure provides a training system for a triage model, comprising:

[0223] The data acquisition module 101 is used to acquire pre-diagnosis dialogue data and in-diagnosis data, and to extract problem data from the pre-diagnosis dialogue data.

[0224] The standard answer acquisition module 102 is used to obtain reference standard answers to question data based on diagnostic data.

[0225] The response content acquisition module 103 is used to input the question data into the collaborative model and the basic model respectively, and to acquire the first response content output by the collaborative model and the second response content output by the basic model.

[0226] The preference data construction module 104 is used to construct preference data based on question data, the content of the first response, the content of the second response, and the reference standard answer;

[0227] The virtual data acquisition module 105 is used to acquire a virtual QRA dataset based on in-diagnosis data;

[0228] The model supervised fine-tuning module 106 is used to supervise and fine-tune the base model based on the virtual QRA dataset to obtain a reference model;

[0229] The triage model training module 107 is used to obtain safety constraint rules. Based on preference data, safety constraint rules and reference model, the triage model is trained to obtain the desired model.

[0230] The various variations and specific examples of the training method for the triage model provided above are also applicable to the training system for the triage model provided in this disclosure. Through the foregoing detailed description of the training method for the triage model, those skilled in the art can clearly understand the implementation method of the training system for the triage model. For the sake of brevity, they will not be described in detail here.

[0231] A computer device according to embodiments of the present disclosure includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0232] The processor may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of this disclosure, the processor is used to run computer-readable instructions stored in the memory, causing the computer device to perform all or part of the steps of the training method for the triage model of the foregoing embodiments of this disclosure.

[0233] Those skilled in the art will understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this disclosure.

[0234] like Figure 9 This is a schematic diagram of a computer device provided for an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the computer device in the embodiments of the present disclosure. Figure 9 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0235] like Figure 9 As shown, a computer device may include a processor (such as a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from storage devices into random access memory (RAM). The RAM also stores various programs and data required for the operation of the computer device. The processor, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0236] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow the computer device to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 9 A computer apparatus with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or included alternatively.

[0237] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the training method of the triage model according to embodiments of this disclosure are performed.

[0238] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0239] A computer-readable storage medium according to embodiments of the present disclosure stores non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the training method for the triage model of the foregoing embodiments of the present disclosure are performed.

[0240] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0241] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0242] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0243] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0244] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0245] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0246] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0247] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0248] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A training method for a patient guidance model, characterized in that, The method comprises the following steps: obtaining pre-diagnosis dialogue data and in-diagnosis data, and extracting question data from the pre-diagnosis dialogue data; obtaining a reference standard answer of the question data based on the in-diagnosis data; inputting the question data into a collaborative model and a basic model respectively, obtaining first reply content output by the collaborative model and second reply content output by the basic model; constructing preference data based on the question data, the first reply content, the second reply content and the reference standard answer; obtaining a virtual QRA data set based on the in-diagnosis data; supervising and fine-tuning the basic model based on the virtual QRA data set to obtain a reference model; obtaining a safety constraint rule, training a triage model based on the preference data, the safety constraint rule and the reference model to obtain an expected model; wherein the obtaining of the safety constraint rule, the training of the triage model based on the preference data, the safety constraint rule and the reference model, and the obtaining of the expected model comprise: constructing an SC-DPO loss function based on the question data contained in the preference data, high-quality reply content and non-high-quality reply content, and safety constraint rules contained in a rule list; training the triage model based on the reference model and the SC-DPO loss function to obtain the expected model; wherein an expression of the SC-DPO loss function is: wherein, denotes the SC-DPO loss function; denotes the distribution of preference data; denotes all preference data samples sampled from the distribution D ; denotes the question data; denotes high-quality answer content; denotes non-high-quality answer content; denotes the sigmoid function; denotes the hyperparameters; denotes the triage model; denotes the parameters of the triage model; denotes the reference model; denotes the parameters of the reference model; denotes the expectation; denotes the probability that the triage model generates an answer given the input ; denotes the probability that the reference model generates an answer given the input ; denotes the probability that the triage model generates an answer given the input ; denotes the probability that the reference model generates an answer given the input ; denotes the safety penalty weight hyperparameter; denotes the list of rules; denotes the safety penalty function, which is if the answer generated by the triage model given the input does not satisfy the safety constraint rules in the list of rules, then , and if the answer generated by the triage model given the input satisfies the safety constraint rules in the list of rules, then . 2.The method of claim 1, wherein, the obtaining of the reference standard answer of the question data based on the in-diagnosis data comprises: preprocessing the in-diagnosis data to obtain key data; aligning the question data, the key data and the in-diagnosis data based on an identifier of a user and a treatment timeline; inputting the aligned question data, key data and in-diagnosis data into a teacher large language model to generate a reference standard answer for each question data. 3.The method of claim 1, wherein, the constructing of the preference data based on the question data, the first reply content, the second reply content and the reference standard answer comprises: comparing the first reply content and the second reply content based on the question data and the corresponding reference standard answer, and determining high-quality reply content and non-high-quality reply content based on a comparison result; combining the question data, the high-quality reply content and the non-high-quality reply content into preference data. 4.The method of claim 1, wherein, the obtaining of the virtual QRA data set based on the in-diagnosis data comprises: constructing QRA samples according to disease categories; splitting the in-diagnosis data to obtain diagnosis and treatment data sets of different disease categories; filling the QRA samples and the diagnosis and treatment data sets into a preset template according to disease categories to form a plurality of prompt information; inputting each prompt information into a teacher model, and constructing virtual QRA data output by the teacher model into a virtual QRA data set. 5.The method of claim 3, wherein, the obtaining of the safety constraint rule, the training of the triage model based on the preference data, the safety constraint rule and the reference model, and the obtaining of the expected model comprise: constructing an initially empty preference data set; determine current preference data through the traversing operation, extract high-quality reply content and non-high-quality reply content from the current preference data; determine whether the high-quality reply content meets the safety constraint rule; if the high-quality reply content meets the safety constraint rule, store the current preference data into the preference data set; if the high-quality reply content does not meet the safety constraint rule, determine whether the non-high-quality reply content meets the safety constraint rule; if the non-high-quality reply content meets the safety constraint rule, perform preference inversion on the high-quality reply content and the non-high-quality reply content to form new preference data, and store the new preference data into the preference data set; if the non-high-quality reply content does not meet the safety constraint rule, delete the current preference data; after the traversal of all the preference data is completed, train the triage model based on the preference data set and the reference model to obtain an expected model.

6. A computer apparatus, comprising: The computer device comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the training method of the triage model according to any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to perform the training method of the triage model according to any one of claims 1-5.

8. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by a processor, implement the steps of the method according to any one of claims 1-5.

Images