Method, device and medium for providing response information to medical inquiries
By extracting medical inquiry information features and searching text embedding with professional data sets, the input prompts for language models is solved, and the problem of insufficient response accuracy in traditional AI tools when processing large-scale clinical data is realized is realized. Accurate response.
Patent Information
- Application Number
- CN202310948615.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-28
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-07-28
AI Technical Summary
Traditional AI-based medical inquiry response tools are difficult to process large-scale, heterogeneous clinical data, resulting in poor accuracy and targeted response suggestions, and it is difficult to achieve accurate responses to medical inquiry questions.
By extracting the user's medical inquiry information characteristics, generate inquiry problem feature information in natural language format, and conduct text embedding search with data sets such as clinical information, drug information, sample case information, diagnosis and treatment information, and gene information to construct prompt input The trained language model is used to generate response information.
It realizes accurate responses to medical inquiry questions, improves the accuracy of response information through semantic matching, understands user inquiry intentions, and provides more accurate responses.
Smart Images

Figure CN117171309B_ABST
Abstract
Description
Technical Field
[0001] The present invention generally relates to information interaction in the medical field, and in particular, to a method, a computing device, and a computer storage medium for providing response information to a medical query. Background Art
[0002] Traditional methods of providing response information for medical inquiries, such as response tools based on artificial intelligence (AI), mainly use machine learning, natural language processing and image recognition technology to analyze clinical data and provide response information (such as but not limited to response information about diagnosis and treatment). However, the above-mentioned AI-based response tools have difficulties in processing large-scale, heterogeneous and actual clinical data; and because they cannot effectively process and analyze complex medical logic and guidelines, the accuracy and pertinence of the response suggestions provided are poor.
[0003] In summary, the disadvantage of the traditional method of providing response information for medical inquiries is that it is difficult to achieve accurate responses to medical inquiry questions. Summary of the invention
[0004] The present invention provides a method, a computing device and a computer storage medium for providing response information for medical inquiries, which can effectively achieve accurate responses to medical inquiry questions.
[0005] According to a first aspect of the present invention, a method for providing response information for medical inquiries is provided. The method comprises: extracting features for the acquired medical inquiry information of the user, so as to generate inquiry question feature information having a predetermined natural language format based at least on the extracted features; performing text embedding search on the inquiry question feature information and a predetermined data set, so as to obtain a search result field, wherein the predetermined data set is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information, and predetermined knowledge base information; constructing a prompt based at least on the inquiry question feature information and the obtained search result field, so as to input a trained language model, so as to generate response information for the medical inquiry information based at least on the output of the language model.
[0006] According to a second aspect of the present invention, a computing device is also provided, comprising: a memory configured to store one or more computer programs; and a processor coupled to the memory and configured to execute the one or more programs so that the device performs the method of the first aspect of the present invention.
[0007] According to a third aspect of the present invention, a non-transitory computer-readable storage medium is further provided, wherein machine-executable instructions are stored on the non-transitory computer-readable storage medium, and when the machine-executable instructions are executed, the machine executes the method of the first aspect of the present invention.
[0008] In some embodiments, the language model is an LLM model, characterized in that generating response information for medical query information based at least on the output of the language model includes: constructing a prompt based at least on the query question feature information and the obtained search result field to input the trained LLM model for polishing so as to generate a first response letter for the medical query information; inputting the query question feature information into the trained fine-tuning model to generate a second response information for the medical query information; and generating response information for presentation to the user based on the first response information and / or the second response information.
[0009] In some embodiments, the trained fine-tuning model is generated via the following: generating a training data set for the fine-tuning model based on locally stored clinical information, tumor somatic gene data, structured diagnosis and treatment information, and a gene-related knowledge base; converting the fine-tuning model training data set into a JSONL document, the JSONL document comprising multiple lines of prompt and response item pairs, each line of prompt and response item pair corresponding to a training example; and based on the JSONL document, fine-tuning a predetermined base model to generate a fine-tuning model.
[0010] In some embodiments, the language model is an LLM model, and the trained language model is generated via the following items: The trained LLM model is generated via the following items: obtaining clinical information, diagnosis and treatment information, tumor somatic gene information and predetermined gene knowledge base information about multiple target objects to generate input data; performing natural language conversion on the generated input data to generate clinical information indicator information and diagnosis and treatment sample information in a predetermined natural language format; determining the generated clinical information indicator information as a training prompt, and determining the diagnosis and treatment sample information as a training response item, so as to generate preprocessed training data and verification data; and training the LLM model based on the preprocessed training data and verification data to obtain a trained LLM model.
[0011] In some embodiments, generating preprocessed training data and verification data includes: dividing training prompts and training response entries into training data and verification data; converting the training prompts and training response entries in the training data and verification data into JSON format to generate prompts and response entries in JSON format; and marking evidence classification levels for the prompts and response entries in JSON format to generate preprocessed training data and verification data.
[0012] In some embodiments, the language model is an LLM model, and training the language model includes: pre-training the LLM model with a first predetermined loss function until convergence to obtain a pre-trained LLM model; and training the pre-trained LLM model with at least one predetermined loss function until convergence to obtain a trained LLM model.
[0013] In some embodiments, the LLM model is constructed based on a deep learning model of a Transformer network architecture, and is characterized in that training the pre-trained LLM model with at least one predetermined loss function until convergence includes: obtaining training data and verification data about a predetermined query task for model adjustment; based on the training data and verification data about the predetermined query task, confirming the type of the current query task; in response to confirming that the type of the current query task is a classification task, training the pre-trained LLM model with a cross entropy loss function until convergence; and in response to confirming that the type of the current query task is a regression task, training the pre-trained LLM model with a mean square error loss function until convergence.
[0014] In some embodiments, generating inquiry question feature information in a predetermined natural language format based at least on the extracted features includes: based on the extracted features, confirming whether the following two predetermined conditions are both met, the two predetermined conditions including: confirming based on a predetermined feature subset that the necessary features for constructing the response information are not yet sufficient; and confirming that the user belongs to a group of people with predetermined attributes; in response to confirming that both of the two predetermined conditions are met, outputting one or more rounds of interaction information to the user via a language interaction model until the necessary features for constructing the response information are sufficient or the interaction information provided by the user indicates that it is impossible to continue to input information corresponding to the necessary features.
[0015] In some embodiments, generating inquiry question feature information in a predetermined natural language format based at least on the extracted features includes: if it is confirmed that both of the two predetermined conditions are not met, generating inquiry question feature information in a predetermined natural language format based on the features extracted for the medical inquiry information; and if it is confirmed that the necessary features for constructing the response information are sufficient through one or more rounds of interaction information or the interaction information provided by the user indicates that the information corresponding to the necessary features cannot be continued to be input, generating inquiry question feature information in a predetermined natural language format based on the features extracted from the medical inquiry information and the interaction information provided by the user.
[0016] In some embodiments, performing a text embedding search on the query question feature information and a predetermined data set to obtain a search result field includes: generating a search input field based on the query question feature information; performing a text embedding search on the predetermined data set based on the search input field and a predetermined embedding model to obtain a floating point number vector, wherein the floating point number vector is used to indicate the distance between the search input field and each field included in the predetermined data set; and obtaining one or more search result fields that match the search input field based on a comparison of the floating point number vector with a predetermined distance threshold.
[0017] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the invention, nor is it intended to limit the scope of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A schematic diagram of a system for a method for providing response information to a medical inquiry according to an embodiment of the present invention is shown.
[0019] Figure 2 A flow chart of a method for generating medical response information according to an embodiment of the present invention is shown.
[0020] Figure 3 A flowchart of a method for generating a trained LLM model according to an embodiment of the present invention is shown.
[0021] Figure 4 A flow chart of a method for performing text embedding search to obtain search result fields according to an embodiment of the present invention is shown.
[0022] Figure 5 The block diagram of an electronic device suitable for implementing the embodiments of the present invention is schematically shown.
[0023] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION
[0024] The preferred embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.
[0025] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.
[0026] As described above, the traditional method of providing response information for medical inquiries has the disadvantage that it is difficult to achieve accurate responses to medical inquiry questions.
[0027] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, an exemplary embodiment of the present invention proposes a solution for providing response information for medical inquiries. In this solution, by using the query feature information and the professional data set to perform text embedding search to obtain the search result field, the present invention can make the search result field semantically more accurately match the user's query. In addition, by using the search result field obtained after the embedded search together with the query feature information to construct a prompt for inputting the LLM model to generate response information via the output of the language model, the present invention can more accurately understand the user's query intention, so that the response information more accurately matches the user's query intention. Therefore, the present invention can effectively achieve accurate responses to medical inquiries.
[0028] Figure 1 FIG. 1 is a schematic diagram showing a system 100 for providing response information to a medical inquiry according to an embodiment of the present invention. Figure 1 As shown, the system 100 includes: a computing device 110, a network 140, and a user terminal 130. In some embodiments, the computing device 110 and the user terminal 130 perform data exchange via the network 140.
[0029] The user terminal 130 is used to provide the user's medical query information to the computing device 110. The user terminal 130 may also provide the computing device 110 with one or more rounds of interaction information.
[0030] Regarding the computing device 110, it is used, for example, to provide response information for medical inquiry information from the user terminal 130. Specifically, the computing device 110 extracts features for the medical inquiry information of the user obtained, so as to generate inquiry question feature information having a predetermined natural language format based at least on the extracted features. The computing device 110 can also be used to perform a text embedding search on the inquiry question feature information and a predetermined data set to obtain a search result field, wherein the predetermined data set is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information, and predetermined knowledge base information; and to construct a prompt based at least on the inquiry question feature information and the obtained search result field to input a trained language model, thereby generating response information for the medical inquiry information based at least on the output of the language model. In some embodiments, the computing device 110 can also locally store: clinical information, drug information, sample case information, diagnosis and treatment information, gene information, and predetermined knowledge base information (for example, including gene knowledge base information). It should be understood that the above information can also be obtained from other servers ( Figure 1 not shown).
[0031] In some embodiments, the computing device 110 may have one or more processing units, including dedicated processing units such as GPUs, FPGAs, and ASICs, and general-purpose processing units such as CPUs. In addition, one or more virtual machines may also be running on each computing device. The computing device 110 includes, for example, a query feature information generation unit 112, a text embedding search unit 114, and a language model input and response information generation unit 116. The query feature information generation unit 112, the text embedding search unit 114, the language model input and response information generation unit 116 may be configured on one or more computing devices 110.
[0032] The inquiry question feature information generating unit 112 is configured to extract features from the acquired medical inquiry information of the user, so as to generate inquiry question feature information having a predetermined natural language format based at least on the extracted features.
[0033] Regarding the text embedding search unit 114, it is used to perform text embedding search on the query question feature information and the predetermined data set to obtain the search result field, and the predetermined data set is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information and predetermined knowledge base information.
[0034] Regarding the language model input and response information generation unit 116, it is used to construct prompts based on at least the query question feature information and the obtained search result fields to input the trained language model to generate response information for the medical query information based on at least the output of the language model.
[0035] The following will be combined Figure 2 A method for providing response information to a medical query according to an embodiment of the present invention is described. Figure 2 FIG. 2 is a flowchart of a method 200 for generating medical response information according to an embodiment of the present invention. It should be understood that the method 200 may be implemented in Figure 5 The electronic device 500 described above is executed. Figure 1 The method 200 is executed at the described computing device 110. It should be understood that the method 200 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0036] At step 202 , the computing device 110 extracts features from the acquired medical inquiry information of the user, so as to generate inquiry question feature information having a predetermined natural language format based at least on the extracted features.
[0037] Regarding medical inquiry information, it can be an inquiry question about medical field knowledge from a user terminal, or an inquiry question about clinical information and diagnosis and treatment related to a specific target object (e.g., a patient). Medical inquiry information can be text data converted based on user voice input, or it can be text data for direct input.
[0038] As for the users, they can be patients, family members of patients, or other non-medical professionals who do not have medical professional background knowledge, or they can be doctors, medical experts, or other professionals in the medical field. It should be understood that there are obvious differences in the professional level of medical terminology application and the degree of understanding of medical information among the above-mentioned people with different attributes.
[0039] For example, the computing device 110 confirms whether the following two predetermined conditions are satisfied based on the extracted features, and the two predetermined conditions include: confirming that the necessary features for constructing the response information are not sufficient based on the predetermined feature subset; and confirming that the user belongs to a predetermined attribute group; in response to confirming that the two predetermined conditions are satisfied, output one or more rounds of interaction information to the user through the language interaction model until the necessary features for constructing the response information are sufficient or the interaction information provided by the user indicates that the information corresponding to the necessary features cannot be input. Further, if the computing device 110 confirms that the two predetermined conditions mentioned above are not satisfied, directly based on the features extracted for the medical inquiry information, generate inquiry question feature information with a predetermined natural language format; and if it is confirmed that the necessary features for constructing the response information are sufficient through one or more rounds of interaction information or the interaction information provided by the user indicates that the information corresponding to the necessary features cannot be input, based on the features extracted for the medical inquiry information and the interaction information provided by the user, generate inquiry question feature information with a predetermined natural language format.
[0040] Regarding the predetermined attribute population, for example, it is a medical professional population, such as doctors, physician assistants, medical experts, and the like.
[0041] Specifically, if the computing device 110 determines that the user who raises the current medical inquiry information belongs to a predetermined attribute group, such as a medical professional attribute group, such as a doctor, and the computing device 110 confirms that the necessary features for indicating the construction of the response information are insufficient based on the predetermined feature subset and the extracted features, the computing device 110 will output interaction information to the user via a language interaction model (for example, but not limited to, an LLM model) until the necessary features for constructing the response information are sufficient. If the computing device 110 determines that the user who raises the current medical inquiry information does not belong to the predetermined attribute group, such as an ordinary patient or other non-medical professional, even if the computing device 110 confirms that the necessary features for indicating the construction of the response information are insufficient based on the predetermined feature subset and the extracted features, the computing device 110 will no longer output interaction information to the user via the language interaction model, but will directly generate inquiry question feature information in a predetermined natural language format based on the features extracted from the medical inquiry information input by the user. By adopting the above means, the present invention can selectively configure whether to further obtain the necessary information for constructing the response information through multiple rounds of interactive information according to the professionalism of the current user who raises the medical inquiry question or the application scenario, thereby taking into account both good user experience and sufficient and efficient acquisition of medical inquiry information.
[0042] For another example, when the interaction information provided by the user indicates that there are no features that can be explained, the computing device 110 generates inquiry question feature information in a predetermined natural language format based on the features obtained through the medical inquiry information and the interaction information provided by the user and their paired feature values.
[0043] Regarding the inquiry question feature information in a predetermined natural language format, it includes, for example, multiple feature identifiers (e.g., features key) and feature values corresponding to the feature identifiers (e.g., features value). For example, the inquiry question feature information includes some patient parameter features: cancer type = NSCLC, stage = stage III, unresectable, with RET fusion, TPS = 1%. Regarding the language interaction model, it is, for example but not limited to, a trained first LLM model. By configuring the answer logic of the LLM model output interaction information to complement the default necessary features based on a predetermined feature subset, the necessary features can be complemented based on multiple rounds of interaction information input via the user terminal.
[0044] Regarding the predetermined feature subset, it is used to indicate necessary features for constructing the response information.
[0045] At step 204, the computing device 110 performs a text embedding search on the query feature information and a predetermined data set to obtain a search result field, wherein the predetermined data set is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information, and predetermined knowledge base information.
[0046] In some embodiments, regarding the search result field, it at least includes, for example: key-value pairs regarding priority medication information, impact information between medications, and drug resistance information.
[0047] Regarding the method for obtaining a search result field, it includes, for example: the computing device 110 generates a search input field based on query question feature information; based on the search input field and a predetermined embedding model, a text embedding search is performed in a predetermined data set to obtain a floating point number vector, wherein the floating point number vector is used to indicate the distance between the search input field and each field included in the predetermined data set; and based on a comparison of the floating point number vector with a predetermined distance threshold, one or more search result fields that match the search input field are obtained.
[0048] Regarding the method of generating a search input field, in some embodiments, the computing device 110 can perform Boolean operations such as "and", "or", and "not" on multiple feature key-value pairs included in the query feature information in accordance with established medical logic, and combine other predetermined conditions to generate a search input field in natural language form.
[0049] Regarding the predetermined data set, it is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information and predetermined knowledge base information, and the predetermined data set includes multiple fields, each field is, for example, a line of natural speech form of clinical, drug, case or gene. In some embodiments, each field is, for example, a structured feature key-value pair generated by natural speech conversion.
[0050] Regarding the floating point vector (Embedding), it is the distance between two vectors. In the present invention, the floating point vector is used to indicate the distance (or correlation) between the search input field and the various fields included in the predetermined data set. It should be understood that a smaller distance represents a high correlation and a larger distance represents a low correlation. Regarding the predetermined distance threshold, in some embodiments, it is determined to be 0.2, for example. The computing device 110 determines the field in the predetermined data set corresponding to the floating point vector less than 0.2 as the search result field that matches the search input field.
[0051] At step 206 , the computing device 110 constructs a prompt based at least on the query question feature information and the obtained search result field to input into the trained language model for generating response information for the medical query information based at least on the output of the language model.
[0052] Regarding the method of generating response information for medical inquiry information, it includes, for example: the computing device 110 constructs a prompt based on at least the inquiry question feature information and the obtained search result field, so as to input the trained LLM model for polishing, so as to generate a first response letter for the medical inquiry information; the inquiry question feature information is input into the trained fine-tuning model, so as to generate a second response information for the medical inquiry information; based on the first response information and / or the second response information, a response letter for presentation to the user is generated. For example, the computing device 110 generates response information for presentation to the user based on the first response information; or generates response information and second response information for presentation to the user based on the second response information; or generates response information for presentation to the user based on the first response information and the second response information. For example, the first response information and the second response information are formed into a union to generate response information for presentation to the user. Regarding the language model, it can be an LLM model, which is, for example, constructed based on a deep learning model of a Transformer network architecture. The LLM model is, for example, the second LLM model of the present invention. It should be understood that the self-attention mechanism is a key part of the deep learning model network based on the Transformer network architecture, which can establish associations between different positions of the input sequence. The self-attention mechanism performs contextual modeling on the input sequence by calculating the attention weight of each position with respect to other positions. The multi-head self-attention mechanism allows the model to focus on different parts of the input sequence to extract richer feature representations. In some embodiments, the LLM model can be: a generalized linear model (GLM) model based on a transformer, or a moss model.
[0053] Regarding the loss function of the LLM model, in some embodiments, it is, for example, but not limited to, Maximum Likelihood Estimation (MLE). During the training process, the LLM model predicts the next tag by maximizing the conditional probability of the next tag. In other embodiments, the loss function of the LLM model is, for example, a negative log-likelihood loss function, which is used to measure the difference between the sequence generated by the model and the true sequence.
[0054] Regarding the training method of the LLM model, it includes, for example: the computing device 110 obtains clinical information, diagnosis and treatment information, tumor somatic gene information and predetermined gene knowledge base information about multiple target objects to generate input data; performs natural language conversion on the generated input data to generate clinical information indicator information and diagnosis and treatment sample information in a predetermined natural language format; determines the generated clinical information indicator information as a training prompt, and determines the diagnosis and treatment sample information as a training response item, so as to generate preprocessed training data and verification data; and trains the LLM model based on the preprocessed training data and verification data to obtain a trained LLM model.
[0055] In some embodiments, the computing device 110 determines whether the obtained trained LLM meets the requirements based on data (e.g., the tag accuracy training_token_accuracy) used to characterize the training results of the model. It should be understood that a higher tag accuracy indicates that the model is more accurate when generating text. For example, if the computing device 110 determines that the tag accuracy is greater than or equal to a predetermined accuracy threshold (e.g., 0.7), it is determined that the obtained trained LLM meets the requirements.
[0056] Regarding the training method of the fine-tuning model, it includes, for example: generating a training data set for the fine-tuning model based on locally stored clinical information, tumor somatic gene data, structured diagnosis and treatment information, and gene-related knowledge base; converting the fine-tuning model training data set into a JSONL document, the JSONL document including multiple lines of prompts and answer item pairs, each line of prompts and answer item pairs corresponding to a training example; based on the JSONL document, fine-tuning a predetermined basic model to generate a fine-tuning model. It should be understood that by using a JSONL document constructed based on a professional fine-tuning model training data set to fine-tune the basic model, the present invention can no longer provide examples in the prompt. It is beneficial to save costs and achieve lower latency requests; in addition, fine-tuning improves learning by training more examples than in the prompt, and the present invention can obtain higher quality second response information.
[0057] Regarding the training dataset for fine-tuning the model, it includes, for example, data columns or key-value pairs corresponding to prompts and completion items. For example, the computing device 110 may convert the training dataset for fine-tuning the model into a JSONL document for fine-tuning the predetermined basic model.
[0058] Regarding the predetermined basic model, it may be a general basic model, which is not limited in the present invention.
[0059] In the above scheme, by using the query feature information and the professional data set to perform text embedding search to obtain the search result field, the present invention can make the search result field semantically more accurately match the user's query. In addition, by using the search result field obtained after the embedded search together with the query feature information to construct a prompt for inputting the LLM model, so as to generate the response information through the output of the language model, the present invention can more accurately understand the user's query intention, so that the response information more accurately matches the user's query intention. Therefore, the present invention can effectively achieve accurate responses to medical query questions.
[0060] The following will combine Figure 3 It is used to illustrate the method for training the LLM model, which will not be repeated here.
[0061] The following will be combined Figure 3 A method for generating a trained LLM model or a method for training an LLM model according to an embodiment of the present invention is described. Figure 3 FIG. 3 is a flowchart of a method 300 for generating a trained LLM model according to an embodiment of the present invention. It should be understood that the method 300 may be, for example, Figure 5 The electronic device 500 described above is executed. Figure 1 The method 300 is executed at the described computing device 110. It should be understood that the method 300 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0062] At step 302, the computing device 110 obtains clinical information, diagnosis and treatment information, tumor somatic gene information, and predetermined gene knowledge base information about multiple target objects to generate input data. Thus, the present invention can make the generated input data include not only structured feature data derived from the professional knowledge base, but also real, highly reliable clinical case data. In some embodiments, the input data includes, for example, multiple feature identifiers and feature values corresponding to the feature identifiers.
[0063] Regarding the predetermined gene knowledge base information, it includes, for example, structured feature data. The following Table 1 exemplarily shows the structured feature data in the predetermined gene knowledge base information.
[0064] Table 1
[0065]
[0066]
[0067] For example, the computing device 110 generates input data based on clinical information, pathological information, tumor somatic gene information, structured diagnosis and treatment information (for example, including treatment follow-up information), and data from a gene-related knowledge base (for example, a professional knowledge base involving relevant parameters) acquired about a large number of cases.
[0068] At step 304, the computing device 110 performs natural language conversion on the generated input data to generate clinical information indicator information and diagnosis and treatment sample information in a predetermined natural language format. In some embodiments, the computing device 110 performs natural language conversion on the feature identifier in the input feature and the feature value corresponding to the feature identifier via the LLM model to convert it into clinical information indicator information and diagnosis and treatment sample information in a predetermined natural language format. Clinical information indicator information is, for example, more concise clinical information associated with the target object. Diagnosis and treatment sample information is, for example, more concise treatment case information. For example, the converted clinical information indicator information and diagnosis and treatment sample information in natural language format are "Referring to the CSCO guidelines, non-small cell lung cancer (NSCLC) and in stage I can undergo surgical resection. The treatment plan is to perform anatomical lobectomy through thoracoscopic surgery and remove the hilar and mediastinal lymph nodes. The recommended treatment level is level 1". It should be understood that the present invention improves the understanding ability of the LLM model by using clinical, tumor somatic gene data, and structured diagnosis and treatment and gene-related knowledge base as input data of the LLM model, converting the data into natural language, and then training it.
[0069] At step 306 , the computing device 110 determines the generated clinical information indicator information as a training prompt, and determines the diagnosis and treatment sample information as a training response item, so as to generate preprocessed training data and verification data.
[0070] A method for generating preprocessed training data and verification data, for example, includes: the computing device 110 divides training prompts and training response entries into training data and verification data; converts the training prompts and training response entries in the training data and verification data into JSON format to generate prompts and response entries in JSON format; and marks evidence classification levels for the prompts and response entries in JSON format to generate preprocessed training data and verification data.
[0071] Specifically, the computing device 110 divides the clinical information indicator information and diagnosis and treatment sample information in the natural language format generated at step 304 into two types: clinical information indicator information and diagnosis and treatment sample information; then, the clinical information indicator information is used as a training prompt for training the LLM model, and the diagnosis and treatment sample information is used as a training response item for training the LLM model. In addition, the computing device 110 divides the training prompt and the training response item into two files, one file as training data, and the other file as validation data. Then, the computing device 110 converts the training prompt and the training response item in the training data and validation data into json format, and marks the evidence classification level for the prompt and response items in json format. For example, the preprocessed training data and validation data generated through json format conversion and evidence classification level labeling are: {"prompt":"Referring to the CSCO guidelines, non-small cell lung cancer (NSCLC) in stage I can be surgically resected. The treatment plan is to perform anatomical lobectomy through thoracoscopic surgery and remove the hilar and mediastinal lymph nodes. The recommended treatment level is level 1"\t"completion":"The evidence priority is A"}. In some embodiments, for training data and validation data of different evidence levels, the computing device 110 performs classification training to improve the accuracy of the model.
[0072] At step 308 , the computing device 110 trains the LLM model based on the preprocessed training data and validation data to obtain a trained LLM model.
[0073] Regarding the method for training an LLM model, it includes, for example: pre-training the LLM model with a first predetermined loss function until convergence, so as to obtain a pre-trained LLM model; and training the pre-trained LLM model with at least one predetermined loss function until convergence, so as to obtain a trained LLM model. Regarding the LLM model, it is, for example, a second LLM model for generating and polishing response information.
[0074] Regarding the first predetermined loss function, it is, for example, a maximum likelihood estimation (Maximum Likelihood Estimation, MLE) or a negative log-likelihood loss function.
[0075] Regarding at least one predetermined loss function, it includes, for example: a cross entropy loss function and a mean square error loss function.
[0076] For example, the computing device 110 obtains training data and verification data for a predetermined query task for model adjustment; based on the training data and verification data for the predetermined query task, the type of the current query task is confirmed; in response to confirming that the type of the current query task is a classification task, the pre-trained LLM model is trained with a cross-entropy loss function until convergence; and in response to confirming that the type of the current query task is a regression task, the pre-trained LLM model is trained with a mean squared error loss function until convergence. By adopting the above means, the LLM model of the present invention can be adapted to specific tasks and data by using different loss functions in the fine-tuning process. For example, a cross-entropy loss function (Cross-EntropyLoss) is used for classification tasks, and a mean squared error loss function (Mean Squared Error Loss) is used for regression tasks, depending on the type of the query task and the user's query requirements.
[0077] In the above scheme, the present invention can train the LLM model by using real, highly reliable sample data containing clinical features, so that the trained LLM model can be more accurately applied to analyze the user's medical (for example, but not limited to, diagnosis and treatment) related inquiries, thereby making the response information more accurate.
[0078] The following will be combined Figure 4 A method 400 for performing text embedding search to obtain search result fields according to an embodiment of the present invention is described. Figure 4 FIG. 4 is a flowchart of a method 400 for performing text embedding search to obtain search result fields according to an embodiment of the present invention. It should be understood that the method 400 can be performed, for example, in Figure 5 The electronic device 500 described above is executed. Figure 1 The method 400 is executed at the described computing device 110. It should be understood that the method 400 may also include additional actions not shown and / or may omit the actions shown, and the scope of the present invention is not limited in this respect.
[0079] At step 402 , the computing device 110 generates a search input field based on the query question feature information.
[0080] At step 404, the computing device 110 performs a text embedding search in a predetermined data set based on the search input field and a predetermined embedding model to obtain a floating point number vector indicating the distance between the search input field and each field included in the predetermined data set.
[0081] Regarding the method for constructing a predetermined data set, in some embodiments, for example, it includes: the computing device 110 generates a data union based on clinical information, drug information, sample case information, diagnosis and treatment information, gene information and predetermined knowledge base information; inputs multiple groups of associated data in the data union into the LLM model respectively so as to be converted into multiple natural language fields, each natural language field corresponding to a group of associated data; constructs a predetermined data set based on the converted multiple natural language fields, so that each natural language field is a row of fields in the predetermined data set, so as to match the search input field during text embedding retrieval. In some embodiments, the computing device 110 obtains diagnosis and treatment related guidelines and knowledge bases to generate a predetermined data set.
[0082] Table 2 below schematically shows a portion of multiple sets of associated data in a data union. Each set of associated data includes multiple feature names and corresponding feature values.
[0083] Table 2
[0084]
[0085]
[0086] For example, each row of data in Table 2 represents a group of associated data. It should be understood that the computing device 110 can input each group of associated feature names and corresponding feature values into the LLM model to output a converted natural language field, thereby constructing a predetermined data set based on all natural language fields.
[0087] At step 406 , the computing device 110 obtains one or more search result fields that match the search input field based on a comparison of the floating point number vector and a predetermined distance threshold.
[0088] For example, the computing device 110 integrates and extracts features from a predetermined data set to obtain a predetermined feature set and a predetermined feature subset. The predetermined feature set or the predetermined feature subset includes a plurality of feature identifiers and feature values corresponding to the feature identifiers, respectively. The predetermined feature set is a set of total features. The predetermined feature subset indicates necessary features for constructing response information. Different feature identifiers and feature values corresponding to the feature identifiers can be combined with different database fields, thereby generating accurate response information for different preset medical inquiry questions.
[0089] By adopting the above-mentioned means, through the search result field obtained after text embedding search using query feature information and professional data sets, the present invention can more accurately understand the search intent and perform semantic search matching in a manner that is more consistent with medical logic, thereby significantly improving the correlation between the search result field and the query feature information.
[0090] Figure 5 The block diagram of an electronic device 500 suitable for implementing an embodiment of the present invention is schematically shown. The electronic device 500 may be a device for implementing Figures 2 to 3 The method 200 to 300 shown. Figure 5 As shown, the electronic device 500 includes a central processing unit (i.e., CPU 501), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (i.e., ROM 502) or loaded from a storage unit 508 to a random access memory (i.e., RAM 503). In RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output interface (i.e., I / O interface 505) is also connected to the bus 504.
[0091] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, and a storage unit 508. The CPU 501 performs the various methods and processes described above, such as performing methods 200 to 500. For example, in some embodiments, methods 200 to 500 may be implemented as a computer software program, which is stored in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more operations of the methods 200 to 500 described above may be performed. Alternatively, in other embodiments, the CPU 501 may be configured to perform one or more actions of the methods 200 to 500 by any other appropriate means (e.g., by means of firmware).
[0092] It should be further noted that the present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0093] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.
[0094] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0095] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.
[0096] Here, various aspects of the present invention are described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to an embodiment of the present invention. It should be understood that each square frame of the flow chart and / or block diagram and the combination of each square frame in the flow chart and / or block diagram can be implemented by computer-readable program instructions.
[0097] These computer-readable program instructions can be provided to a processor in a voice interaction device, a general-purpose computer, a special-purpose computer, or a processing unit of other programmable data processing devices, thereby producing a machine, so that when these instructions are executed by a processing unit of a computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions enable the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0099] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of this module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0100] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
[0101] The above are only optional embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for providing response information to a medical inquiry, characterized in that: include: Extracting features from the acquired medical inquiry information of the user, so as to generate inquiry question feature information having a predetermined natural language format based at least on the extracted features; Performing a text embedding search on the query feature information and a predetermined data set to obtain a search result field, wherein the predetermined data set is constructed based on at least one of clinical information, drug information, sample case information, diagnosis and treatment information, gene information, and predetermined knowledge base information; Constructing a prompt based at least on the inquiry question feature information and the obtained search result field to input into a trained language model, so as to generate response information for the medical inquiry information based at least on the output of the language model; Wherein, generating the inquiry question feature information in a predetermined natural language format based at least on the extracted features includes: based on the extracted features, confirming whether the following two predetermined conditions are satisfied, the two predetermined conditions including: confirming based on the predetermined feature subset that the necessary features for constructing the response information are not sufficient; and Confirm that the user belongs to the group of people with predetermined attributes; In response to confirming that both of the two predetermined conditions are met, one or more rounds of interaction information are output to the user via the language interaction model until the necessary features for constructing the response information are sufficient or the interaction information provided by the user indicates that it is impossible to continue to input information corresponding to the necessary features.
2. According to the method of claim 1, the language model is an LLM model, characterized in that: Generating response information for the medical query information based at least on the output of the language model includes: Constructing a prompt based at least on the inquiry question feature information and the obtained search result fields, so as to input the prompt into the trained LLM model for polishing, so as to generate a first response letter for the medical inquiry information; Inputting the inquiry question feature information into the trained fine-tuning model to generate second response information for the medical inquiry information; Based on the first response information and / or the second response information, response information is generated for presentation to the user.
3. The method according to claim 2, characterized in that The trained fine-tuned model is generated by: Generate training data sets for fine-tuning models based on locally stored clinical information, tumor somatic gene data, structured diagnosis and treatment information, and gene-related knowledge bases; Convert the fine-tuning model training dataset into a JSONL document, wherein the JSONL document includes multiple lines of prompt and answer item pairs, and each line of prompt and answer item pairs corresponds to a training example; Based on the JSONL document, fine-tuning is performed on a predetermined base model to generate a fine-tuned model.
4. The method according to claim 1, characterized in that: The language model is an LLM model, and the trained language model is generated by: Acquiring clinical information, diagnosis and treatment information, tumor somatic cell gene information, and predetermined gene knowledge base information about a plurality of target subjects to generate input data; Performing natural language conversion on the generated input data to generate clinical information indicator information and diagnosis and treatment sample information in a predetermined natural language format; Determining the generated clinical information indicator information as a training prompt, and determining the diagnosis and treatment sample information as a training response item, so as to generate preprocessed training data and verification data; as well as Based on the preprocessed training data and validation data, the LLM model is trained to obtain a trained LLM model.
5. The method according to claim 4, characterized in that Generating preprocessed training and validation data involves: Divide the training prompts and training response items into training data and verification data; Converting the training prompts and training answer items in the training data and the validation data into JSON format to generate prompts and answer items in JSON format; and The evidence classification levels are marked for the prompts and answer items in JSON format to generate preprocessed training data and validation data.
6. The method according to claim 1, characterized in that The language model is an LLM model, and training the language model includes: Pre-training the LLM model with a first predetermined loss function until convergence to obtain a pre-trained LLM model; and The pre-trained LLM model is trained with at least one predetermined loss function until convergence to obtain a trained LLM model.
7. According to the method of claim 6, the LLM model is constructed based on a deep learning model of a Transformer network architecture, characterized in that: Training the pre-trained LLM model with at least one predetermined loss function until convergence includes: Obtain training data and validation data for model tuning on predetermined query tasks; Based on the training data and the validation data about the predetermined query task, determining the type of the current query task; In response to confirming that the type of the current query task is a classification task, training the pre-trained LLM model with a cross entropy loss function until convergence; and In response to confirming that the type of the current query task is a regression task, the pre-trained LLM model is trained with a mean square error loss function until convergence.
8. The method according to claim 1, characterized in that Generating inquiry question feature information having a predetermined natural language format based at least on the extracted features further includes: If it is confirmed that both of the two predetermined conditions are not satisfied, generating inquiry question feature information in a predetermined natural language format based on the features extracted from the medical inquiry information; and If it is confirmed through one or more rounds of interactive information that the necessary features for constructing the response information are sufficient, or the interactive information provided by the user indicates that it is impossible to continue to input the information corresponding to the necessary features, inquiry question feature information with a predetermined natural language format is generated based on the features extracted from the medical inquiry information and the interactive information provided by the user.
9. The method according to claim 1, characterized in that: The query feature information is searched with a predetermined data set through text embedding, so as to obtain search result fields including: Generate search input fields based on the query question feature information; Based on the search input field and the predetermined embedding model, performing a text embedding search in a predetermined data set to obtain a floating point number vector, wherein the floating point number vector is used to indicate the distance between the search input field and each field included in the predetermined data set; and Based on the comparison of the floating point number vector with a predetermined distance threshold, one or more search result fields matching the search input field are obtained.
10. A computing device, characterized in that include: at least one processing unit; At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the steps of the method according to any one of claims 1 to 9.
11. A computer-readable storage medium, characterized in that: A computer program is stored on a computer-readable storage medium, and when the computer program is executed by a machine, the method according to any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
Question and answer method and device
CN109829045A
Problem generation model training method, problem generation method and related equipment thereof
CN111639163A
Emotion-fused dialogue response method and device, terminal and storage device
CN114595314A
Question and answer model determination method, question and answer method and device, medium and equipment
CN114780703A
Method for training dialog model and method for outputting response information
CN115080739A
