Evaluation method and device for medical consultation dialogue model, electronic device and storage medium

Through multi-dimensional evaluation of the consultation dialogue model, combined with correlation values, diagnostic values ​​and prescription values, the problem of inaccurate evaluation of the consultation dialogue model in the prior art is solved, and the accuracy of the evaluation is improved.

CN114613493BActive Publication Date: 2024-11-19PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210252476.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-15
Publication Date
2024-11-19
Estimated Expiration
2042-03-15

AI Technical Summary

Technical Problem

Existing methods of consultation dialogue model evaluation, such as BLEU, cannot accurately evaluate the output of consultation dialogue model from the perspective of medical knowledge, resulting in low evaluation accuracy.

Method used

By entering the patient information to be tested into the consultation dialogue model, the dialogue data set is obtained, and information extraction is carried out to obtain the target symptom information, target prescription information and target diagnosis information, and calculate the correlation value, diagnostic value and prescription value based on the preset information. Form multi-dimensional assessment.

Benefits of technology

The evaluation accuracy of the consultation dialogue model is improved, and it can more accurately determine whether the model has asked about the main symptoms during the consultation process, and evaluate the accuracy of diagnosis and prescriptions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114613493B_ABST
    Figure CN114613493B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides an evaluation method and device, electronic device and storage medium for a medical consultation dialogue model, which belongs to the field of artificial intelligence technology. The method includes: inputting the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set; obtaining the original symptom information corresponding to the patient information to be tested; performing information extraction processing on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information; performing association processing on the target symptom information and the original symptom information to obtain an association value; the association value is used to characterize the degree of association between the original symptom information and the target symptom information; according to the preset original diagnosis information and the target diagnosis information, the diagnosis value is calculated; according to the preset original prescription information and the target prescription information, the prescription value is calculated; according to the association value, the diagnosis value and the prescription value, the target evaluation value is obtained. The technical solution of the embodiment of the present application improves the accuracy of the evaluation of the medical consultation dialogue model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an evaluation method and device for a medical consultation dialogue model, an electronic device, and a storage medium. Background Art

[0002] The medical consultation dialogue model needs to be evaluated to understand the accuracy of the output of the medical consultation dialogue model.

[0003] In related technologies, BLEU (Bilingual Evaluation Understudy) is used to evaluate text dialogue models. However, BLEU only evaluates the consultation dialogue model from the perspective of text fluency or richness, and cannot evaluate it from the perspective of medical knowledge, resulting in low evaluation accuracy. Therefore, how to improve the accuracy of the consultation dialogue model evaluation has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a method and device for evaluating a medical consultation dialogue model, an electronic device and a storage medium, aiming to improve the accuracy of the evaluation of the medical consultation dialogue model.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for evaluating a medical consultation dialogue model, the method comprising:

[0006] Input the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set;

[0007] Obtaining original symptom information corresponding to the patient information to be tested;

[0008] Performing information extraction processing on the dialogue data set to obtain target symptom information, target prescription information, and target diagnosis information;

[0009] Associating the target symptom information with the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information;

[0010] A diagnostic value is calculated based on the preset original diagnostic information and the target diagnostic information; the diagnostic value is used to characterize the accuracy of the target diagnostic information;

[0011] Calculating a prescription value based on the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information;

[0012] A target evaluation value is obtained according to the associated value, the diagnostic value and the prescription value.

[0013] In some embodiments, the extracting information from the conversation data set to obtain target symptom information, target prescription information, and target diagnosis information includes:

[0014] Performing information extraction processing on the conversation data set to obtain the target symptom information and drug information;

[0015] The target prescription information and the target diagnosis information are obtained according to the preset symptom-drug mapping relationship and the drug information.

[0016] In some embodiments, the associating the target symptom information with the original symptom information to obtain an associated value includes:

[0017] Acquire common symptom information of the original symptom information and the target symptom information;

[0018] An original symptom value is calculated based on the common symptom information and the original symptom information; the original symptom value is used to represent the proportion of the common symptom information in the original symptom information;

[0019] A target symptom value is calculated based on the common symptom information and the target symptom information; the target symptom value is used to represent the proportion of the common symptom information in the target symptom information;

[0020] The associated value is calculated based on the original symptom value and the target symptom value.

[0021] In some embodiments, calculating the association value according to the original symptom value and the target symptom value includes:

[0022] Performing a sum operation on the original symptom value and the target symptom value to obtain a symptom sum value;

[0023] Performing a product operation on the original symptom value and the target symptom value to obtain a symptom product value;

[0024] The correlation value is calculated based on the symptom sum value and the symptom product value.

[0025] In some embodiments, obtaining a target evaluation value according to the association value, the diagnosis value and the prescription value includes:

[0026] Obtaining a preset association weight, a preset diagnosis weight, and a preset prescription weight;

[0027] Performing weighted processing on the association value according to the association weight to obtain an association weight value;

[0028] Performing weighted processing on the diagnosis value according to the diagnosis weight to obtain a diagnosis weight value;

[0029] Performing weighted processing on the prescription value according to the prescription weight to obtain a prescription weight value;

[0030] The target evaluation value is obtained by summing up the associated weight value, the diagnostic weight value and the prescription weight value.

[0031] In some embodiments, before inputting the patient information to be tested into the medical consultation dialogue model to obtain the dialogue data set, the method further includes:

[0032] Constructing the medical consultation dialogue model specifically includes:

[0033] Acquire original training data; wherein the original training data includes: original patient data and original conversation data;

[0034] Performing enhancement processing on the original conversation data according to the original patient data to obtain conversation enhancement data;

[0035] The preset original dialogue model is trained according to the dialogue enhancement data and the original patient data to obtain the medical consultation dialogue model.

[0036] In some embodiments, the training process of the preset original dialogue model according to the dialogue enhancement data and the original patient data to obtain the consultation dialogue model includes:

[0037] Obtaining a target loss function of the original dialogue model;

[0038] Performing feature extraction on the dialogue enhancement data to obtain a dialogue enhancement vector;

[0039] Performing feature extraction on the original patient data to obtain an original patient vector;

[0040] Performing encoding processing on the dialogue enhancement vector to obtain dialogue encoding data;

[0041] Performing encoding processing on the original patient vector to obtain patient encoding data;

[0042] Decoding the conversation coded data and the patient coded data to obtain target reply information;

[0043] Calculating the loss value of the target loss function according to the target response information to obtain a target loss value;

[0044] The parameters of the original dialogue model are updated according to the target loss value to obtain the medical consultation dialogue model.

[0045] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an evaluation device for a medical consultation dialogue model, the device comprising:

[0046] An input module is used to input the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set;

[0047] An acquisition module, used for acquiring original symptom information corresponding to the patient information to be tested;

[0048] An extraction module, used to perform information extraction processing on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information;

[0049] An association module, used for performing association processing on the target symptom information and the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information;

[0050] A first calculation module, used to calculate a diagnosis value according to preset original diagnosis information and the target diagnosis information; the diagnosis value is used to characterize the accuracy of the target diagnosis information;

[0051] A second calculation module is used to calculate a prescription value based on the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information;

[0052] The third calculation module is used to obtain a target evaluation value according to the associated value, the diagnostic value and the prescription value.

[0053] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program, when executed by the processor, implements the method described in the first aspect above.

[0054] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method described in the first aspect above.

[0055] The evaluation method and device, electronic device and storage medium of the inquiry dialogue model proposed in the present application, which obtains a dialogue data set by inputting the patient information to be tested into the inquiry dialogue model, and obtains the original symptom information corresponding to the patient information to be tested, then extracts information from the dialogue data set to obtain the target symptom information, target prescription information and target diagnosis information generated by the inquiry dialogue model according to the patient information to be tested, and then associates the target symptom information with the original symptom information to obtain an association value, wherein the association value is used to characterize the degree of association between the original symptom information and the target symptom information. In this way, it can be determined whether the main symptoms of the patient information to be tested are all inquired when the inquiry dialogue model conducts an inquiry dialogue. Then, according to the preset original diagnosis information and target diagnosis information, a diagnosis value is calculated, wherein the diagnosis value is used to characterize the accuracy of the target diagnosis information; then, according to the preset original prescription information and target prescription information, a prescription value is calculated, wherein the prescription value is used to characterize the accuracy of the target prescription information. Then, according to the association value, the diagnosis value and the prescription value, a target evaluation value is obtained. Through this setting, the medical consultation dialogue model is evaluated from three dimensions: association value, diagnosis value and prescription value, thereby improving the accuracy of the medical consultation dialogue model evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of an evaluation method of a medical consultation dialogue model provided in an embodiment of the present application;

[0057] Figure 2 is another flow chart of the evaluation method of the medical consultation dialogue model provided in the embodiment of the present application;

[0058] Figure 3 It is a structural diagram of the medical consultation dialogue model provided in the embodiment of the present application;

[0059] Figure 4 yes Figure 2 Flow chart of the specific method of step S1000;

[0060] Figure 5 yes Figure 1 A flowchart of the specific method of step S300 in FIG.

[0061] Figure 6 yes Figure 1 A flowchart of the specific method of step S400 in FIG.

[0062] Figure 7 yes Figure 6 A flowchart of the specific method of step S440 in FIG.

[0063] Figure 8 yes Figure 1 A flowchart of the specific method of step S700;

[0064] Fig. 9 It is a structural schematic diagram of an evaluation device for a medical consultation dialogue model provided in an embodiment of the present application;

[0065] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0069] First, some nouns involved in this application are analyzed:

[0070] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0071] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese, English, etc.). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text understanding. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0072] Information Extraction (NER): A text processing technology that extracts specified types of entity, relationship, event and other factual information from natural language text and forms structured data output. Information extraction is a technology that extracts specific information from text data. Text data is composed of some specific units, such as sentences, paragraphs, and chapters. Text information is composed of some small specific units, such as characters, words, phrases, sentences, paragraphs, or a combination of these specific units. Extracting noun phrases, names, place names, etc. from text data are all text information extraction. Of course, the information extracted by text information extraction technology can be various types of information.

[0073] Embedding: Embedding is a vector representation, which means using a low-dimensional vector to represent an object. The object can be a word, a product, a movie, etc. The property of this embedding vector is that it can make objects corresponding to vectors with similar distances have similar meanings. For example, the distance between embedding (Avengers) and embedding (Iron Man) will be very close, but the distance between embedding (Avengers) and embedding (Gone with the Wind) will be farther. Embedding is essentially a mapping from semantic space to vector space, while trying to maintain the relationship of the original samples in the semantic space in the vector space as much as possible. For example, the positions of two semantically similar words in the vector space are also relatively close. Embedding can encode objects with low-dimensional vectors and retain their meaning. It is often used in machine learning. In the process of building machine learning models, objects are encoded into a low-dimensional dense vector and then passed to DNN to improve efficiency.

[0074] Medical cloud: Medical cloud refers to the use of cloud computing to create a medical and health service cloud platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communication, big data, and the Internet of Things, combined with medical technology, to achieve the sharing of medical resources and the expansion of medical scope. Because of the application of cloud computing technology, medical cloud improves the efficiency of medical institutions and facilitates residents to seek medical treatment. For example, hospital appointment registration, electronic medical records, medical insurance, etc. are all products of the combination of cloud computing and the medical field. Medical cloud also has the advantages of data security, information sharing, dynamic expansion, and overall layout.

[0075] The medical consultation dialogue model needs to be evaluated to understand the accuracy of the output of the medical consultation dialogue model.

[0076] In related technologies, BLEU (Bilingual Evaluation Understudy) is used to evaluate text dialogue models. However, BLEU only evaluates the consultation dialogue model from the perspective of text fluency or richness, and cannot evaluate it from the perspective of medical knowledge, resulting in low evaluation accuracy. Therefore, how to improve the accuracy of the consultation dialogue model evaluation has become a technical problem that needs to be solved urgently.

[0077] Based on this, the embodiments of the present application provide a method and device for evaluating a medical consultation dialogue model, an electronic device, and a storage medium, aiming to improve the accuracy of the evaluation of the medical consultation dialogue model.

[0078] The evaluation method and device, electronic device and storage medium of the medical consultation dialogue model provided in the embodiments of the present application are specifically explained through the following embodiments. First, the evaluation method of the medical consultation dialogue model in the embodiments of the present application is described.

[0079] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0080] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0081] The evaluation method of the medical consultation dialogue model provided in the embodiment of the present application relates to the field of artificial intelligence technology. The evaluation method of the medical consultation dialogue model provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the evaluation method of the medical consultation dialogue model, etc., but is not limited to the above forms.

[0082] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0083] The technical solution of the embodiment of the present application is further described below in conjunction with the accompanying drawings.

[0084] Figure 1 is an optional flowchart of the evaluation method of the medical consultation dialogue model provided in the embodiment of the present application. Figure 1 The method may include step S100, step S200, step S300, step S400, step S500, step S600 and step S700. It should be understood that the evaluation method of the medical consultation dialogue model of the embodiment of the present application includes but is not limited to step S100 to step S700. Figure 1 These seven steps are introduced in detail.

[0085] Step S100, inputting the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set;

[0086] Step S200, obtaining original symptom information corresponding to the patient information to be tested;

[0087] Step S300, performing information extraction processing on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information;

[0088] Step S400, performing association processing on the target symptom information and the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information;

[0089] Step S500, calculating a diagnostic value based on the preset original diagnostic information and target diagnostic information; the diagnostic value is used to characterize the accuracy of the target diagnostic information;

[0090] Step S600, calculating a prescription value based on the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information;

[0091] Step S700, obtaining a target evaluation value according to the associated value, the diagnosis value and the prescription value.

[0092] The evaluation method of the medical consultation dialogue model of the embodiment of the present application, which obtains a dialogue data set by inputting the patient information to be tested into the medical consultation dialogue model, and obtains the original symptom information corresponding to the patient information to be tested, then extracts information from the dialogue data set to obtain the target symptom information, target prescription information and target diagnosis information generated by the medical consultation dialogue model according to the patient information to be tested, and then associates the target symptom information with the original symptom information to obtain the association value, wherein the association value is used to characterize the association between the original symptom information and the target symptom information. In this way, it can be determined whether the main symptoms of the patient information to be tested are all inquired when the medical consultation dialogue model conducts the medical consultation dialogue. Then, according to the preset original diagnosis information and target diagnosis information, the diagnosis value is calculated, wherein the diagnosis value is used to characterize the accuracy of the target diagnosis information; then, according to the preset original prescription information and target prescription information, the prescription value is calculated, wherein the prescription value is used to characterize the accuracy of the target prescription information. Then, according to the association value, the diagnosis value and the prescription value, the target evaluation value is obtained. Through this setting, the medical consultation dialogue model is evaluated from the three dimensions of association value, diagnosis value and prescription value, thereby improving the accuracy of the evaluation of the medical consultation dialogue model.

[0093] Please refer to Figure 2 and Figure 3 , Figure 2 Another flow chart of the evaluation method of the medical consultation dialogue model provided in some embodiments of the present application, Figure 3 A schematic diagram of the structure of the medical consultation dialogue model provided in the embodiment of the present application.

[0094] Before step S100 in some embodiments, the method for evaluating the medical consultation dialogue model further includes the following steps:

[0095] Constructing a medical consultation dialogue model, specifically including steps S800, S900, and S1000, Figure 2 and Figure 3 These three steps are introduced in detail.

[0096] Step S800, obtaining original training data; wherein the original training data includes: original patient data and original conversation data;

[0097] Step S900, enhancing the original conversation data according to the original patient data to obtain conversation enhanced data;

[0098] Step S1000, training a preset original dialogue model according to the dialogue enhancement data and the original patient data to obtain a consultation dialogue model.

[0099] In step S800 of some embodiments, the original patient data is called persona, and the original dialogue data is called dialogue. The original patient data mainly comes from the consultation cases compiled by the doctor, which can be obtained through the medical cloud server; the original dialogue data mainly comes from a large number of online consultation dialogues, which can be obtained through the medical cloud server. However, it should be noted that the original patient data and the original dialogue data should have a certain correlation. When training the consultation dialogue model, the consultation dialogue model will answer the doctor's questions based on the input persona information.

[0100] In step S900 of some embodiments, in the related technology, when training the dialogue model, the scenario adopted is Open-Domain Question Answering. In this case, the dialogue model is unstable when facing questions unrelated to persona information, and some answers are randomly generated. This open-ended answer scenario has little impact on chatting. However, the technical solution of the embodiment of the present application is a medical consultation dialogue model. In the medical consultation scenario, the requirement for accuracy is strong. If an incorrect answer is generated, it will have a serious impact on subsequent medical consultations, diagnoses and prescriptions. Therefore, in this application, data enhancement processing of person information is required.

[0101] Specifically, during the training of the consultation dialogue model, some question-answer pairs are added to enhance the information related to the original patient data. For example, when there is no information related to chest pain in the original patient data, the following question-answer pairs are added to the consultation part:

[0102] "Doctor: Do you have chest pain?"

[0103] “Patient: I don’t know.”

[0104] Through similar data enhancement processing methods, when training the medical consultation dialogue model, if faced with questions unrelated to the original patient data, some erroneous information will not be introduced, thereby further improving the accuracy of the medical consultation dialogue model.

[0105] Please refer to Figure 3 and Figure 4 In some embodiments, step S1000 includes but is not limited to step S1010 to step S1080. Figure 3 and Figure 4 The specific process of "training the original dialogue model according to the dialogue enhancement data and the original patient data to obtain the medical consultation dialogue model" in the embodiment of the present application is described in detail.

[0106] Step S1010, obtaining the target loss function of the original dialogue model;

[0107] Step S1020, extracting features from the dialogue enhancement data to obtain a dialogue enhancement vector;

[0108] Step S1030, extracting features from the original patient data to obtain an original patient vector;

[0109] Step S1040, encoding the dialogue enhancement vector to obtain dialogue encoding data;

[0110] Step S1050, encoding the original patient vector to obtain patient encoding data;

[0111] Step S1060, decoding the conversation coded data and the patient coded data to obtain target reply information;

[0112] Step S1070, calculating the loss value of the target loss function according to the target reply information to obtain the target loss value;

[0113] Step S1080, updating the parameters of the original dialogue model according to the target loss value to obtain a medical consultation dialogue model.

[0114] In step S1010 of some embodiments, the target loss function may be a softmax function or other loss functions.

[0115] In step S1020 of some embodiments, feature extraction is performed on the dialogue enhancement data through the embedding layer of the original dialogue model to obtain a dialogue enhancement vector.

[0116] In step S1030 of some embodiments, similar to the aforementioned steps, feature extraction is performed on the original patient data through the embedding layer of the original dialogue model to obtain the corresponding original patient vector.

[0117] In steps S1040 to S1060 of some embodiments, the dialogue enhancement vector is passed through the encoder layer of the original dialogue model to obtain the encoding of the original dialogue data, that is, the dialogue encoding data is obtained; the original patient vector is passed through the encoder layer of the original dialogue model to obtain the encoding of the persona, that is, the patient encoding data. At this time, the original dialogue model already has the information of the original dialogue data and the target personality information of the original patient data, and the goal is to generate a response message. Therefore, the dialogue encoding data and the patient encoding data are passed through the decoder layer of the original dialogue model to obtain the target response information. The target response information may contain the target personality information of the original patient data, or may not contain it at all. Therefore, the target response information can be expressed by the following formula (1), which is specifically:

[0118] (1)

[0119] In formula (1), represents the target personality information of the original patient data, Represents the original conversation data, Represents the target reply information. From formula (1), we can see that The larger the value is, the greater the target personality information of the original patient data will be in the target response information. Therefore, training the original dialogue model is equivalent to training training.

[0120] In steps S1070 to S1080 of some embodiments, since the goal of this solution is to train a medical consultation dialogue model, in order to ensure the accuracy of the medical consultation, it is necessary to have requirements for the target personality information of the original patient data. Therefore, it can be determined whether the target reply information contains keywords related to the original patient data to determine whether it is an answer to the original patient data. Specifically, the target reply information can be annotated: if the target reply information contains keywords related to the original patient data, the target reply information is annotated as 1, otherwise it is 0, and then the loss value of formula (1) is calculated, and then the loss value of the original dialogue model is calculated based on the target reply information of the original dialogue model. Finally, the loss value of formula (1) and the loss value of the original dialogue model are combined to update the parameters of the original dialogue model to obtain the medical consultation dialogue model.

[0121] It should be noted that in the related art, since the dialogue is conducted in the "Open-Domain Question Answering" scenario, the top_k, top_p decoding strategy is adopted. However, this method sacrifices the accuracy of the dialogue and improves the richness of the dialogue, but it is not suitable for scenarios such as medical consultations that require extremely high dialogue accuracy. Therefore, the greedy_search decoding strategy is adopted in this application to ensure the accuracy of the dialogue.

[0122] In step S100 of some embodiments, a web crawler can be written to crawl data in a targeted manner after setting the data source to obtain the patient information to be tested. The patient information to be tested can also be directly obtained from the medical cloud server. After obtaining the patient information to be tested, the patient information to be tested is input into the medical consultation dialogue model to obtain a dialogue data set. The medical consultation dialogue model is obtained by training in the above steps.

[0123] It should be noted that the technical solution of the embodiment of the present application is to evaluate the medical consultation dialogue model, so the original symptom information corresponding to the patient information to be tested is known.

[0124] Please refer to Figure 5 In some embodiments of the present application, step S300 includes step S310 and step S320. It should be understood that step S300 includes but is not limited to step S310 to step S320. Figure 5 These two steps are introduced in detail.

[0125] Step S310, extracting information from the conversation data set to obtain target symptom information and drug information;

[0126] Step S320, obtaining target prescription information and target diagnosis information according to the preset symptom-drug mapping relationship and drug information.

[0127] Specifically, in this embodiment, the dialogue data set is concentrated by information extraction, and the consultation dialogue model extracts the response information generated according to the patient information to be tested to obtain the target symptom information and drug information. The target symptom information includes but is not limited to one symptom, and then the target prescription information and target diagnosis information given by the consultation dialogue model in the dialogue data set are inferred based on the drug information involved in the dialogue information set and the preset symptom-drug mapping relationship.

[0128] It should be noted that, in this embodiment, a guiding text may also be input at the end of the input part of the medical consultation dialogue model so that the medical consultation dialogue model is more inclined to generate sentences that can be used as prescriptions or diagnoses.

[0129] For example, the text "Based on your situation, consider:" is spliced ​​as a historical dialogue to the end of the input of the consultation dialogue model to guide the consultation dialogue model to be more inclined to generate sentences that can be used as diagnoses.

[0130] For another example, the text "It is recommended that you take:" is spliced ​​as a historical dialogue to the end of the input of the medical consultation dialogue model to guide the medical consultation dialogue model to be more inclined to generate sentences that can be used as prescriptions.

[0131] Please refer to Figure 6 In some embodiments of the present application, step S400 includes step S410, step S420, step S430 and step S440. It should be understood that step S400 includes but is not limited to step S410 to step S420. Figure 6 These four steps are explained in detail.

[0132] Step S410, obtaining common symptom information of the original symptom information and the target symptom information;

[0133] Step S420, calculating the original symptom value according to the common symptom information and the original symptom information; the original symptom value is used to represent the proportion of the common symptom information in the original symptom information;

[0134] Step S430, calculating a target symptom value according to the common symptom information and the target symptom information; the target symptom value is used to represent the proportion of the common symptom information in the target symptom information;

[0135] Step S440, calculating the correlation value based on the original symptom value and the target symptom value.

[0136] Specifically, in this embodiment, represents the set of symptoms extracted from the responses generated by the consultation dialogue model, that is, Indicates target symptom information, with Represents the original symptom information corresponding to the patient information to be tested, then the target symptom value and the original symptom value can be calculated by the following formula:

[0137] R= (2)

[0138] P= (3)

[0139] In formula (2) and formula (3), R represents the target symptom value, P represents the original symptom value, and then the associated value is obtained based on the original symptom value calculated by formula (3) and the target symptom value calculated by formula (2).

[0140] Please refer to Figure 7In some embodiments of the present application, step S440 specifically includes step S441, step S442 and step S443. It should be understood that step S440 includes but is not limited to step S441 to step S443. Figure 7 These three steps are explained in detail.

[0141] Step S441, performing a sum operation on the original symptom value and the target symptom value to obtain a symptom sum value;

[0142] Step S442, performing a product operation on the original symptom value and the target symptom value to obtain a symptom product value.

[0143] Step S443, calculating the correlation value according to the symptom sum value and the symptom product value.

[0144] Specifically, in this embodiment, the original symptom value and the target symptom value are first summed to obtain the symptom sum value, then the original symptom value and the target symptom value are multiplied to obtain the symptom multiplication value, and then the correlation value is calculated based on the symptom sum value and the symptom multiplication value. Specifically, it can be expressed by formula (4), which is:

[0145] SCORE_sym= (4)

[0146] In formula (4), SCORE_sym represents the association value, P represents the original symptom value, and R represents the target symptom value.

[0147] In step S500 of some embodiments, the original diagnosis information refers to the diagnosis information corresponding to the patient information to be tested, and the target diagnosis information refers to the diagnosis information output by the consultation dialogue model according to the patient information to be tested. In this embodiment, the diagnosis value can be calculated by the following formula:

[0148] SCORE_diag = accuracy(original diagnostic information, target diagnostic information) (5)

[0149] In formula (5), SCORE_diag represents the diagnostic value, and accuracy() is the function for finding the accuracy value.

[0150] In step S600 of some embodiments, similar to the aforementioned step S500, the original prescription information refers to the prescription information corresponding to the patient information to be tested, and the target prescription information refers to the prescription information output by the consultation dialogue model according to the patient information to be tested. In this embodiment, the prescription value can be calculated by the following formula:

[0151] SCORE_prescibe = accuracy(original prescription information, target prescription information) (6)

[0152] In formula (6), SCORE_prescibe represents the prescription value, and accuracy() is the function for finding the accuracy value.

[0153] Please refer to Figure 8 In some embodiments of the present application, step S700 includes step S710, step S720, step S730, step S740 and step S750. It should be understood that step S700 includes but is not limited to step S710 to step S750. Figure 8 These five steps are introduced in detail.

[0154] Step S710, obtaining a preset association weight, a preset diagnosis weight, and a preset prescription weight;

[0155] Step S720, weighting the association value according to the association weight to obtain an association weight value;

[0156] Step S730, weighting the diagnosis value according to the diagnosis weight to obtain a diagnosis weight value;

[0157] Step S740, weighting the prescription value according to the prescription weight to obtain a prescription weight value;

[0158] Step S750, calculating and summing the associated weight value, the diagnosis weight value and the prescription weight value to obtain a target evaluation value.

[0159] Specifically, in this embodiment, the target evaluation value is calculated by the following formula:

[0160] Score=a1* SCORE_sym+a2* SCORE_diag+a3*SCORE_prescibe (7)

[0161] In formula (7), Score represents the target evaluation value, a1 represents the association weight, SCORE_sym represents the association value, a2 represents the diagnosis weight, SCORE_diag represents the diagnosis value, a3 represents the prescription weight, and SCORE_prescibe represents the prescription value.

[0162] The target evaluation value is calculated by formula (7), thereby realizing the evaluation of the medical consultation dialogue model and improving the accuracy of the medical consultation dialogue model evaluation.

[0163] Please refer to Fig. 9 Some embodiments of the present application also propose an evaluation device for a medical consultation dialogue model, which includes an input module 1100, an acquisition module 1200, an extraction module 1300, an association module 1400, a first calculation module 1500, a second calculation module 1600 and a third calculation module 1700.

[0164] The input module 1100 is used to input the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set.

[0165] The acquisition module 1200 is used to acquire original symptom information corresponding to the patient information to be tested.

[0166] The extraction module 1300 is used to perform information extraction processing on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information.

[0167] The association module 1400 is used to associate the target symptom information with the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information.

[0168] The first calculation module 1500 is used to calculate a diagnosis value according to preset original diagnosis information and target diagnosis information; the diagnosis value is used to represent the accuracy of the target diagnosis information.

[0169] The second calculation module 1600 is used to calculate the prescription value according to the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information.

[0170] The third calculation module 1700 is used to obtain a target evaluation value according to the associated value, the diagnosis value and the prescription value.

[0171] The evaluation device of the medical consultation dialogue model of the embodiment of the present application inputs the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set, and obtains the original symptom information corresponding to the patient information to be tested, then performs information extraction processing on the dialogue data set to obtain the target symptom information, target prescription information and target diagnosis information generated by the medical consultation dialogue model according to the patient information to be tested, and then performs association processing on the target symptom information and the original symptom information to obtain the association value, wherein the association value is used to characterize the association degree between the original symptom information and the target symptom information. In this way, it can be determined whether the main symptoms of the patient information to be tested are all inquired when the medical consultation dialogue model conducts the medical consultation dialogue. Then, according to the preset original diagnosis information and target diagnosis information, the diagnosis value is calculated, wherein the diagnosis value is used to characterize the accuracy of the target diagnosis information; then, according to the preset original prescription information and target prescription information, the prescription value is calculated, wherein the prescription value is used to characterize the accuracy of the target prescription information. Then, according to the association value, the diagnosis value and the prescription value, the target evaluation value is obtained. Through this setting, the medical consultation dialogue model is evaluated from the three dimensions of association value, diagnosis value and prescription value, thereby improving the accuracy of the evaluation of the medical consultation dialogue model.

[0172] It should be noted that the evaluation device of the medical inquiry dialogue model in the embodiment of the present application corresponds to the aforementioned evaluation method of the medical inquiry dialogue model. For the specific processing flow, please refer to the aforementioned evaluation method of the medical inquiry dialogue model, which will not be repeated here.

[0173] The embodiment of the present application also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the evaluation method of the above-mentioned medical consultation dialogue model. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0174] The electronic device of the embodiment of the present application, by executing the evaluation method of the aforementioned consultation dialogue model, obtains a dialogue data set by inputting the patient information to be tested into the consultation dialogue model, and obtains the original symptom information corresponding to the patient information to be tested, then extracts information from the dialogue data set to obtain the target symptom information, target prescription information and target diagnosis information generated by the consultation dialogue model according to the patient information to be tested, and then associates the target symptom information with the original symptom information to obtain an association value, wherein the association value is used to characterize the degree of association between the original symptom information and the target symptom information. In this way, it can be determined whether the main symptoms of the patient information to be tested are all inquired when the consultation dialogue model conducts a consultation dialogue. Then, according to the preset original diagnosis information and target diagnosis information, a diagnosis value is calculated, wherein the diagnosis value is used to characterize the accuracy of the target diagnosis information; then, according to the preset original prescription information and target prescription information, a prescription value is calculated, wherein the prescription value is used to characterize the accuracy of the target prescription information. Then, according to the association value, the diagnosis value and the prescription value, a target evaluation value is obtained. Through this setting, the medical consultation dialogue model is evaluated from three dimensions: association value, diagnosis value and prescription value, thereby improving the accuracy of the medical consultation dialogue model evaluation.

[0175] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0176] The processor 1800 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0177] The memory 1900 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1900 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1900, and the processor 1800 calls and executes the evaluation method of the medical consultation dialogue model in the embodiment of this application;

[0178] Input / output interface 2000, used to implement information input and output;

[0179] Communication interface 2100, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0180] A bus 2200 that transmits information between various components of the device (e.g., the processor 1800, the memory 1900, the input / output interface 2000, and the communication interface 2100);

[0181] The processor 1800 , the memory 1900 , the input / output interface 2000 , and the communication interface 2100 are connected to each other in communication within the device via the bus 2200 .

[0182] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the evaluation method of the above-mentioned medical consultation dialogue model.

[0183] The storage medium of the embodiment of the present application, by executing the evaluation method of the aforementioned consultation dialogue model, obtains a dialogue data set by inputting the patient information to be tested into the consultation dialogue model, and obtains the original symptom information corresponding to the patient information to be tested, then extracts information from the dialogue data set, obtains the target symptom information, target prescription information and target diagnosis information generated by the consultation dialogue model according to the patient information to be tested, and then associates the target symptom information with the original symptom information to obtain an association value, wherein the association value is used to characterize the degree of association between the original symptom information and the target symptom information, and in this way, it can be determined whether the main symptoms of the patient information to be tested are all inquired when the consultation dialogue model conducts a consultation dialogue. Then, according to the preset original diagnosis information and target diagnosis information, a diagnosis value is calculated, wherein the diagnosis value is used to characterize the accuracy of the target diagnosis information; then, according to the preset original prescription information and target prescription information, a prescription value is calculated, wherein the prescription value is used to characterize the accuracy of the target prescription information. Then, according to the association value, the diagnosis value and the prescription value, a target evaluation value is obtained. Through this setting, the medical consultation dialogue model is evaluated from three dimensions: association value, diagnosis value and prescription value, thereby improving the accuracy of the medical consultation dialogue model evaluation.

[0184] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0185] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0186] It can be understood by those skilled in the art that Figure 1-8 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0187] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0189] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0190] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0192] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0195] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for evaluating a medical consultation dialogue model, characterized in that: The method comprises: Input the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set; Obtaining original symptom information corresponding to the patient information to be tested; Performing information extraction processing on the dialogue data set to obtain target symptom information, target prescription information, and target diagnosis information; Associating the target symptom information with the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information; A diagnostic value is calculated based on the preset original diagnostic information and the target diagnostic information; the diagnostic value is used to characterize the accuracy of the target diagnostic information; Calculating a prescription value based on the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information; Obtaining a target evaluation value according to the association value, the diagnosis value and the prescription value; The step of associating the target symptom information with the original symptom information to obtain an associated value includes: Acquire common symptom information of the original symptom information and the target symptom information; An original symptom value is calculated based on the common symptom information and the original symptom information; the original symptom value is used to represent the proportion of the common symptom information in the original symptom information; A target symptom value is calculated based on the common symptom information and the target symptom information; the target symptom value is used to represent the proportion of the common symptom information in the target symptom information; Calculating the correlation value according to the original symptom value and the target symptom value; The step of calculating the correlation value according to the original symptom value and the target symptom value includes: Performing a sum operation on the original symptom value and the target symptom value to obtain a symptom sum value; Performing a product operation on the original symptom value and the target symptom value to obtain a symptom product value; Calculating the correlation value according to the symptom sum value and the symptom product value; Wherein, obtaining a target evaluation value according to the associated value, the diagnostic value and the prescription value comprises: Obtaining a preset association weight, a preset diagnosis weight, and a preset prescription weight; Performing weighted processing on the association value according to the association weight to obtain an association weight value; Performing weighted processing on the diagnosis value according to the diagnosis weight to obtain a diagnosis weight value; Performing weighted processing on the prescription value according to the prescription weight to obtain a prescription weight value; The target evaluation value is obtained by summing up the associated weight value, the diagnostic weight value and the prescription weight value.

2. The method according to claim 1, characterized in that The information extraction process is performed on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information, including: Performing information extraction processing on the conversation data set to obtain the target symptom information and drug information; The target prescription information and the target diagnosis information are obtained according to the preset symptom-drug mapping relationship and the drug information.

3. The method according to any one of claims 1 to 2, characterized in that: Before inputting the patient information to be tested into the medical consultation dialogue model to obtain the dialogue data set, the method further includes: Construct a consultation dialogue model, including: Acquire original training data; wherein the original training data includes: original patient data and original conversation data; Performing enhancement processing on the original conversation data according to the original patient data to obtain conversation enhancement data; The preset original dialogue model is trained according to the dialogue enhancement data and the original patient data to obtain the medical consultation dialogue model.

4. The method according to claim 3, characterized in that The training process of the preset original dialogue model according to the dialogue enhancement data and the original patient data to obtain the consultation dialogue model includes: Obtaining a target loss function of the original dialogue model; Performing feature extraction on the dialogue enhancement data to obtain a dialogue enhancement vector; Performing feature extraction on the original patient data to obtain an original patient vector; Performing encoding processing on the dialogue enhancement vector to obtain dialogue encoding data; Performing encoding processing on the original patient vector to obtain patient encoding data; Decoding the conversation coded data and the patient coded data to obtain target reply information; Calculating the loss value of the target loss function according to the target response information to obtain a target loss value; The parameters of the original dialogue model are updated according to the target loss value to obtain the medical consultation dialogue model.

5. An evaluation device for a medical consultation dialogue model, characterized in that: The device comprises: An input module is used to input the patient information to be tested into the medical consultation dialogue model to obtain a dialogue data set; An acquisition module, used for acquiring original symptom information corresponding to the patient information to be tested; An extraction module, used to perform information extraction processing on the dialogue data set to obtain target symptom information, target prescription information and target diagnosis information; An association module, used for performing association processing on the target symptom information and the original symptom information to obtain an association value; the association value is used to represent the degree of association between the original symptom information and the target symptom information; A first calculation module, used to calculate a diagnosis value according to preset original diagnosis information and the target diagnosis information; the diagnosis value is used to characterize the accuracy of the target diagnosis information; A second calculation module is used to calculate a prescription value based on the preset original prescription information and the target prescription information; the prescription value is used to characterize the accuracy of the target prescription information; A third calculation module, used for obtaining a target evaluation value according to the association value, the diagnosis value and the prescription value; The association module is used to associate the target symptom information with the original symptom information to obtain an association value, including: Acquire common symptom information of the original symptom information and the target symptom information; An original symptom value is calculated based on the common symptom information and the original symptom information; the original symptom value is used to represent the proportion of the common symptom information in the original symptom information; A target symptom value is calculated based on the common symptom information and the target symptom information; the target symptom value is used to represent the proportion of the common symptom information in the target symptom information; Calculating the correlation value according to the original symptom value and the target symptom value; The step of calculating the correlation value according to the original symptom value and the target symptom value includes: Performing a sum operation on the original symptom value and the target symptom value to obtain a symptom sum value; Performing a product operation on the original symptom value and the target symptom value to obtain a symptom product value; Calculating the correlation value according to the symptom sum value and the symptom product value; The third calculation module is used to obtain a target evaluation value according to the association value, the diagnosis value and the prescription value, including: Obtaining a preset association weight, a preset diagnosis weight, and a preset prescription weight; Performing weighted processing on the association value according to the association weight to obtain an association weight value; Performing weighted processing on the diagnosis value according to the diagnosis weight to obtain a diagnosis weight value; Performing weighted processing on the prescription value according to the prescription weight to obtain a prescription weight value; The target evaluation value is obtained by summing up the associated weight value, the diagnostic weight value and the prescription weight value.

6. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the evaluation method of the medical consultation dialogue model as described in any one of claims 1 to 4 are implemented.

7. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the evaluation method of the medical consultation dialogue model described in any one of claims 1 to 4.