Simulated inquiry interaction analysis method, device and equipment based on man-machine conversation

Abstract

The invention discloses a simulated inquiry interaction analysis method, device and equipment based on man-machine conversation, and relates to the technical field of artificial intelligence and medical treatment, and the method comprises the steps: obtaining inquiry conversation data generated by interaction between a user object and a virtual user; performing multi-dimensional interaction analysis on the inquiry dialogue data based on a preset medical knowledge graph and preset diagnosis information to generate a corresponding evaluation index and a corresponding index score; and generating and outputting an evaluation report containing the index score and the user object-oriented inquiry dialogue improvement suggestion. According to the method, the objectivity and the accuracy of an evaluation result are ensured, prejudice and inconsistency caused by subjective experience difference in manual evaluation are overcome, the evaluation result is accurate, and a user can clearly know the expression level of the user in each dimension and can also obtain a targeted inquiry optimization direction.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical technology, and in particular to a method, apparatus, and equipment for simulated medical consultation interaction analysis based on human-computer dialogue. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI-based simulated consultation training systems are increasingly being used in medical education and training. In these systems, users can log in and act as doctors, engaging in consultations with virtual patients to improve their diagnostic skills. However, there is currently a lack of effective methods to accurately evaluate the effectiveness of these consultations. Traditional evaluation methods often rely on manual assessment, which is inefficient and highly subjective. Furthermore, existing technologies struggle to comprehensively and objectively quantify the content of conversations, diagnostic accuracy, and procedural standardization during the consultation process from multiple dimensions. Currently, there is a lack of an automated and efficient method to perform multi-dimensional quantitative analysis of the simulated consultation process and provide actionable improvement suggestions, resulting in low efficiency and poor effectiveness in consultation skills training. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, apparatus, and device for analyzing simulated medical consultations based on human-computer dialogue, capable of automatically and efficiently performing multi-dimensional quantitative analysis of the simulated medical consultation process, and providing methods with operable improvement suggestions. The specific solution is as follows: Firstly, this application discloses a simulated medical consultation interaction analysis method based on human-computer dialogue, including: Acquire consultation dialogue data generated from interactions between user objects and virtual users; Based on a preset medical knowledge graph and preset diagnostic information, the consultation dialogue data is subjected to multi-dimensional interactive analysis to generate corresponding evaluation indicators and corresponding indicator scores; wherein, the multi-dimensional interactive analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators of the consultation dialogue data based on the preset medical knowledge graph, matching analysis of the simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data. Generate and output an evaluation report that includes the scores of the aforementioned metrics and suggestions for improving the consultation dialogue for the aforementioned user group.

[0004] Optionally, the step of performing multi-dimensional interactive analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores includes: Extract the target medical entity queried by the user from the consultation dialogue data, and match and cover the target medical entity with the dialogue information set associated with the target disease in the preset medical knowledge graph to obtain the consultation completeness index and the corresponding first index score. The simulated diagnosis results output by the user object are extracted from the consultation dialogue data. The simulated diagnosis results are compared with the preset diagnosis information of the virtual user through a preset semantic similarity calculation model to obtain the diagnosis accuracy index and the corresponding second index score. Based on a preset dialogue state transition model, the dialogue sequence of the user object asking questions in the consultation dialogue data is verified and the deviation is analyzed to obtain the dialogue fluency index and the corresponding third index score.

[0005] Optionally, the step of matching and covering the target medical entity with the dialogue information set associated with the target disease in a preset medical knowledge graph to obtain a consultation completeness index and the corresponding first index score includes: The target medical entity is identified from a preset medical knowledge graph as having a set of target dialogue information associated with the target disease. Calculate the coverage rate of the target medical entity in the target dialogue information set to obtain the consultation completeness index; The consultation completeness index is scored using a preset dialogue scoring model to obtain the corresponding first index score.

[0006] Optionally, the step of comparing the simulated diagnostic results with the virtual user's preset diagnostic information using a preset semantic similarity calculation model to obtain a diagnostic accuracy index and a corresponding second index score includes: The simulated diagnostic results and the virtual user's preset diagnostic information are vectorized using a preset semantic similarity calculation model to obtain a first vector and a second vector, respectively. The cosine similarity value between the first vector and the second vector is calculated based on a preset cosine similarity algorithm to obtain a diagnostic accuracy index. The diagnostic accuracy index is scored using a preset dialogue scoring model to obtain the corresponding second index score.

[0007] Optionally, the step of verifying and analyzing the dialogue sequence of the user's questions in the consultation dialogue data based on a preset dialogue state transition model to obtain a dialogue fluency index and a corresponding third index score includes: The consultation dialogue data is judged based on a preset dialogue state transition model to determine the current dialogue state of the consultation dialogue data, so as to obtain the dialogue fluency index. Check whether the state transition between the current dialogue state and the previous dialogue state meets the preset dialogue state transition conditions. If not, the weight of the third indicator score corresponding to the dialogue fluency indicator is adjusted to obtain the adjusted weight information. The dialogue fluency index is scored using a preset dialogue scoring model and the adjusted weight information to obtain the corresponding third index score.

[0008] Optionally, the simulated medical consultation interaction analysis method based on human-computer dialogue further includes: Obtain a historical consultation dialogue sample set; wherein, the historical consultation dialogue sample set includes historical consultation dialogue data between multiple sets of user objects and virtual users, and preset medical condition information of the virtual users corresponding to each historical consultation dialogue data; Labeling and scoring are performed on each of the historical consultation dialogue data in the historical consultation dialogue sample set to generate a training dataset; wherein, the labeling includes at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index, and the scoring includes the actual index score for at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index; The training dataset carrying corresponding labels and real index scores is input into the initial machine learning model for training, so that the initial machine learning model learns the correlation between each historical consultation dialogue data and the corresponding labels and real index scores, and outputs the corresponding predicted labels and predicted index scores. The error between the predicted index score and the actual index score is calculated using the backpropagation algorithm. The model parameters of the initial machine learning model are then iteratively adjusted until the machine learning model meets the preset iteration termination condition, thereby obtaining the preset dialogue scoring model.

[0009] Optionally, after obtaining the consultation dialogue data generated by the interaction between the user object and the virtual user, the method further includes: Natural language processing is performed on the consultation dialogue data to obtain structured semantic information; Accordingly, the multi-dimensional interactive analysis of the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores includes: Based on a preset medical knowledge graph and preset diagnostic information, the structured semantic information is subjected to multi-dimensional interactive analysis to generate corresponding evaluation indicators and corresponding indicator scores.

[0010] Secondly, this application discloses a simulated medical consultation interaction analysis device based on human-computer dialogue, comprising: The data acquisition module is used to acquire consultation dialogue data generated by the interaction between the user object and the virtual user; An interactive analysis module is used to perform multi-dimensional interactive analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores. The multi-dimensional interactive analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators based on the preset medical knowledge graph, matching analysis of simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data. The report generation module is used to generate and output an evaluation report that includes the scores of the aforementioned indicators and suggestions for improving the consultation dialogue for the aforementioned user.

[0011] Thirdly, this application discloses an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the aforementioned disclosed simulated medical consultation interaction analysis method based on human-computer dialogue.

[0012] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed simulated medical consultation interaction analysis method based on human-computer dialogue.

[0013] As can be seen, this application discloses a method for acquiring consultation dialogue data generated by interactions between a user and a virtual user; performing multi-dimensional interaction analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores; wherein, the multi-dimensional interaction analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators based on the preset medical knowledge graph, matching analysis of simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data; generating and outputting an evaluation report containing the indicator scores and consultation dialogue improvement suggestions for the user. Therefore, by using a pre-set medical knowledge graph and pre-set diagnostic information as the analytical benchmark, the objectivity and accuracy of the evaluation results are ensured. Through multi-dimensional quantitative analysis of the completeness of the consultation process, the matching degree of diagnostic results, and the fluency of the dialogue, the bias and inconsistency caused by subjective experience differences in human evaluation are overcome, making the evaluation results accurate. An evaluation report containing specific indicator scores and improvement suggestions is generated and output, so that users can not only clearly understand their own performance level in various dimensions, but also obtain targeted consultation optimization directions. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This application discloses a flowchart of a simulated medical consultation interaction analysis method based on human-computer dialogue. Figure 2 This application discloses a specific flowchart of a simulated medical consultation interaction analysis method based on human-computer dialogue. Figure 3 This is a schematic diagram of the structure of a simulated medical consultation interaction analysis device based on human-computer dialogue disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0017] With the rapid development of artificial intelligence technology, AI-simulated consultation training systems are gradually being adopted in medical education and training. In these systems, users can log in and act as doctors, engaging in consultations with virtual patients to improve the consultation skills of medical professionals. However, there is currently a lack of effective means to accurately evaluate the effectiveness of consultations. Traditional evaluation methods often rely on manual assessment, which is inefficient and highly subjective. Furthermore, existing technologies struggle to comprehensively and objectively quantify the content of conversations, diagnostic accuracy, and procedural standardization during the consultation process from multiple dimensions. Currently, there is a lack of a method that can automatically and efficiently perform multi-dimensional quantitative analysis of simulated consultation processes and provide actionable improvement suggestions, resulting in low efficiency and poor effectiveness in consultation skills training.

[0018] To this end, the present invention provides a simulated medical consultation interaction analysis scheme based on human-computer dialogue, which can automatically and efficiently perform multi-dimensional quantitative analysis of the simulated medical consultation process and provide a method with operable improvement suggestions.

[0019] Reference Figure 1 As shown, this embodiment of the invention discloses a simulated medical consultation interaction analysis method based on human-computer dialogue, including: Step S11: Obtain the consultation dialogue data generated from the interaction between the user object and the virtual user.

[0020] In this embodiment, during the consultation dialogue between the user object and the virtual user, the dialogue text data is collected in real time, including the questions of the doctor (user object) and the answers of the virtual patient (virtual user), etc., to obtain the consultation dialogue data. At the same time, relevant auxiliary information is collected, such as the preset disease condition information of the virtual patient, the diagnostic criteria, the relevant examination report information, etc.

[0021] In this embodiment, natural language processing is performed on the consultation dialogue data to obtain structured semantic information; it can be understood that the collected dialogue text data is preprocessed, including operations such as word segmentation, stop word removal, and词性标注 (pos tagging). For example, use a natural language processing toolkit to process the text. The sentence "Recently coughing and having an uninterrupted fever for a month" is word-segmented and stop words are removed, finally obtaining ["Recently", "cough", "fever", "last", "a month", "symptom"], and the词性标注 (pos tagging) is performed on ["Recently", "cough", "fever", "last", "a month", "symptom"] to obtain structured semantic information.

[0022] Specifically, the dialogue between the user and the virtual patient is monitored in real time. When the user inputs a question, the question content and related information such as the time stamp are recorded; when the virtual patient gives an answer, its answer content and related information are also recorded to obtain the consultation dialogue data. At the same time, the preset disease condition information of the virtual patient, including the disease name, symptom manifestations, medical history, etc., is obtained from the system's case database. Further, the collected dialogue text is preprocessed using natural language processing tools. For example, use the word segmentation function of the NLTK toolkit to segment the text. The sentence "The patient has a headache, nausea, and vomiting" is segmented into ["patient", "headache", "nausea", "vomiting"]. Then, stop words in the text, such as meaningless words like "of", "already", "in", etc., are removed. Next, a词性标注 (pos tagging) tool is used to perform词性标注 (pos tagging) on the segmented words to label each word as a noun, verb, adjective, etc.

[0023] Step S12: Perform multi-dimensional interaction analysis on the consultation dialogue data based on the preset medical knowledge graph and the preset diagnosis information to generate corresponding evaluation indicators and corresponding indicator scores; where the multi-dimensional interaction analysis includes matching analysis of the consultation dialogue process integrity evaluation indicator of the consultation dialogue data based on the preset medical knowledge graph, matching analysis of the simulated diagnosis result in the consultation dialogue data based on the preset diagnosis information, and dialogue fluency analysis based on the context dialogue state in the consultation dialogue data.

[0024] It should be noted that the term "词性标注 (pos tagging)" in the original text seems to be a Chinese term that should be translated into the appropriate English term for "词性标注" in the context. Here, I left it in Chinese because the English term is not provided in the original text. If you have the correct English term for it, you can replace it in the translation.In this embodiment, multi-dimensional interactive analysis is performed on the structured semantic information based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and their scores. It can be understood that by performing multi-dimensional interactive analysis on the structured semantic information, the corresponding scores for three indicators are obtained. Specifically, the three indicators and their corresponding scores are: the consultation completeness indicator and its corresponding first indicator score, the diagnostic accuracy indicator and its corresponding second indicator score, and the dialogue fluency indicator and its corresponding third indicator score.

[0025] In this embodiment, the target medical entity queried by the user is extracted from the consultation dialogue data. The target medical entity is then matched and its coverage is calculated against the dialogue information set associated with the target disease in a preset medical knowledge graph to obtain a consultation completeness index and a corresponding first index score. Specifically, the target medical entity is used to determine the target dialogue information set associated with the target disease from the preset medical knowledge graph. The coverage rate of the target medical entity in the target dialogue information set is calculated to obtain the consultation completeness index. The consultation completeness index is scored using a preset dialogue scoring model to obtain the corresponding first index score. It is understood that, based on the medical knowledge graph, it determines whether the doctor has inquired about key information related to the virtual patient's condition, such as the duration, severity, and triggering factors of symptoms. For example, when a user asks, "Do you have a fever?" the structured semantic information is [fever]. When a user asks, "How long have you been coughing?" the structured semantic information is [cough, duration]. When a user asks, "Do you have phlegm? What color is it?" the structured semantic information is [coughing up phlegm, phlegm color]. Furthermore, the extracted list of user medical entities [fever, cough, duration, sputum, sputum color] is matched with the complete set of key information in the knowledge graph. The target medical entity is each item in the user's medical entity list. The system then matches the extracted list of user medical entities [fever, cough, duration, sputum, sputum color] with the target dialogue information set in the medical knowledge graph. This target dialogue information set is generated based on clinical guidelines or standard consultation paths corresponding to the target disease. For example, when the target disease is determined to be community-acquired pneumonia, the target dialogue information set obtained from the medical knowledge graph would be [chest pain, dyspnea, highest fever, smoking history, imaging examinations, fever, cough, duration, sputum, sputum color].

[0026] In addition, a medical knowledge graph is constructed, which contains the relationships between various diseases and related symptoms, medical history, examination items, etc. For example, for the common cold, the symptoms associated in the knowledge graph may include fever, cough, runny nose, etc., and the medical history may include whether the patient has been exposed to cold recently.

[0027] In this way, when a doctor asks a question, the system matches the question content with the knowledge graph to determine whether it involves key information. For example, if a doctor asks, "Does the patient have a fever?", by matching the knowledge graph, it is determined that the question involves key symptom information related to a cold. By traversing all the doctor's questions, the system calculates the ratio of the number of key information actually asked to the total number of key information questions, and obtains the first indicator score.

[0028] In one specific implementation, the construction of a medical knowledge graph can digitize standard clinical guidelines and other materials, and structure the diagnostic criteria and essential consultation points within them. These structured points are then mapped to corresponding nodes in the medical knowledge graph, forming a set of standard consultation templates. When a target disease, such as community-acquired pneumonia, is identified, to obtain the target dialogue information set, the standard consultation template associated with that disease is automatically retrieved from the medical knowledge graph. All the information nodes contained in this template constitute the target dialogue information set.

[0029] In another specific implementation, the target dialogue information set is formed by traversing the medical knowledge graph and extracting all entities that are directly related to the target disease node and whose relationship type is symptom-related, has physical signs, or requires examination. Specifically, starting with the disease node "community-acquired pneumonia," the graph query language is used to find all entities directly connected to this node whose relationship edges are of a preset key relationship type (symptom-related). The set of all these entities constitutes the target dialogue information set.

[0030] Therefore, the coverage rate of the dialogue information set associated with the target disease in the preset medical knowledge graph is calculated based on the first indicator score calculation formula. The first indicator score calculation formula is as follows: The first indicator score = (number of target medical entities / number of target dialogue information sets) × 100%.

[0031] For example, in the above analysis scenario, if the number of target medical entities is 5 and the number of target dialogue information sets is 10, then the score for the first indicator is 50%.

[0032] This shows that for virtual patients, if the doctor does not obtain entity information from the target dialogue information set in the medical knowledge graph, the score of the first indicator decreases.

[0033] In this embodiment, the simulated diagnostic results output by the user are extracted from the consultation dialogue data. A preset semantic similarity calculation model is used to compare the simulated diagnostic results with the preset diagnostic information of the virtual user to obtain a diagnostic accuracy index and a corresponding second index score. Specifically, the preset semantic similarity calculation model is used to perform vector transformation on the simulated diagnostic results and the preset diagnostic information of the virtual user to obtain a first vector and a second vector, respectively. A preset cosine similarity algorithm is used to calculate the cosine similarity value between the first vector and the second vector to obtain the diagnostic accuracy index. A preset dialogue scoring model is used to score the diagnostic accuracy index to obtain the corresponding second index score. It can be understood that the simulated diagnostic results finally given by the doctor are compared with the preset diagnostic information of the virtual patient. A semantic similarity algorithm is used to calculate the similarity between the two. For example, if the doctor diagnoses pneumonia, and the correct diagnosis is bacterial pneumonia, the degree of similarity between the two can be obtained through semantic similarity calculation, and the second index score = semantic similarity value × 100%.

[0034] After the doctor provides a diagnosis, the simulated diagnosis is compared with the preset diagnosis information for the virtual patient. For example, the doctor diagnoses "gastritis," while the preset diagnosis information is "gastric ulcer."

[0035] The cosine similarity algorithm is used to calculate the similarity between the two. First, the diagnosis results are converted into vector representations. For example, "gastritis" and "gastric ulcer" are converted into a first vector and a second vector, respectively, using a word vector model (Word2Vec). Then, the cosine similarity value between the two vectors is calculated. Assuming the calculated semantic similarity value is 0.6, the second index score = 0.6 × 100% = 60%.

[0036] In addition, the edit distance algorithm can also be used when calculating semantic similarity values, which will not be elaborated further.

[0037] In this embodiment, the dialogue sequence of questions asked by the user in the consultation dialogue data is verified and analyzed for deviation based on a preset dialogue state transition model to obtain a dialogue fluency index and a corresponding third indicator score. Specifically, the consultation dialogue data is judged based on the preset dialogue state transition model to determine the current dialogue state of the consultation dialogue data, thereby obtaining a dialogue fluency index; it is checked whether the state transition between the current dialogue state and the previous dialogue state meets the preset dialogue state transition conditions; if not, the weight of the third indicator score corresponding to the dialogue fluency index is adjusted to obtain the adjusted weight information; the dialogue fluency index is scored using a preset dialogue scoring model and the adjusted weight information to obtain the corresponding third indicator score. It is understood that this involves analyzing whether the logical order of the doctor's questions is reasonable and whether there are repeated or skipped questions. By constructing a dialogue state transition model, it is determined whether the question sequence conforms to a common consultation process. For example, the normal process is to first inquire about symptoms and then about medical history. If the doctor inquires about family medical history before inquiring about symptoms, the dialogue fluency is affected, thereby reducing the corresponding weight parameters. The dialogue fluency score is calculated based on the degree of matching of the dialogue state transition model to obtain the third indicator score.

[0038] Specifically, a dialogue state transition model is constructed, which defines common consultation process states, such as asking about symptoms, asking about medical history, asking about family history, and making a preliminary diagnosis, as well as the legal transition relationships between each state. For example, one can transition from asking about symptoms to asking about medical history, but it is generally not allowed to directly transition from asking about family history to making a preliminary diagnosis.

[0039] During the doctor's questioning process, the system determines the current dialogue state based on the question content and checks whether the state transition conforms to the model definition. For example, if the doctor first asks, "What symptoms does the patient have?", the current dialogue state is determined to be in the symptom inquiry state. Then, if the doctor asks, "Has the patient had similar illnesses before?", the system determines that the transition is from the symptom inquiry state to the medical history inquiry state. This transition conforms to the model definition, and the dialogue fluency score is not affected. If a transition that does not conform to the model definition occurs, the score of the third indicator is reduced.

[0040] In this embodiment, a historical consultation dialogue sample set is obtained; wherein, the historical consultation dialogue sample set contains historical consultation dialogue data between multiple sets of user objects and virtual users, and preset medical condition information of the virtual users corresponding to each historical consultation dialogue data; each historical consultation dialogue data in the historical consultation dialogue sample set is labeled and scored to generate a training dataset; wherein, the labeling includes at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index, and the scoring includes the actual index score for at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index; the training dataset carrying the corresponding labels and actual index scores is input into an initial machine learning model for training, so that the initial machine learning model learns the correlation between each historical consultation dialogue data and the corresponding labels and actual index scores, and outputs the corresponding predicted labels and predicted index scores; the error value between the predicted index score and the actual index score is calculated through the backpropagation algorithm to iteratively adjust the model parameters of the initial machine learning model until the machine learning model meets the preset iteration termination condition to obtain a preset dialogue scoring model. Understandably, a large number of online consultation dialogue samples are collected and manually annotated by professional doctors. The annotations include evaluation results on aspects such as consultation completeness, diagnostic accuracy, and dialogue fluency, forming a training dataset. It's important to note that the machine learning algorithms used in building and training the dialogue scoring model include neural networks, decision trees, or support vector machines; there are no specific limitations. When using machine learning algorithms to train the dataset and build the dialogue scoring model, a multilayer perceptron (MLP) can be used as an example. The input is preprocessed dialogue text features and related auxiliary information, and the output is the score for each evaluation indicator. The model parameters are continuously adjusted through backpropagation to minimize the error between the model's predictions and the manually annotated results.

[0041] Specifically, a large number of online consultation dialogue samples were collected, drawn from consultations between different users and virtual patients. Professional doctors were invited to manually annotate each sample, including scores for consultation completeness, diagnostic accuracy, and dialogue fluency. For example, for a single consultation dialogue sample, a professional doctor, based on their experience and expertise, would assign it a score of 80 for the first indicator, 60 for the second, and 70 for the third.

[0042] These labeled samples are organized into a training dataset. Each data point in the dataset contains preprocessed dialogue text features, preset medical information of the virtual patient, and manually labeled evaluation scores. Taking a neural network as an example, the training dataset is input into a Multilayer Perceptron (MLP) model for training. The input layer of the MLP model receives dialogue text features and auxiliary information, which are then processed through multiple hidden layers. The output layer outputs the predicted scores for each evaluation metric. During training, the backpropagation algorithm is used to continuously adjust the model's weights and bias parameters based on the error between the model's predicted scores and the manually labeled scores, gradually reducing the error. After multiple iterations of training, when the model's performance on the validation dataset reaches a certain requirement, training ends, resulting in the trained evaluation model, i.e., the preset dialogue scoring model.

[0043] Furthermore, after the user completes a consultation with the virtual patient, the pre-processed dialogue data and the virtual patient's pre-defined medical condition information are input into a pre-trained dialogue scoring model. The model calculates scores for each evaluation indicator based on the input data. For example, the pre-defined dialogue scoring model outputs a score of 85 for the first indicator, 70 for the second, and 75 for the third.

[0044] The overall evaluation score is calculated based on the preset weights. If weight 1 = 0.4, weight 2 = 0.3, and weight 3 = 0.3, then the overall evaluation score is 85 × 0.4 + 70 × 0.3 + 75 × 0.3 = 77.5 points.

[0045] Step S13: Generate and output an evaluation report containing the scores of the aforementioned indicators and suggestions for improving the consultation dialogue for the aforementioned user.

[0046] In this embodiment, the scores for each evaluation indicator are first listed, followed by a detailed analysis of each indicator, highlighting its strengths and weaknesses. For example, regarding the completeness of the consultation, it is pointed out that while the doctor inquired about most of the key symptom information, factors aggravating the symptoms were overlooked. Regarding diagnostic accuracy, the analysis suggests that the doctor's diagnostic errors might be due to an inaccurate understanding of certain symptoms. Finally, improvement suggestions are given based on the analysis results, such as recommending that doctors strengthen their knowledge of symptom aggravation factors and improve their symptom analysis abilities. The evaluation report is presented to the user in a visual manner, allowing the user to view their consultation performance evaluation through the system interface, thus enabling targeted learning and improvement.

[0047] As can be seen, this application discloses a method for acquiring consultation dialogue data generated by interactions between a user and a virtual user; performing multi-dimensional interaction analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores; wherein, the multi-dimensional interaction analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators based on the preset medical knowledge graph, matching analysis of simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data; generating and outputting an evaluation report containing the indicator scores and consultation dialogue improvement suggestions for the user. Therefore, by using a pre-set medical knowledge graph and pre-set diagnostic information as the analytical benchmark, the objectivity and accuracy of the evaluation results are ensured. Through multi-dimensional quantitative analysis of the completeness of the consultation process, the matching degree of diagnostic results, and the fluency of the dialogue, the bias and inconsistency caused by subjective experience differences in human evaluation are overcome, making the evaluation results accurate. An evaluation report containing specific indicator scores and improvement suggestions is generated and output, so that users can not only clearly understand their own performance level in various dimensions, but also obtain targeted consultation optimization directions.

[0048] like Figure 2 As shown, Figure 2 A specific method for evaluating the consultation effect of an AI-simulated consultation training system is disclosed, including the following steps: 1) Collect real-time consultation dialogue text data and related auxiliary information between users and virtual patients, and preprocess the dialogue text data.

[0049] 2) The preprocessed dialogue text data is evaluated based on multi-dimensional evaluation indicators to obtain the consultation integrity score, diagnosis accuracy score, and dialogue fluency score.

[0050] 3) The above scores are processed using the trained evaluation model (dialogue scoring model) to calculate the overall evaluation score; wherein, the evaluation model is obtained by scoring training and fine-tuning using a training set containing dialogue sample 1, dialogue sample 2 and dialogue sample 3. The specific training process is described in the aforementioned disclosed embodiments and will not be repeated here.

[0051] 4) Generate an evaluation report that includes scores, strengths and weaknesses analysis, and improvement suggestions, and provide feedback to the user.

[0052] In this way, the above-mentioned methods and processes enable automated evaluation of consultation effectiveness, allowing for comprehensive and objective quantitative analysis of the consultation process, thus overcoming the shortcomings of traditional manual evaluation which is highly subjective.

[0053] Reference Figure 3As shown, this embodiment of the invention discloses a simulated medical consultation interaction analysis device based on human-computer dialogue, comprising: Data acquisition module 11 is used to acquire consultation dialogue data generated by the interaction between the user object and the virtual user; The interaction analysis module 12 is used to perform multi-dimensional interaction analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information, so as to generate corresponding evaluation indicators and corresponding indicator scores; wherein, the multi-dimensional interaction analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators of the consultation dialogue data based on the preset medical knowledge graph, matching analysis of the simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data. The report generation module 13 is used to generate and output an evaluation report that includes the scores of the indicators and suggestions for improving the consultation dialogue for the user.

[0054] Therefore, this device can automatically evaluate the effectiveness of consultations, greatly improving evaluation efficiency and reducing the workload of manual evaluation. Through multi-dimensional evaluation indicators, it can comprehensively and objectively quantify and analyze the consultation process, overcoming the shortcomings of traditional manual evaluation which is highly subjective, and improving the accuracy and reliability of the evaluation. It provides users with detailed evaluation reports and improvement suggestions, helping users to quickly improve their consultation skills.

[0055] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0056] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the simulated medical consultation interaction analysis method based on human-computer dialogue disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0057] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0058] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0059] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0060] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the human-computer dialogue-based simulated consultation interaction analysis method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0061] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed simulated medical consultation interaction analysis method based on human-computer dialogue. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0063] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (Compact Disc-Read Only Memory), or any other form of storage medium known in the art.

[0064] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A simulated medical consultation interaction analysis method based on human-computer dialogue, characterized in that, include: Acquire consultation dialogue data generated from interactions between user objects and virtual users; Based on a preset medical knowledge graph and preset diagnostic information, the consultation dialogue data is subjected to multi-dimensional interactive analysis to generate corresponding evaluation indicators and corresponding indicator scores; wherein, the multi-dimensional interactive analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators of the consultation dialogue data based on the preset medical knowledge graph, matching analysis of the simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data. Generate and output an evaluation report that includes the scores of the aforementioned metrics and suggestions for improving the consultation dialogue for the aforementioned user group.

2. The simulated medical consultation interaction analysis method based on human-computer dialogue according to claim 1, characterized in that, The process involves multi-dimensional interactive analysis of the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and scores, including: Extract the target medical entity queried by the user from the consultation dialogue data, and match and cover the target medical entity with the dialogue information set associated with the target disease in the preset medical knowledge graph to obtain the consultation completeness index and the corresponding first index score. The simulated diagnosis results output by the user object are extracted from the consultation dialogue data. The simulated diagnosis results are compared with the preset diagnosis information of the virtual user through a preset semantic similarity calculation model to obtain the diagnosis accuracy index and the corresponding second index score. Based on a preset dialogue state transition model, the dialogue sequence of the user object asking questions in the consultation dialogue data is verified and the deviation is analyzed to obtain the dialogue fluency index and the corresponding third index score.

3. The simulated medical consultation interaction analysis method based on human-computer dialogue according to claim 2, characterized in that, The step of matching and covering the target medical entity with the dialogue information set associated with the target disease in the preset medical knowledge graph to obtain the consultation completeness index and the corresponding first index score includes: The target medical entity is identified from a preset medical knowledge graph as having a set of target dialogue information associated with the target disease. Calculate the coverage rate of the target medical entity in the target dialogue information set to obtain the consultation completeness index; The consultation completeness index is scored using a preset dialogue scoring model to obtain the corresponding first index score.

4. The simulated medical consultation interaction analysis method based on human-computer dialogue according to claim 2, characterized in that, The step of comparing the simulated diagnostic results with the virtual user's preset diagnostic information using a preset semantic similarity calculation model to obtain a diagnostic accuracy index and a corresponding second index score includes: The simulated diagnostic results and the virtual user's preset diagnostic information are vectorized using a preset semantic similarity calculation model to obtain a first vector and a second vector, respectively. The cosine similarity value between the first vector and the second vector is calculated based on a preset cosine similarity algorithm to obtain a diagnostic accuracy index. The diagnostic accuracy index is scored using a preset dialogue scoring model to obtain the corresponding second index score.

5. The simulated medical consultation interaction analysis method based on human-computer dialogue according to claim 2, characterized in that, The process of verifying and analyzing the dialogue sequence of user questions in the consultation dialogue data based on a preset dialogue state transition model to obtain a dialogue fluency index and a corresponding third index score includes: The consultation dialogue data is judged based on a preset dialogue state transition model to determine the current dialogue state of the consultation dialogue data, so as to obtain the dialogue fluency index. Check whether the state transition between the current dialogue state and the previous dialogue state meets the preset dialogue state transition conditions. If not, the weight of the third indicator score corresponding to the dialogue fluency indicator is adjusted to obtain the adjusted weight information. The dialogue fluency index is scored using a preset dialogue scoring model and the adjusted weight information to obtain the corresponding third index score.

6. The simulated medical consultation interaction analysis method based on human-computer dialogue according to any one of claims 3 to 5, characterized in that, Also includes: Obtain a historical consultation dialogue sample set; wherein, the historical consultation dialogue sample set includes historical consultation dialogue data between multiple sets of user objects and virtual users, and preset medical condition information of the virtual users corresponding to each historical consultation dialogue data; Labeling and scoring are performed on each of the historical consultation dialogue data in the historical consultation dialogue sample set to generate a training dataset; wherein, the labeling includes at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index, and the scoring includes the actual index score for at least one of consultation completeness index, diagnostic accuracy index, and dialogue fluency index; The training dataset carrying corresponding labels and real index scores is input into the initial machine learning model for training, so that the initial machine learning model learns the correlation between each historical consultation dialogue data and the corresponding labels and real index scores, and outputs the corresponding predicted labels and predicted index scores. The error between the predicted index score and the actual index score is calculated using the backpropagation algorithm. The model parameters of the initial machine learning model are then iteratively adjusted until the machine learning model meets the preset iteration termination condition, thereby obtaining the preset dialogue scoring model.

7. The simulated medical consultation interaction analysis method based on human-computer dialogue according to claim 1, characterized in that, After obtaining the consultation dialogue data generated by the interaction between the user object and the virtual user, the method further includes: Natural language processing is performed on the consultation dialogue data to obtain structured semantic information; Accordingly, the multi-dimensional interactive analysis of the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores includes: Based on a preset medical knowledge graph and preset diagnostic information, the structured semantic information is subjected to multi-dimensional interactive analysis to generate corresponding evaluation indicators and corresponding indicator scores.

8. A simulated medical consultation interaction analysis device based on human-computer dialogue, characterized in that, include: The data acquisition module is used to acquire consultation dialogue data generated by the interaction between the user object and the virtual user; An interactive analysis module is used to perform multi-dimensional interactive analysis on the consultation dialogue data based on a preset medical knowledge graph and preset diagnostic information to generate corresponding evaluation indicators and corresponding indicator scores. The multi-dimensional interactive analysis includes matching analysis of the consultation dialogue process completeness evaluation indicators based on the preset medical knowledge graph, matching analysis of simulated diagnostic results in the consultation dialogue data based on the preset diagnostic information, and dialogue fluency analysis based on the contextual dialogue state in the consultation dialogue data. The report generation module is used to generate and output an evaluation report that includes the scores of the aforementioned indicators and suggestions for improving the consultation dialogue for the aforementioned user.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the simulated consultation interaction analysis method based on human-computer dialogue as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the steps of the simulated consultation interaction analysis method based on human-computer dialogue as described in any one of claims 1 to 7.

Images