Method for generating depression diagnosis dialogue dataset, electronic device and storage medium

Abstract

An embodiment of the present invention provides a method for generating a depression diagnosis dialogue dataset. The method includes: initiating a dialogue with a real population including depression patients based on the clinical criteria for depression diagnosis to generate a patient portrait of the real population; simulating the depressive outpatient dialogue records between the patient role and the doctor role based on the patient portrait; supervising and filtering the depressive outpatient dialogue records to obtain a depression diagnosis dialogue dataset that meets the clinical criteria. The depression diagnosis dialogue dataset collected in the embodiments of the present invention is of great significance for studying the end-to-end depression consultation dialogue and symptom-based depression diagnosis. The dialogue system also plays a very important role in the large-scale screening and follow-up of depression, and can effectively help alleviate the problems caused by the imbalance of medical resources.

Description

Technical Field

[0001] The present invention relates to the field of intelligent voice, and particularly to a method for generating a depression diagnosis dialogue dataset, an electronic device, and a storage medium. Background Art

[0002] The impact of depression is extensive and has become a major threat to the global life expectancy. Therefore, the automatic diagnosis method of depression has become a new research hotspot. Through dialogue, the emotional and cognitive states of patients can be better understood from an objective perspective, which is closely related to the diagnosis of depression, and at the same time, emotional support can be provided. In addition, the dialogue system can provide a generalizable carrier for diagnosis methods based on text, voice, and multi-modal. Through detailed dialogue annotation, the interpretability of diagnosis can be effectively enhanced. Therefore, the dialogue system has important value for the large-scale screening of depression and the regular follow-up of depression patients.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the related technologies:

[0004] Clinical diagnosis is a complex process, the purpose of which is to collect and summarize the key symptom information of a patient, and at the same time provide a chat-like dialogue experience. In clinical practice, psychiatrists communicate with patients according to practical experience and various diagnostic criteria and provide diagnostic results. Such an automatic diagnosis model of depression requires a large number of dialogues between patients and psychiatrists. Due to the privacy protection of psychiatric medical data, it is difficult to collect a large number of dialogues between real patients and psychiatrists. And the medical dialogues in other fields cannot be simply modified and transferred to the training of the depression diagnosis model. For example, the dialogue data about physical diseases that can be crawled from the network cannot be directly migrated to mental diseases, while the dialogue data for psychological counseling lacks a certain degree of professionalism and cannot serve for diagnosis. Summary of the Invention

[0005] In order to at least solve the problems in the prior art that it is difficult to obtain depression diagnosis dialogue data between real patients and psychiatrists, and the dialogue data crawled from the network lacks professionalism and cannot be used for the training of the automatic diagnosis model of depression. In a first aspect, an embodiment of the present invention provides a method for generating a depression diagnosis dialogue dataset, including:

[0006] Based on the clinical criteria for depression diagnosis, initiate a dialogue with the real population including depression patients to generate a patient portrait of the real population;

[0007] Based on the patient portrait, simulate the depressive outpatient dialogue records between the patient role and the doctor role;

[0008] Supervise and filter the depressive outpatient dialogue records to obtain a depression diagnosis dialogue dataset that meets the clinical criteria.

[0009] In a second aspect, an electronic device is provided, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to perform the steps of the method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention.

[0010] In a third aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored, and characterized in that when the program is executed by a processor, the steps of the method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention are implemented.

[0011] The beneficial effects of the embodiments of the present invention are as follows: The depression diagnosis dialogue data set collected by using this method is of great significance for studying end-to-end depression interview dialogues and symptom-based depression diagnosis. The dialogue system also plays a very important role in large-scale screening and follow-up visits for depression, and can effectively help alleviate the problems caused by the imbalance of medical resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 is a flowchart of a method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention;

[0014] Figure 2 is a schematic diagram of three-stage data collection of a method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention;

[0015] Figure 3 is a schematic diagram of risk estimation of a patient portrait of a method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention;

[0016] Figure 4 is a schematic diagram of quality control standard data of a method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention;

[0017] Figure 5 is a data schematic diagram of a method for generating a depression diagnosis dialogue data set according to an embodiment of the present invention;

[0018] Figure 6It is a schematic diagram of the consistency between doctor's diagnosis and patient portrait in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0019] Figure 7 It is a schematic diagram of the diagnostic consistency analysis of a doctor for the same patient portrait in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0020] Figure 8 It is a schematic diagram of the statistical chart of the severity of depression in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0021] Figure 9 It is a schematic diagram of the distribution of the most common words in the symptom summaries of different degrees of depression in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0022] Figure 10 It is a schematic diagram of the topic statistics in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0023] Figure 11 It is a schematic diagram of the topic transition in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0024] Figure 12 It is a schematic diagram of the comparison with related datasets in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0025] Figure 13 It is a schematic diagram of the evaluation results of reply generation and topic prediction in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0026] Figure 14 It is a schematic diagram of the evaluation results of the dialogue box summary task in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0027] Figure 15 It is a schematic diagram of the evaluation results of severity classification in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0028] Figure 16 It is a schematic diagram of the human evaluation results of reply generation in a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention;

[0029] Figure 17 It is a schematic diagram of the structure of an embodiment of an electronic device for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention. Detailed implementation manners

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] As Figure 1 shown in the flowchart of a method for generating a depression diagnosis dialogue dataset provided by an embodiment of the present invention, the method includes the following steps:

[0032] S11: Based on the clinical criteria for depression diagnosis, initiate a dialogue with the real population including depression patients to generate a patient portrait of the real population;

[0033] S12: Based on the patient portrait, simulate the depression outpatient dialogue record between the patient role and the doctor role;

[0034] S13: Supervise and filter the depression outpatient dialogue record to obtain a depression diagnosis dialogue dataset that meets the clinical criteria.

[0035] In this embodiment, in order to imitate the real clinical consultation scenario for depression and involve three stages of collecting diagnostic dialogues, as Figure 2 shown.

[0036] For step S11, in order to simulate medical records and ensure the professionalism of clinical conversations, initiate a dialogue with the real population including depression patients from the clinical criteria for depression diagnosis. Among them, the clinical criteria for depression diagnosis include the clinical depression diagnosis criteria ICM-11 and DSM-5. Fixed questions can be extracted from these criteria to initiate a dialogue with the real population. Since these questions are fixedly extracted from the clinical depression diagnosis criteria ICM-11 and DSM-5, the generated conversations are based on the clinical depression diagnosis criteria and the scope is controllable. Such dialogue data has a certain degree of professionalism and can guide the real population to express the corresponding information of the patient portrait. And the real population includes depression patients. Since depression patients need to communicate, in the communication with depression patients, it is possible to guide depression patients to express the patient portrait information existing in a certain type of depression patients. In this interaction process, only part of the patient portrait existing in a certain type of depression patients is collected, and the interactive conversations are not collected. Therefore, the privacy of depression patients is not involved.

[0037] As an implementation manner, the step of generating a patient portrait of the real population by initiating a dialogue with the real population including depression patients based on the clinical criteria for depression diagnosis includes:

[0038] Based on the clinical criteria for depression diagnosis, a conversation is initiated with a real population including depression patients using a state machine-based consultation chatbot to generate patient portraits of the real population, where the patient portraits include: patient demographic information attributes and diagnostic attributes including at least mood, interest, and mental state.

[0039] In this embodiment, since the purpose is only to collect patient portraits and overly professional depression discrimination is not required, a state machine-based consultation chatbot can be used to initiate a conversation with a real population including depression patients, reducing the manual collection cost and obtaining patient portraits.

[0040] To overcome the impracticality of obtaining patient medical records covered by the doctor-patient confidentiality agreement, a state machine-based consultation chatbot is designed. It uses fixed questions in the clinical criteria to record the depressive symptoms and demographic information of each user, such as age, gender, marital status, and occupation. Corresponding prompts for core depressive symptoms, including mood, interest, mental state, sleep, appetite, social function, and special tendencies. Invite users to answer concisely, such as yes / no answers and severity estimates. Combined, a voluntary and legal depression portrait is obtained. As Figure 2 shown, the demographic information attributes of the patient's age, gender, marriage, and job can be collected, and the clinical attributes of the clinical criteria for depression diagnosis such as persistent low mood, low interest, difficulty concentrating, and lack of self-confidence can also be collected.

[0041] Estimate the severity of depressive episodes and suicide risk according to the clinical criteria ICD-11 and DSM-5 of each patient portrait, and the results are as Figure 3 shown. "Controllable" means no risk, and "mild", "moderate", and "severe" indicate the severity of the risk. For example, 68 patient portrait providers reported that they were diagnosed with depression in an authorized clinic. Among these providers, 53 are currently experiencing a depressive episode.

[0042] For step S12, in step S11, patient portraits of various types of depression can be collected. Using these patient portraits, various types of simulated patient roles with depression can be simulated. Through direct simulated psychological consultation conversations between the simulated patient roles and the simulated doctor roles, in this way, depression outpatient consultation records can be obtained. To ensure the quantity, quality, and professionalism of the consultation conversations.

[0043] Under the guidance of psychiatrists, a dialogue simulation was conducted based on the self-reports of real depression patients. In particular, a small number of conversations between doctors and patients in real scenarios were first collected. Based on the above prerequisites and the clinical depression diagnostic criteria ICD-11 and DSM-5, simulation tasks were issued to the simulators. The whole process is as follows: (1) Design and training: The workers were first professionally trained and then divided into the roles of doctors and patients; (2) Annotation: During the conversation, they were required to annotate the topic / symptom conversion; (3) Peer evaluation: The doctor and patient roles evaluated each other from multiple dimensions after the conversation.

[0044] As an implementation manner, the records of the depression outpatient clinic conversations between the simulated patient role and the doctor role include:

[0045] Based on the patient portrait, the first worker simulates the patient role to express the inner feelings about the event during the conversation with the simulated doctor role;

[0046] Based on the clinical criteria for depression diagnosis, the second worker simulates the doctor role to ask the patient role about events related to the clinical criteria for depression diagnosis and provide annotations, where the annotations include the conversation topics of the records of the depression outpatient clinic conversations.

[0047] In this implementation manner, for the simulated patient role, most of the patient workers do not have depression, and these workers act as actors playing patients. To help them understand the symptoms in the patient portrait, detailed explanations of the symptoms in the patient portrait are provided, including the severity and duration, as well as some self-reports of patients to help them understand the inner feelings of patients. According to the accurately expressed symptoms, they need to imagine the possible life events of the patient portrait provider by themselves and talk to the doctor to express the inner feelings of the patient during the process of telling the event.

[0048] For the simulated doctor role, psychiatrists and clinical psychotherapists were invited to conduct consultation conversations with actual depression patients to collect reference conversations. Then, according to their questions, combined with ICD-11 and DSM-5, the information that doctors need to know during diagnosis was compiled. The actors in the simulated doctor role need to obtain sufficient information during the conversation with the simulated patient role, and show sympathy and comfort when the simulated patient role confides in what they are experiencing. At the end of the conversation, they need to remind the patients who may have depression to seek medical treatment in time.

[0049] Among them, according to the core symptoms covered by clinical criteria for the annotation of dialogue topics, the topics are divided into emotions, interests, mental state, sleep, appetite, physical symptoms, social function, suicide tendency, and screening. It should be noted that empathy and comfort are listed as a special topic because it is an important part of clinical practice. Actors simulating the role of doctors are required to annotate the topics of their conversation records (emotions, interests... screening as described above) during the chat. Through the above steps, outpatient depression conversation records are obtained, ensuring the quantity, quality, and professionalism of the consultation conversations.

[0050] For step S13, since the outpatient depression conversation records obtained in step S12 are still simulated, there may be some outpatient depression conversations that do not meet the standards (non-standard, incorrect). In order to filter out outpatient depression conversations that do not meet the clinical standards, the obtained outpatient depression conversation records will be supervised and filtered.

[0051] As an implementation method, professional psychiatrists are used to supervise the outpatient depression conversation records to obtain the professional evaluations of the professional psychiatrists on the outpatient depression conversation records;

[0052] Obtain the role performance evaluations of the simulated patient role and the doctor role with respect to each other;

[0053] Filter the outpatient depression conversation records based on the professional evaluations, the role performance evaluations, and the conversation length of the outpatient depression conversation records.

[0054] The patient role evaluation includes: the naturalness of the conversation evaluated by the simulated doctor role paired with the patient role, the consistency of the narration, and the matching degree between the severity of the symptoms in the patient portrait evaluated by the professional psychiatrist and the expression;

[0055] The doctor role evaluation includes: the similarity of the doctor evaluated by the patient role paired with the doctor role and the similarity of the doctor evaluated by the medical staff.

[0056] In this implementation method, to obtain the role performance evaluations of the actors, after the outpatient depression conversation ends, the actors simulating the patient role and the doctor role will also score each other to judge whether the other party has entered their corresponding role and whether they have expressed the characteristics of a depression patient or the professional level of a doctor.

[0057] Patient role assessment: The task performance of the patient role is evaluated from three dimensions: the naturalness of expression, the consistency of narration, and the matching degree between the severity of the described symptoms and the expression (for example, whether the speech is natural, whether the narration is consistent before and after for a certain thing, and whether the patient role shows the depression situation recorded in the patient portrait). The first two scores are given by another participant in the conversation, and the third score is given by the medical professional who examines the conversation.

[0058] Doctor role assessment: The completion of the doctor's tasks is evaluated by the degree of similarity between the actor and a real doctor. This degree is evaluated by the patient role in the conversation and professional doctors respectively.

[0059] To ensure compliance with the clinical protocol, professional psychiatrists and clinical psychologists are further invited to conduct dialogue evaluations. They screen the conversations that meet the diagnostic requirements and provide psychiatric diagnosis results and symptom summaries. They score the simulated doctor role and the simulated patient role respectively with the similarity of real scenarios.

[0060] For better training purposes, multiple paradigms are used for quality inspection. Minimum limits are set for the conversation length (the conversation length of the depression outpatient conversation record, the average discourse length of the doctor in each conversation), the mutual ratings of the actors, and the ratings given by psychiatrists, as Figure 4 shown, * represents provided by the doctor, and unqualified conversations are excluded.

[0061] Finally, a total of 4428 conversations were collected through the above steps. After strict quality screening, 1339 (30%) depression diagnosis conversation datasets were finally retained. The depression automatic diagnosis model is trained using these depression diagnosis conversation datasets that meet the clinical standards for automatic depression diagnosis. As more and more depression outpatient conversations that ensure quality and professionalism are generated, the trained depression automatic diagnosis model will become more and more accurate.

[0062] It can be seen from this implementation method that the depression diagnosis conversation dataset collected by this method is of great significance for studying end-to-end depression interview conversations and symptom-based depression diagnosis. The dialogue system also plays a very important role in the large-scale screening and follow-up visits of depression, and can effectively help alleviate the problems caused by the imbalance of medical resources.

[0063] As an implementation method, in this embodiment, the depression outpatient conversation record between the simulated patient role and the doctor role further includes:

[0064] Using the simulated patient robot trained based on the patient portrait;

[0065] Using the simulated doctor robot trained by collecting the conversations of professional psychiatrists;

[0066] Generate a depression outpatient consultation dialogue record based on the interaction between the simulated patient robot and the simulated doctor robot, as well as mutual role performance evaluations.

[0067] Use the simulated doctor robot to conduct a simulated doctor evaluation of the depression outpatient consultation dialogue record according to the clinical criteria for depression diagnosis;

[0068] Filter the depression outpatient consultation dialogue record through role performance evaluations, simulated doctor evaluations, and the dialogue length of the depression outpatient consultation dialogue record.

[0069] In this embodiment, considering the progress of artificial intelligence, neural networks with different symptoms for different patients are constructed using each person's patient profile, and then extended and augmented using a knowledge graph to generate multiple simulated patient robots with the same depressive symptoms but different personalities.

[0070] For example, working in a securities company, through the knowledge graph, personalities with stronger career ambitions and stronger communication skills can be associated. In this way, one patient profile can simulate multiple simulated patient robots with the same depressive symptoms but different personalities. On the side of the simulated patient role, richer conversations can also be generated.

[0071] For the simulated doctor robot, in the conversations between professional psychiatrists and depression patients, each sentence has a certain intention, and certain questions are also supported by professional medical theories and diagnoses for those questions. The simulated doctor robot is trained by collecting the conversations of these professional psychiatrists. In this way, on the side of the simulated doctor role, questions with certain depression characteristics can be issued, and appropriate diagnostic results can be given for depressive symptoms.

[0072] Since the depressive outpatient dialogue records in this part are also simulated and generated, they also need to be supervised and filtered. At this time, since both are trained robots, the evaluation of the role performance is whether the dialogue between the two parties is smooth and logical. For example, after the simulated doctor robot asks a question, in the content feedback by the simulated patient robot, if the simulated doctor robot cannot extract the answer to the question from the feedback content, from the perspective of the simulated doctor robot, the evaluation of the simulated patient robot is poor. On the contrary, if the content proposed by the simulated patient robot is such that the simulated doctor robot does not ask questions or provide answers based on the relevant content (that is, the keywords of the content proposed by the simulated patient robot cannot be extracted from the feedback content of the simulated doctor robot), then the simulated patient robot and the simulated doctor robot can evaluate each other. Then, use the trained simulated doctor robot to re-judge the entire dialogue and make a comprehensive judgment based on the dialogue length to obtain a filtered dataset of depressive disorder diagnosis dialogues. With the continuous collection of various data and the continuous training of the simulated doctor robot and the simulated patient robot, more depressive disorder diagnosis dialogue datasets that meet clinical standards can be obtained.

[0073] An experiment is conducted on this method, and the overall statistical data of the dataset are as Figure 5 shown. As can be seen in such a diagnostic scenario, a sufficient number of dialogue turns are required: the diagnostic dialogue shows an average of 21.6 turns per dialogue. The average symptom summary provided by psychiatrists reaches 83.1 words. These statistics are significantly longer than previous related datasets, indicating the differences in data requirements for the diagnostic dialogue task.

[0074] It should be noted that the simulated symptoms in the dialogue data are based on the portraits of real people. To verify that the simulated dialogue truly reflects the symptoms in the patient portrait and to draw valid diagnostic conclusions from such dialogues, the consistency between the patient portrait and the corresponding content of the psychiatrist's symptom summary, as well as the diagnostic results of different doctors for the same portrait, is analyzed.

[0075] Portrait diagnosis consistency. The patient portrait contains depressive symptoms, and the actor playing the role of the simulated patient added more details in the actual description on this basis, resulting in a more comprehensive diagnostic summary. Therefore, the hit rate of the doctor's diagnostic summary (as Figure 6 shown) is used to measure the consistency. It can be seen that the accuracy of most diagnoses is very high, with an average of 86.1%, proving the authenticity of the patient imitation and the comprehensiveness of the summary. In addition, psychiatrists were also asked to rate the matching degree between the patient's expression and the symptom severity, with an average score of 3.9 out of 5, that is, the degree of compliance reached 78%.

[0076] Doctor's consistency. The diagnostic results of the same patient portrait from different doctors may vary slightly. For Figure 7The three indicators shown calculate the average values derived from different portrait standards (excluding portraits diagnosed individually), indicating a relatively high consistency in the diagnostic results and being less affected by the subjectivity of workers.

[0077] As shown in Figure 8 statistics of patients with depression of different severities are presented. As the severity of depression increases, the number of turns and the length of the conversation become longer due to doctors' more in-depth questioning on specific topics. The content in the diagnostic summary becomes longer to list more depressive symptoms.

[0078] The most common topics also change with the severity: "suicidal tendency" is more likely to be questioned in more severe patients.

[0079] As shown in Figure 9 significant differences in the hot words in the diagnostic summaries of different severities are observed. As shown in (a), patients mostly have superficial symptoms such as difficulty in decision-making and decreased confidence, which are usually present in the healthy population. As the condition worsens, more obvious symptoms such as difficulty in sleeping occur frequently, and doctors will advise patients to take timely measures. In the most severe patients, the risk of suicide and despair become frequent in chart (d).

[0080] The characteristic statistics of different topics are shown in Figure 10 as follows. Core depressive symptoms account for 68.3% of the conversation, followed by empathy and comfort, accounting for 23.1%. By analyzing the topics that appear for the first time, it can be seen that mild symptoms such as mood and interest are usually asked at the beginning, and gradually turn to suicide and somatic symptoms, which are usually experienced by severe patients. This echoes clinical practice, and the consultation follows a progressive and in-depth manner.

[0081] Figure 11 illustrates the process of topic conversion in the conversation. Different from other common topics that rarely span more than one turn of conversation, diagnostic topics always appear across turns. Due to the particularity of the depression diagnosis scenario, the ratio of empathy and comfort remains stable at all stages. Due to its sensitivity, the risk of suicide is usually addressed at a later stage, where core represents core, empathy represents empathy, suicide represents the risk of suicide, behavior represents habit, and screening represents screening.

[0082] The relevant datasets are introduced and compared with the proposed diagnostic datasets, as shown in Figure 12 as follows. Depression diagnosis requires precise collection of symptom information through more turns of conversation, a longer time, and sufficient emotional support.

[0083] Task-oriented dialogue datasets are one of the most important components in dialogue system research and consist of various datasets for this purpose, namely MultiWOZ, MSR-E2E, CamRest, Frames (the above four are all existing datasets). However, these dialogue datasets target common scenarios in life. Therefore, there are few dialogue turns. Moreover, little attention is paid to the emotions of users, sympathizing with or comforting them during the dialogue.

[0084] Psychological counseling datasets and dialogue research related to mental health have addressed emotions during the dialogue and tried to motivate users. Networks like ESConv have started to focus on emotional support dialogue systems. However, they mainly concern providing encouragement and advice to patients rather than providing professional diagnostic advice.

[0085] There are some medical dialogue datasets for diagnosis, such as MedDG and MedDialog. However, these mainly focus on somatic symptoms and somatic diseases. Although MedDialog has a small amount of psychiatric data, it lacks professional annotation and cannot be used for the depression diagnosis dialogue system. In addition, the diagnosis process of depression is very different from that of somatic diseases. According to ICD-11 (World Health Organization), in addition to somatic symptoms, patients often have symptoms in multiple dimensions such as emotions, interests, mental state, and social dysfunction. For this reason, psychiatrists need comprehensive information to provide an accurate diagnosis, so the dialogue will belong to and contain multiple knowledge domains.

[0086] Some dialogue datasets are closely related to depression, such as the multimodal dataset DAIC-WOZ. This dataset consists of face-to-face counseling dialogues between interviewers and patients with depression, anxiety, etc., and researchers use these dialogues to diagnose depression. However, this dataset only has 189 dialogues, which is not enough for the dialogue generation task.

[0087] To experiment and compare several models, Transformer is used for reply generation and topic prediction experiments in the sequence-to-sequence model. The implementation used is HuggingFace. These parameters are loaded from the transformer pre-trained on the Chinese medical dialogue dataset MedDialog. BART is a denoising sequence-to-sequence pre-trained model and is an entry-level model for text generation and summarization tasks. For this reason, the Bart dataset trained on the Chinese dataset is used to perform reply generation and dialogue summarization tasks.

[0088] CPT is a novel Chinese pre-trained unbalanced transformer model, which is not only effective in the generation task but also has strong classification ability. Therefore, it is selected as the backbone model for the generation task, and its classification task performance is compared with BART.

[0089] BERT is effectively used in a wide range of language understanding tasks, such as question answering and language inference. Therefore, version 2 pre-trained on Chinese datasets is used to perform classification tasks.

[0090] Objective metrics are used to evaluate response generation tasks and dialogue summarization tasks, including BLEU2, Rouge-L, METEOR to measure the similarity between the responses generated by the model and the labels. To show the diversity of generations, DIST-2 is calculated. Jieba3 is implemented for tokenization and word-level metrics are calculated.

[0091] The results of the response generation task are as Figure 13 shown. Three observations can be made:

[0092] (1) BART and CPT show similar generation performance on the datasets generated by this method;

[0093] (2) Most of the outputs of the two models execute Transformer, which is pre-trained on medical corpora, indicating that depression diagnosis is different from traditional body-oriented medical conversations;

[0094] (3) Given the gold topic, BART* can further improve the generation performance.

[0095] The results of the topic prediction accuracy are shown as topics. A similar trend is observed in Figure 13 : BART≈CPT>Transformer. Since the ten topics are classified according to core symptoms and emotional support, the uncertainty and language ambiguity of depression diagnosis conversations undoubtedly increase the prediction difficulty.

[0096] The results of the dialogue summarization are listed in Figure 14 . In terms of the overlap of N-grams with human references, CPT is comparable to BART. Nevertheless, CPT shows a higher DIST-2 score, indicating its advantage in generation diversity.

[0097] Severity classification binary and 4-level classification are evaluated by average weighted precision, recall, and F1, as Figure 15 shown.

[0098] To better evaluate the performance of the responses generated by the model, 15 staff members were hired to score the responses generated by BART and CPT and the real responses of doctors respectively. 100 responses from different topics were randomly selected for each model, and 3 workers were asked to evaluate the same responses from 4 aspects: Fluency measures the fluency of the generated sentences; Rationality measures the reasonableness of giving this response based on the conversation history; Similarity to doctors measures the similarity of the response to the words of doctors; Comfort measures the comfort level of the response. The evaluation results are as Figure 16 shown. Generally, manual evaluation is based on objective measures: CPT and BART showed similar performance, but both lagged behind the ground truth. Regarding the detailed manual evaluation metrics, both models can generate fluent responses. However, in order to produce reasonable, reassuring, and doctor-like responses.

[0099] Overall, this method constructs a depression diagnosis dataset that meets clinical standards, which contains 1339 conversations and the diagnostic summaries of psychiatrists. In addition, the state-of-the-art model is used to conduct experimental verification on multiple tasks, and the results are compared with objective and manual evaluations. The model can generate fluent and human-like responses.

[0100] The embodiment of the present invention also provides a non-volatile computer storage medium, and the computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute the method for generating a depression diagnosis dialogue dataset in the above embodiment;

[0101] As an implementation manner, the non-volatile computer storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are set as:

[0102] Based on the clinical standards for depression diagnosis, initiate a conversation with the real population including depression patients to generate a patient portrait of the real population;

[0103] Based on the patient portrait, simulate the depression outpatient dialogue records between the patient role and the doctor role;

[0104] Supervise and filter the depression outpatient dialogue records to obtain a depression diagnosis dialogue dataset that meets clinical standards.

[0105] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the method in the embodiment of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, execute the method for generating a depression diagnosis dialogue dataset in the above embodiment.

[0106] Figure 17FIG. 0 is a schematic hardware structure diagram of an electronic device for a method of generating a depression diagnosis dialogue dataset provided by another embodiment of the present application, as Figure 17 shown. The device includes:

[0107] One or more processors 1710 and a memory 1720, Figure 17 Taking one processor 1710 as an example. The device for the method of generating a depression diagnosis dialogue dataset may further include: an input device 1730 and an output device 1740.

[0108] The processor 1710, the memory 1720, the input device 1730, and the output device 1740 may be connected through a bus or other means, Figure 17 Taking connection through a bus as an example.

[0109] The memory 1720, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions / modules corresponding to the method of generating a depression diagnosis dialogue dataset in the embodiments of the present application. The processor 1710 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 1720, that is, implements the method of generating a depression diagnosis dialogue dataset in the above method embodiments.

[0110] The memory 1720 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data, etc. In addition, the memory 1720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 1720 may optionally include a memory remotely set relative to the processor 1710, and these remote memories may be connected to the mobile device through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0111] The input device 1730 can receive input digital or character information. The output device 1740 may include a display device such as a display screen.

[0112] The one or more modules are stored in the memory 1720, and when executed by the one or more processors 1710, execute the method of generating a depression diagnosis dialogue dataset in any of the above method embodiments.

[0113] The above product can execute the method provided by the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be referred to the method provided by the embodiments of the present application.

[0114] A non-volatile computer-readable storage medium may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the non-volatile computer-readable storage medium may optionally include a memory remotely provided with respect to the processor, and these remote memories can be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0115] Embodiments of the present invention also provide an electronic device, which includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the method for generating a depression diagnosis dialogue data set according to any embodiment of the present invention.

[0116] The electronic device according to the embodiments of the present application exists in various forms, including but not limited to:

[0117] (1) Mobile communication devices: The characteristics of such devices are that they have mobile communication functions and mainly aim to provide voice and data communication. Such terminals include: smart phones, multimedia phones, functional phones, and low-end phones, etc.

[0118] (2) Ultra-mobile personal computer devices: Such devices belong to the category of personal computers, have computing and processing functions, and generally also have the characteristic of mobile Internet access. Such terminals include: PDA, MID, and UMPC devices, etc., such as tablet computers.

[0119] (3) Portable entertainment devices: Such devices can display and play multimedia content. Such devices include: audio and video players, handheld game consoles, e-books, and smart toys and portable vehicle navigation devices.

[0120] (4) Other electronic devices with data processing functions.

[0121] In this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising" and "including" not only include those elements, but also other elements not explicitly listed, or elements inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the said element.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0123] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A method for generating a depression diagnosis dialogue dataset, comprising: Based on the clinical criteria for depression diagnosis, initiating a dialogue with a real population including depression patients to generate a patient portrait of the real population; Based on the patient portrait, simulating the depressive outpatient dialogue records between the patient role and the doctor role, including, based on the patient portrait, using a first worker to simulate the patient role to express the inner feelings about events with a simulated doctor role, and based on the clinical criteria for depression diagnosis, using a second worker to simulate the doctor role to ask the patient role about events related to the clinical criteria for depression diagnosis and provide annotations, wherein the annotations include the dialogue topics of the depressive outpatient dialogue records; Supervising and filtering the depressive outpatient dialogue records, including using professional psychiatrists to supervise the depressive outpatient dialogue records, obtaining the professional evaluations of the professional psychiatrists on the depressive outpatient dialogue records, and obtaining the mutual role performance evaluations of the simulated patient role and the doctor role, wherein the role performance evaluations include: patient role evaluation, doctor role evaluation, the patient role evaluation includes: the naturalness of the dialogue evaluated by the simulated doctor role paired with the patient role, the consistency of the narration, and the matching degree between the severity of the symptoms in the patient portrait and the expression evaluated by the professional psychiatrist, the doctor role evaluation includes: the doctor similarity evaluated by the patient role paired with the doctor role and the doctor similarity evaluated by medical staff; Filtering the depressive outpatient dialogue records based on the professional evaluations, the role performance evaluations, and the dialogue length of the depressive outpatient dialogue records to obtain a depression diagnosis dialogue dataset that meets the clinical criteria.

2. The method according to claim 1, wherein The step of, based on the clinical criteria for depression diagnosis, initiating a dialogue with a real population including depression patients to generate a patient portrait of the real population includes: Based on the clinical criteria for depression diagnosis, using a state machine-based consultation chatbot to initiate a dialogue with a real population including depression patients to generate a patient portrait of the real population, wherein the patient portrait includes: patient demographic information attributes and diagnostic attributes including at least emotions, interests, and mental states.

3. A method for generating a depression diagnosis dialogue dataset, comprising: Based on the clinical criteria for depression diagnosis, initiating a dialogue with a real population including depression patients to generate a patient portrait of the real population; Based on the patient portrait, simulating the depressive outpatient dialogue records between the patient role and the doctor role, including using a simulated patient robot trained based on the patient portrait, using a simulated doctor robot trained by collecting the dialogues of professional psychiatrists, generating depressive outpatient dialogue records based on the interaction between the simulated patient robot and the simulated doctor robot, and mutual role performance evaluations, and using the simulated doctor robot to conduct a simulated doctor evaluation of the depressive outpatient dialogue records through the clinical criteria for depression diagnosis; Filter the depression outpatient consultation record through role performance evaluation, simulated doctor evaluation, and the dialogue length of the depression outpatient consultation record to obtain a dataset of depression diagnosis dialogues that meet clinical criteria.

4. The method according to claim 1, wherein The method further includes: Training an automatic diagnosis model for depression using the dataset of depression diagnosis dialogues that meet clinical criteria.

5. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-4.

6. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images